Investigation of pulmonary nodule classification using multi-scale residual network enhanced with 3DGAN-synthesized volumes

It is often difficult to distinguish between benign and malignant pulmonary nodules using only image diagnosis. A biopsy is performed when malignancy is suspected based on CT examination. However, biopsies are highly invasive, and patients with benign nodules may undergo unnecessary procedures. In this study, we performed automated classification of pulmonary nodules using a three-dimensional convolutional neural network (3DCNN). In addition, to increase the number of training data, we utilized generative adversarial networks (GANs), a deep learning technique used as a data augmentation method. In this approach, three-dimensional regions of different sizes centered on pulmonary nodules were extracted from CT images, and a large number of pseudo-pulmonary nodules were synthesized using 3DGAN. The 3DCNN has a multi-scale structure in which multiple nodules in each region are inputted and integrated into the final layer. During the training of multi-scale 3DCNN, pre-training was first performed using 3DGAN-synthesized nodules, and the pulmonary nodules were then comprehensively classified by fine-tuning the pre-trained model using real nodules. Using an evaluation process that involved 60 confirmed cases of pathological diagnosis based on biopsies, the sensitivity was determined to be 90.9% and specificity was 74.1%. The classification accuracy was improved compared to the case of training with only real nodules without pre-training. The 2DCNN results of our previous study were slightly better than the 3DCNN results. However, it was shown that even though 3DCNN is difficult to train with limited data such as in the case of medical images, classification accuracy can be improved by GAN.