Skip to main content

Advertisement

Log in

A hierarchical GAN method with ensemble CNN for accurate nodule detection

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Lung cancer can evolve into one of the deadliest diseases whose early detection is one of the major survival factors. However, early detection is a challenging task due to the unclear structure, shape, and the size of the nodule. Hence, radiologists need automated tools to make accurate decisions.

Methods

This paper develops a new approach based on generative adversarial network (GAN) architecture for nodule detection to propose a two-step GAN model containing lung segmentation and nodule localization. The first generator comprises a U-net network, while the second utilizes a mask R-CNN. The task of lung segmentation involves a two-class classification of the pixels in each image, categorizing lung pixels in one class and the rest in the other. The classifier becomes imbalanced due to numerous non-lung pixels, decreasing the model performance. This problem is resolved by using the focal loss function for training the generator. Moreover, a new loss function is developed as the nodule localization generator to enhance the diagnosis quality. Discriminator nets are implemented in GANs as an ensemble of convolutional neural networks (ECNNs), using multiple CNNs and connecting their outputs to make a final decision.

Results

Several experiments are designed to assess the model on the well-known LUNA dataset. The experiments indicate that the proposed model can reduce the error of the state-of-the-art models on the IoU criterion by about 35 and 16% for lung segmentation and nodule localization, respectively.

Conclusion

Unlike recent studies, the proposed method considers two loss functions for generators, further promoting the goal achievements. Moreover, the network of discriminators is regarded as ECNNs, generating rich features for decisions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Ferlay J et al Global Cancer Observatory: Cancer Today. Lyon: International Agency for Research on Cancer; 2020 (https://gco.iarc.fr/today.) Accessed February 2021.

  2. Coppola F et al (2021) Human all too human? an all-around appraisal of the “artificial intelligence revolution” in medical imaging. Front Psychol. 12:710982

    Article  PubMed  PubMed Central  Google Scholar 

  3. Jan E et al (2022) Medical deep learning—a systematic meta-review. Computer Methods Program Biomed 221:106874

    Article  Google Scholar 

  4. Moravvej SV, Mousavirad SJ, Moghadam MH, and Saadatmand M (2021) An LSTM-Based plagiarism detection via attention mechanism and a population-based approach for pre-training parameters with imbalanced classes. In International Conference on Neural Information Processing. Springer

  5. Moravvej SV, Joodaki M, Kahaki MJM, and Sartakhti MS (2021) A method Based on an attention mechanism to measure the similarity of two sentences. In 2021 7th International Conference on Web Research (ICWR). IEEE

  6. Moravvej SV, Kahaki MJM, Sartakhti MS, and Mirzaei A (2021) A method based on attention mechanism using bidirectional long-short term memory (BLSTM) for question answering. In 2021 29th Iranian Conference on Electrical Engineering (ICEE). IEEE.

  7. Sartakhti MS, Kahaki MJM, Moravvej SV, Javadi Joortani M, and Bagheri A (2021) Persian language model based on BiLSTM model on COVID-19 Corpus. In 2021 5th International Conference on Pattern Recognition and Image Analysis (IPRIA). IEEE.

  8. Sobhaninia Z, Karimi N, Khadivi P, and Samavi S (2021) Brain tumor classification by cascaded multiscale multitask learning framework based on feature aggregation. http://arxiv.org/abs/math/2112.14320

  9. Kallenberg M, Petersen K, Nielsen M, Ng AY, Diao P, Igel C, Vachon CM, Holland K, Winkel RR, Karssemeijer N (2016) Unsupervised deep learning applied to breast density segmentation and mammographic risk scoring. IEEE Trans Med Imag 35(5):1322–1331

    Article  Google Scholar 

  10. Shin H-C, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imag 35(5):1285–1298

    Article  Google Scholar 

  11. Long J, Shelhamer E, and Darrell T (2015) Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  12. Wang S, Zhou M, Gevaert O, Tang Z, Dong D, Liu Z, and Jie T (2017) A multi-view deep convolutional neural networks for lung nodule segmentation. In 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE.

  13. Mukherjee S, Huang X, and Bhagalia RR (2017) Lung nodule segmentation using deep learned prior based graph cut. In 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017). IEEE.

  14. Zhu W, Vang YS, Huang Y, and Xie X (2018) Deepem: Deep 3d convnets with em for weakly supervised pulmonary nodule detection. in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer.

  15. Xie Z (2017) 3D region proposal u-net with dense and residual learning for lung nodule detection. LUNA16

  16. Moravvej SV, Mirzaei A, and Safayani M (2021) Biomedical text summarization using Conditional Generative Adversarial Network (CGAN). http://arxiv.org/abs/math/2110.11870

  17. Moravvej S, Maleki Kahaki M, Salimi Sartakhti M, Joodaki M (2021) Efficient GAN-based method for extractive summarization. J Electr Computer Eng Innov (JECEI) 10:287

    Google Scholar 

  18. Shin H-C, Tenenholtz NA, Rogers JK, Schwarz CG, Senjem ML, Gunter JL, Andriole KP, and Michalski M (2018) Medical image synthesis for data augmentation and anonymization using generative adversarial networks. In International Workshop on Simulation and Synthesis in Medical Imaging. Springer

  19. Tan J, Jing L, Huo Y, Li L, Akin O, Tian Y (2021) Lgan: lung segmentation in ct scans using generative adversarial network. Comput Med Imag Graph 87:101817

    Article  Google Scholar 

  20. Huang X, Sun W, Tseng T-LB, Li C, Qian W (2019) Fast and fully-automated detection and segmentation of pulmonary nodules in thoracic CT scans using deep convolutional neural networks. Comput Med Imag Graph 74:25–36

    Article  Google Scholar 

  21. Armato SG, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP, Zhao B, Aberle DR, Henschke CI, Hoffman EA (2011) The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med Phys 38(2):915–931

    Article  PubMed  PubMed Central  Google Scholar 

  22. Rezatofighi, H, Tsoi N, Gwak J, Sadeghian A, Reid I, and Savarese S (2019) Generalized intersection over union: A metric and a loss for bounding box regression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

  23. Rockafellar RT, Wets RJ-B (2009) Variational analysis. Springer Science & Business Media

    Google Scholar 

  24. Kalinovsky A and Kovalev V (2016) Lung image Segmentation using deep learning methods and convolutional neural networks. In: Pattern recognition and information processing. Publishing Center of BSU, Minsk

  25. Neff T, Payer C, Stern D, and Urschler M (2017) Generative adversarial network based synthesis for supervised medical image segmentation. in Proceedings OAGM and ARW Joint Workshop

  26. Ronneberger O, Fischer P, and Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer

  27. Jégou S, Drozdzal M, Vazquez D, Romero A, and Bengio Y (2017) The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops

  28. Mansoor A, Bagci U, Foster B, Xu Z, Papadakis GZ, Folio LR, Udupa JK, Mollura DJ (2015) Segmentation and image analysis of abnormal lungs at CT: current approaches, challenges, and future trends. Radiographics 35(4):1056–1076

    Article  PubMed  Google Scholar 

  29. Pihur V, Datta S, Datta S (2007) Weighted rank aggregation of cluster validation measures: a monte carlo cross-entropy approach. Bioinformatics 23(13):1607–1615

    Article  CAS  PubMed  Google Scholar 

  30. Xie S and Tu Z (2015) Holistically-nested edge detection. In Proceedings of the IEEE International Conference on Computer Vision

  31. Sudre CH, Li W, Vercauteren T, Ourselin S, Cardoso MJ (2017) Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations deep learning in medical image analysis and multimodal learning for clinical decision support. Springer

    Google Scholar 

  32. Salehi SSM, Erdogmus D, Gholipour A (2017) Tversky loss function for image segmentation using 3D fully convolutional deep networks. In International Workshop on Machine Learning in Medical Imaging. Springer

  33. Hashemi SR, Salehi SSM, Erdogmus D, Prabhu SP, Warfield SK, Gholipour A (2018) Asymmetric loss functions and deep densely-connected networks for highly-imbalanced medical image segmentation: application to multiple sclerosis lesion detection. IEEE Access 7:1721–1735

    Article  Google Scholar 

  34. Huang X, Shan, J and Vaidya V (2017) Lung nodule detection in CT using 3D convolutional neural networks. In 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017). IEEE

  35. Wu B, Zhou Z, Wang J, and Wang Y (2018) Joint learning for pulmonary nodule segmentation, attributes and malignancy prediction. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). IEEE

  36. Maqsood M, Yasmin S, Mehmood I, Bukhari M, Kim M (2021) An efficient DA-net architecture for lung nodule segmentation. Mathematics 9(13):1457

    Article  Google Scholar 

  37. Shen W, Zhou M, Yang F, Yu D, Dong D, Yang C, Zang Y, Tian J (2017) Multi-crop convolutional neural networks for lung nodule malignancy suspiciousness classification. Pattern Recogn 61:663–673

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Abbas Ahmadi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All studies involving postmortem subjects followed the procedures for informed consent as described in the Declaration of Helsinki.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rezaei, S.R., Ahmadi, A. A hierarchical GAN method with ensemble CNN for accurate nodule detection. Int J CARS 18, 695–705 (2023). https://doi.org/10.1007/s11548-022-02807-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-022-02807-9

Keywords