Abstract
Automated benign and malignant breast masses classification is a crucial yet challenging topic. Recently, many studies based on convolutional neural network (CNN) are presented to address this task, but most of these CNN-based methods neglect the effective global contextual information. Moreover, their methods do not further analyze the reliability and interpretability of CNN models, which does not correspond to the clinical diagnosis. In this work, we firstly propose a novel multi-level global-guided branch-attention network (MGBN) for mass classification, which aims to fully leverage the multi-level global contextual information to refine the feature representation. Specifically, the MGBN includes a stem module and a branch module. The former extracts the local information through standard local convolutional operations of ResNet-50. The latter embeds the global contextual information and establishes the relationships of different feature levels via global pooling and Multi-layer Perceptron (MLP). The final prediction is computed by local information and global information together. Then, we discuss the reliability and interpretability of our mass classification network by visualizing the coarse localization map through Gradient-weighted Class Activation Mapping (Grad-CAM), which is important in clinical diagnosis. Finally, our proposed MGBN is greatly demonstrated on two public mammographic mass classification databases including the DDSM and INbreast databases, resulting in AUC of 0.8375 and 0.9311, respectively.
Similar content being viewed by others
References
Amrane M, Oukid S, Gagaoua I, Ensarİ T (2018) Breast cancer classification using machine learning. In: 2018 Electric Electronics, Computer Science, Biomedical Engineerings’ Meeting (EBBT). IEEE, pp 1–4
Angra S, Ahuja S (2017) Machine learning and its applications: A review. In: 2017 International Conference on Big Data Analytics and Computational Intelligence (ICBDAC), pp 57–60. IEEE
Carneiro G, Nascimento J, Bradley AP (2017) Automated analysis of unregistered multi-view mammograms with deep learning. IEEE Trans Med Imaging 36(11):2355–2365
Cheng H-D, Shi X J, Min R, Hu L M, Cai X P, Du H N (2006) Approaches for automated detection and classification of masses in mammograms. Pattern Recogn 39(4):646–668
Chowdhary CL, Mittal M, Pattanaik PA, Marszalek Z et al (2020) An efficient segmentation and classification system in medical images using intuitionist possibilistic fuzzy c-mean clustering and fuzzy svm algorithm. Sensors 20 (14):3903
DeSantis CE, Ma J, Gaudet MM, Newman LA, Miller KD, Sauer AG, Jemal A, Siegel Rebecca L (2019) Breast cancer statistics, 2019. CA: A Cancer J Clin 69(6):438–451
Deng J, Dong W, Socher R, Li L-J, Li K, Fei LF (2009) Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, pp 248–255. IEEE
Dhungel N, Carneiro G, Bradley AP (2016) The automated learning of deep features for breast mass classification from mammograms. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, pp 106–114
Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H (2019) Dual attention network for scene segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
Fu Y, Xue P, Ji H, Cui W, Dong E (2020) Deep model with siamese network for viable and necrotic tumor regions assessment in osteosarcoma. Med Phys 47(10):4895–4905
Fukui H, Hirakawa T, Yamashita T, Fujiyoshi H (2019) Attention branch network: Learning of attention mechanism for visual explanation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 10705–10714
Giger ML, Karssemeijer N, Schnabel JA (2013) Breast image analysis for risk assessment, detection, diagnosis, and treatment of cancer. Ann Rev Biomed Eng 15:327–357
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Heidari M, Mirniaharikandehei S, Liu W, Hollingsworth AB, Liu H, Zheng B (2019) Development and assessment of a new global mammographic image feature analysis scheme to predict likelihood of malignant cases. IEEE Trans Med Imaging 39(4):1235–1244
Henriksen EL, Carlsen JF, Vejborg IMM, Nielsen MB, Lauridsen CA (2019) The efficacy of using computer-aided detection (cad) for detection of breast cancer in mammography screening: a systematic review. Acta Radiol 60(1):13–18
Hu J, Li S, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708
Huang Q, Zhang F, Li X (2018) Machine learning in ultrasound computer-aided diagnostic systems: a survey. BioMed Research International 2018
Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning
Itti L, Koch C (2001) Computational modelling of visual attention. Nat Rev Neurosci 2(3):194–203
Izonin I, Tkachenko R, Kryvinska N, Tkachenko P et al (2019) Multiple linear regression based on coefficients identification using non-iterative sgtm neural-like structure. In: International Work-Conference on Artificial Neural Networks. Springer, pp 467–479
Khan HN, Shahid AR, Raza B, Dar AH, Alquhayz H (2019) Multi-view feature fusion based four views model for mammogram classification using convolutional neural network. IEEE Access 7:165724–165733
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
Larochelle H, Hinton GE (2010) Learning to combine foveal glimpses with a third-order boltzmann machine. In: Advances in neural information processing systems, pp 1243–1251
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444
Lee RS, Gimenez F, Hoogi A, Miyake KK, Gorovoy M, Rubin DL (2017) A curated mammography data set for use in computer-aided detection and diagnosis research. Sci Data 4:170177
Lehman CD, Arao RF, Sprague BL, Lee JM, Buist DSM, Kerlikowske K, Henderson LM, Onega T, Tosteson ANA, Rauscher GH et al (2017) National performance benchmarks for modern screening digital mammography: update from the breast cancer surveillance consortium. Radiology 283(1):49–58
Li X, Wang W, Hu X, Yang J (2019) Selective kernel networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 510–519
Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, Van Der Laak JA, Van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88
Mnih V, Heess N, Graves A et al (2014) Recurrent models of visual attention. In: Advances in neural information processing systems, pp 2204–2212
Monshi MMA, Poon J, Chung V (2020) Deep learning in generating radiology reports A survey. Artificial Intelligence in Medicine, pp 101878
Moreira IC, Amaral I, Domingues I, Cardoso A, Cardoso MJ, Cardoso JS (2012) Inbreast: toward a full-field digital mammographic database. Acad Radiol 19(2):236–248
Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. In: International Conference on Machine Learning
Newell A, Yang K, Deng J (2016) Stacked hourglass networks for human pose estimation. In: European conference on computer vision. Springer, pp 483–499
Oliver A, Freixenet J, Marti J, Perez E, Pont J, Denton ERE, Zwiggelaar R (2010) A review of automatic mass detection and segmentation in mammographic images. Med Image Anal 14(2):87–110
Pang T, Wong JHD, Ng WL, Chan CS (2020) Deep learning radiomics in breast cancer with different modalities Overview and future. Expert Systems with Applications, pp 113501
Park J, Woo S, Lee J-Y, Kweon IS (2018) Bam: Bottleneck attention module. In: The British Machine Vision Conference
Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L et al (2019) Pytorch: an imperative style, high-performance deep learning library. In: Advances in neural information processing systems, pp 8026–8037
Perek S, Ness L, Amit M, Barkan E, Amit G (2019) Learning from longitudinal mammography studies. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, pp 712–720
Qi Y, Yang Z, Lei J, Lian J, Liu J, Feng W, Ma Y (2020) Morph_spcnn model and its application in breast density segmentation. Multimedia Tools and Applications
Rampun A, Scotney BW, Morrow PJ, Wang H (2018) Breast mass classification in mammograms using ensemble convolutional neural networks. In: IEEE 20th International Conference on e-Health Networking, Applications and Services (Healthcom). IEEE, pp 1–6
Rodríguez-Ruiz A, Krupinski E, Mordang J-J, Schilling K, Heywang-Köbrunner SH, Sechopoulos I, Mann RM (2019) Detection of breast cancer with mammography: effect of an artificial intelligence support system. Radiology 290(2):305–314
Rouhi R, Jafari M, Kasaei S, Keshavarzian P (2015) Benign and malignant breast tumors classification based on region growing and cnn segmentation. Expert Syst Appl 42(3):990–1002
Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision
Shams S, Platania R, Zhang J, Kim J, Lee K, Park S-J (2018) Deep generative breast cancer screening and diagnosis. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, pp 859–867
Siegel RL, Miller KD, Jemal A (2019) Cancer statistics, 2019. CA: A Cancer J Clin 69(1):7–34
Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations
Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-first AAAI conference on artificial intelligence
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826
Talo M (2019) Automated classification of histopathology images using transfer learning. Artif Intell Med 101:101743
Tkachenko R, Doroshenko A, Izonin I, Tsymbal Y, Havrysh B (2018) Imbalance data classification via neural-like structures of geometric transformations model: Local and global approaches. In: International conference on computer science, engineering and education applications, pages 112–122. Springer
Tkachenko R, Izonin I (2018) Model and principles for the implementation of neural-like structures based on geometric data transformations. In: International Conference on Computer Science, Engineering and Education Applications. Springer, pages 578–587
Wang N, Bian C, Wang Y, Xu M, Qin C, Yang X, Wang T, Li A, Shen D, Ni D (2018) Densely deep supervised networks with threshold loss for cancer detection in automated breast ultrasound. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, pp 641–648
Wang H, Feng J, Zhang Z, Su H, Cui L, He H, Li L (2018) Breast mass classification via deeply integrating the contextual information from multi-view data. Pattern Recogn 80:42–52
Wang F, Jiang M, Qian C, Yang S, Li C, Zhang H, Wang X, Tang X (2017) Residual attention network for image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3156–3164
Wang R, Ma Y, Sun W, Guo Y, Wang W, Qi Y, Gong X (2019) Multi-level nested pyramid network for mass segmentation in mammograms. Neurocomputing 363:313–320
Waring J, Lindvall C, Umeton R (2020) Automated machine learning: Review of the state-of-the-art and opportunities for healthcare. Artif Intell Med:101822
Wei C-H, Chen SY, Liu X (2012) Mammogram retrieval on similar mass lesions. Comput Methods Programs Biomed 106(3):234–248
Woo S, Park J, Lee J-Y, Kweon IS (2018) Cbam Convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19
Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1492–1500
Xie W, Li Y, Ma Y (2016) Breast mass classification in digital mammography based on extreme learning machine. Neurocomputing 173:930–941
Yala A, Lehman C, Schuster T, Portnoi T, Barzilay R (2019) A deep learning mammography-based model for improved breast cancer risk prediction. Radiology 292(1):60–66
Yassin NIR, Omran S, El Houby EMF, Allam H (2018) Machine learning techniques for breast cancer computer aided diagnosis using different image modalities A systematic review. Comput Methods Program Biomed 156:25–45
Yi X, Walia E, Babyn P (2019) Generative adversarial network in medical imaging A review. Med Image Anal 58:101552
Zagoruyko S, Komodakis N (2016) Wide residual networks. In: The British Machine Vision Conference
Zhang F, Luo L, Sun X, Zhou Z, Li X, Yu Y, Wang Y (2019) Cascaded generative and discriminative learning for microcalcification detection in breast mammograms. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 12578–12586
Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2015) Object detectors emerge in deep scene cnns. In: International Conference on Learning Representations
Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2921–2929
Zhu W, Lou Q, Vang YS, Xie X (2017) Deep multi-instance networks with sparse label assignment for whole mammogram classification. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp 603–611. Springer
Acknowledgements
We would like to thank the Breast Research Group, INESC Porto, Portugal for the INbreast database. This work is jointly supported by the National Natural Science Foundation of China (Nos.61961037), Natural Science Foundation of Gansu Province (Nos.18JR3RA288), and the Fundamental Research Funds for the Central Universities (Nos.lzuxxxy-2019-tm23).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare that there is no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lou, M., Wang, R., Qi, Y. et al. MGBN: Convolutional neural networks for automated benign and malignant breast masses classification. Multimed Tools Appl 80, 26731–26750 (2021). https://doi.org/10.1007/s11042-021-10929-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-021-10929-6