Skip to main content
SpringerLink
Log in

Aggregated pyramid attention network for mass segmentation in mammograms

  • 1176: Artificial Intelligence and Deep Learning for Biomedical Applications
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Intra-class inconsistency and inter-class indistinction are intractable problems that commonly exist in breast mass segmentation from mammograms. In this work, a novel deep learning segmentation model is presented to address these problems. Firstly, we propose a simple yet effective aggregated pyramid attention module (APAM) for capturing intra-class dependencies, aiming at effectively aggregating contextual dependencies from different receptive fields to reinforce feature representations. Then, a novel aggregated pyramid attention network (APANet) is developed for further releasing the limitation of both intra-class inconsistency and inter-class indistinction. The APANet can combine low-level spatial details and high-level contextual information via encoder-decoder structure for further refining semantic representations. Finally, our proposed APANet is greatly demonstrated on two public mammographic databases including the DDSM-BCRP and INbreast, separately achieving the Dice Similarity Coefficient (DSC) of 91.04% and 94.02%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. The INbreast database can be acquired by addressing the e-mail medicalresearch@inescporto.pt

  2. http://www.eng.usf.edu/cvprg/Mammography/DDSM/BCRP/bcrp.html

References

  1. Al-Najdawi N, Biltawi M, Tedmori S (2015) Mammogram image visual enhancement, mass segmentation and classification. Appl Soft Comput 35:175–185

    Article  Google Scholar 

  2. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  3. Beller M, Stotzka R, Müller TO, Gemmeke H (2005) An example-based system to support the segmentation of stellate lesions. In Bildverarbeitung für die Medizin 2005, Algorithmen - Systeme - Anwendungen, Proceedings des Workshops vom 13.-15. März 2005 in Heidelberg

  4. Birdwell RL, Ikeda DM, O’Shaughnessy KF, Sickles EA (2001) Mammographic characteristics of 115 missed cancers later detected with screening mammography and the potential utility of computer-aided detection. Radiology 219(1):192–202

    Article  Google Scholar 

  5. Chan HP, Sahiner B, Lam KL, Petrick N, Helvie MA, Goodsitt MM, Adler DD (2007) Computerized analysis of mammographic microcalcifications in morphological and texture feature spaces. Med Phys 25(10):1998

    Google Scholar 

  6. Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  7. Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), pages 801–818

  8. Cheng H-D, Shi XJ, Rui M, Hu LM, Cai XP, Du HN (2006) Approaches for automated detection and classification of masses in mammograms. Pattern Recogn 39(4):646–668

    Article  Google Scholar 

  9. CO Communities and SOO Communities (2010) Health statistics: atlas on mortality in the european union. Office for Official Publications of the European Communities

  10. DeSantis CE, Ma J, Gaudet MM, Newman LA, Miller KD, Sauer AG, Jemal A, Siegel RL (2019) Breast cancer statistics, 2019. CA Cancer J Clin 69(6):438–451

    Article  Google Scholar 

  11. Desautels JEL, Rangayyan R, Mudigonda NR (2000) Gradient and texture analysis for the classification of mammographic masses. IEEE Trans Med Imaging 19(10):1032–1043

    Article  Google Scholar 

  12. Dhungel N, Carneiro G, Bradley AP (2015) Deep structured learning for mass segmentation from mammograms. In 2015 IEEE international conference on image processing (ICIP), pages 2950–2954. IEEE

  13. Dhungel N, Carneiro G, Bradley AP. (2015) Tree re-weighted belief propagation using deep learning potentials for mass segmentation from mammograms. In 2015 IEEE 12th international symposium on biomedical imaging (ISBI), pages 760–763. IEEE

  14. Dhungel N, Carneiro G, Bradley AP (2015) Deep learning and structured prediction for the segmentation of mass in mammograms. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 605–612. Springer

  15. Freer TW, Ulissey MJ (2001) Screening mammography with computer-aided detection: prospective study of 12,860 patients in a community breast center1. Radiology 220(3):781–786

    Article  Google Scholar 

  16. Giger ML, Karssemeijer N, Schnabel JA (2013) Breast image analysis for risk assessment, detection, diagnosis, and treatment of cancer. Annu Rev Biomed Eng 15(1):327–357

    Article  Google Scholar 

  17. Guliato D (2007) Rangaraj M Rangayyan, Juliano D Carvalho, and Sérgio a Santiago. Polygonal modeling of contours of breast tumors with the preservation of spicules. IEEE Trans Biomed Eng 55(1):14–20

    Article  Google Scholar 

  18. He J, Deng Z, Yu Q (2019) Dynamic multi-scale filters for semantic segmentation. In Proceedings of the IEEE International Conference on Computer Vision, pages 3562–3572

  19. He J, Deng Z, Zhou L, Wang Y, Yu Q (2019) Adaptive pyramid context network for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7519–7528

  20. He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916

    Article  Google Scholar 

  21. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778

  22. Heath M, Bowyer K, Kopans D, Kegelmeyer P, Moore R, Chang K, Munishkumaran S (1998) Current status of the digital database for screening mammography. In Digital mammography, pages 457–460. Springer

  23. Horsch A, Hapfelmeier A, Elter M (2011) Needs assessment for next generation computer-aided mammography reference image databases and evaluation studies. Int J Comput Assist Radiol Surg

  24. Hou Q, Zhang L, Cheng M-M, Feng J (2020) Strip pooling: rethinking spatial pooling for scene parsing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  25. Ibtehaz N, Rahman MS (2020) Multiresunet: Rethinking the u-net architecture for multimodal biomedical image segmentation. Neural Netw 121:74–87

    Article  Google Scholar 

  26. Inês C, Moreira IA, Domingues I, Cardoso A, Cardoso MJ, Cardoso JS (2012) Inbreast: toward a full-field digital mammographic database. Acad Radiol 19(2):236–248

    Article  Google Scholar 

  27. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning

  28. Jaime S, Cardoso ID, Helder P (2015) Oliveira Closed shortest path in the original coordinates with an application to breast cancer. Int J Pattern Recognit Artif Intell 29(1):1555002.1–1555002.24

    Google Scholar 

  29. Jun F, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H (2019) Dual attention network for scene segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3146–3154

  30. Kilday J, Palmieri F, Fox MD (1993) Classifying mammographic lesions using computerized image analysis. IEEE Trans Med Imaging 12(4):669

    Article  Google Scholar 

  31. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  32. Lehman CD, Arao RF, Sprague BL, Lee JM, Buist DSM, Kerlikowske K, Henderson LM, Onega T, Tosteson ANA, Garth H, Rauscher (2017) National performance benchmarks for modern screening digital mammography: Update from the breast cancer surveillance consortium. Radiology 283(1):49–58

    Article  Google Scholar 

  33. Li H, Chen D, Nailon WH, Davies ME, Laurenson D (2018) Improved breast mass segmentation in mammograms with conditional residual u-net. In Image Analysis for Moving Organ, Breast, and Thoracic Images, pages 81–89. Springer

  34. Li H, Chen D, Nailon WH, Davies ME, Laurenson D (2019) A deep dual-path network for improved mammogram image processing. In ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1224–1228. IEEE

  35. Lin G, Milan A, Shen C, Reid I (2017) Refinenet: multi-path refinement networks for high-resolution semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1925–1934

  36. Liu W, Rabinovich A, Berg AC (2016) Parsenet: Looking wider to see better. In International Conference on Learning Representations

  37. Løberg M, Lousdal ML, Bretthauer M, Kalager M (2015) Benefits and harms of mammography screening. Breast Cancer Res 17(1):63

    Article  Google Scholar 

  38. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3431–3440

  39. Lu X, Dong M, Ma Y, Wang K (2015) Automatic mass segmentation method in mammograms based on improved vfc snake model. In Emerging Trends in Image Processing, Computer Vision and Pattern Recognition, pages 201–217. Elsevier

  40. Misra S, Solomon NL, Moffat FL, Koniaris LG (2010) Screening criteria for breast cancer. Adv Surg 44(1):87–100

    Article  Google Scholar 

  41. Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. In International Conference on Machine Learning

  42. Noh H, Hong S, Han B (2015) Learning deconvolution network for semantic segmentation. In Proceedings of the IEEE international conference on computer vision, pages 1520–1528

  43. Oktay O, Schlemper J, Le Folgoc L, Lee M, Heinrich M, Misawa K, Mori K, McDonagh S, Hammerla N Y, Kainz B, et al. (2018) Attention u-net: Learning where to look for the pancreas. In International Conference on Medical Imaging with Deep Learning

  44. Oliver A, Freixenet J, Martí J, Pérez E, Pont J, Denton ERE, Zwiggelaar R (2010) A review of automatic mass detection and segmentation in mammographic images. Med Image Anal 14(2):87–110

    Article  Google Scholar 

  45. Pang T, Wong JHD, Ng Wei L, Chan CS (2020) Deep learning radiomics in breast cancer with different modalities: Overview and future. Expert Syst Appl, page 113501

  46. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, et al. (2019) Pytorch: an imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems, pages 8024–8035

  47. Rahmati P, Adler A, Hamarneh G (2012) Mammography segmentation with maximum likelihood active contours. Med Image Anal 16(6):1167–1186

    Article  Google Scholar 

  48. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer

  49. Sahiner B, Petrick N, Chan HP, Hadjiiski LM, Paramagul C, Helvie MA, Gurcan MN (2001) Computer-aided characterization of mammographic masses: accuracy of mass segmentation and its effects on characterization. IEEE Trans Med Imaging 20(12):1275–1284

    Article  Google Scholar 

  50. Sanders D, and E (1988) Breast cancer detection: Mammography and other methods in breast imaging. 2d ed. Radiology

  51. Siegel RL, Miller KD, Jemal A (2019) Cancer statistics, 2019. CA Cancer J Clin 69(1):7–34

    Article  Google Scholar 

  52. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations

  53. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI conference on artificial intelligence

  54. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1–9

  55. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818–2826

  56. Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7794–7803

  57. Wang Z, Zou N, Shen D, Ji S (2020) Non-local u-nets for biomedical image segmentation. In AAAI 34:6315–6322

    Article  Google Scholar 

  58. Warren LJ (2009) Variability in interpretive performance at screening mammography and radiologists’ characteristics associated with accuracy : Elmore jg, Jackson sl, abraham l, etal (univ of Washington school of medicine, Seattle; group health res inst, Seattle, Wa; etal). Radiology 21(4):330–332

    Google Scholar 

  59. Woo S, Park J, Lee J-Y, Kweon I S (2018) Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV), pages 3–19

  60. Yu C, Wang J, Peng C, Gao C, Yu G, Sang N (2018) Learning a discriminative feature network for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1857–1866

  61. Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2881–2890

  62. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2019) Unet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imaging 39(6):1856–1867

    Article  Google Scholar 

  63. Zhu W, Xiang X, Tran TD, Hager GD, Xie X (2018) Adversarial deep structured nets for mass segmentation from mammograms. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pages 847–850. IEEE

Download references

Acknowledgements

We would like to thank the Breast Research Group, INESC Porto, Portugal for the INbreast database. This work is jointly supported by the National Natural Science Foundation of China (Nos.61961037), the Natural Science Foundation of Gansu Province (Nos. 18JR3RA288), and the Fundamental Research Funds for the Central Universities (Nos.lzuxxxy-2019-tm23).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Lanzhou University, Lanzhou, Gansu, China

    Meng Lou, Yunliang Qi, Xiaorong Li, Chunbo Xu, Wenwei Zhao & Yide Ma

  2. College of Physics and Electronic Engineering, Northwest Normal University, Lanzhou, Gansu, China

    Xiangyu Deng

Authors
  1. Meng Lou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yunliang Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaorong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chunbo Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wenwei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xiangyu Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yide Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yide Ma.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lou, M., Qi, Y., Li, X. et al. Aggregated pyramid attention network for mass segmentation in mammograms. Multimed Tools Appl 81, 13335–13353 (2022). https://doi.org/10.1007/s11042-021-10940-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-10940-x

Keywords

Search

Navigation