Skip to main content
SpringerLink
Log in

Transformer-based mass detection in digital mammograms

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

In the last decade, Convolutional Neural Networks (CNNs) have been the de facto approach for automated medical image detection. Recently, Vision Transformers have emerged in computer vision as an alternative to CNNs. Specifically, the Shifted Window (Swin) Transformer is a general-purpose backbone that learns attention-based hierarchical features and achieves state-of-the-art performances in a variety of vision tasks. In this work, for the first time, we design and experiment transformer-based models for mass detection in digital mammograms leveraging Swin transformer as a backbone multiscale feature extractor. Experiments on the largest publicly available mammography image database OMI-DB yield a True Positive Rate (TPR) of \(75.7\%\) at 0.1 False Positives per Image (FPpI) for the best transformer model, with \(2.5\%\) TPR improvement over its convolutional counterpart and a massive \(7.4\%\) TPR over the state-of-the-art. We also combine transformer- and convolution-based detectors with weighted box fusion, achieving an additional \(2.4\%\) TPR improvement reaching \(78.1\%\) TPR at 0.1 FPpI.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availibility

The OMI-DB dataset (Halling-Brown et al. 2020) employed in the current study is publicly available at https://medphys.royalsurrey.nhs.uk/omidb/. The list of images of the OMI-H-MD subset that we extracted and used in our experiments are available from the corresponding author upon reasonable request.

References

  • Agarwal R, Díaz O, Yap MH, Llado X, Marti R (2020) Deep learning for mass detection in Full Field Digital Mammograms. Comput Biol Med 121:103774

    Article  Google Scholar 

  • Aly GH, Marey M, El-Sayed SA, Tolba MF (2021) YOLO based breast masses detection and classification in Full-Field Digital Mammograms. Comput Methods Programs Biomed 200:105823

    Article  Google Scholar 

  • Balleyguier C, Kinkel K, Fermanian J, Malan S, Djen G, Taourel P, Helenon O (2005) Computer-aided detection (CAD) in mammography: does it help the junior or the senior radiologist? Eur J Radiol 54(1):90–96

    Article  Google Scholar 

  • Bria A, Karssemeijer N, Tortorella F (2014) Learning from unbalanced data: a cascade-based approach for detecting clustered microcalcifications. Med Image Anal 18(2):241–252

    Article  Google Scholar 

  • Cao H, Pu S, Tan W, Tong J (2021) Breast mass detection in digital mammography based on anchor-free architecture. Comput Methods Programs Biomed 205:106033

    Article  Google Scholar 

  • Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S (2020) End-to-end object detection with transformers. In: European conference on computer vision, pp 213–229. Springer

  • CDC (2022) Breast cancer screening guidelines for women. https://www.cdc.gov/cancer/breast/pdf/breast-cancer-screening-guidelines-508.pdf. Accessed: 2022-05-20

  • Chen X, Zhang K, Abdoli N, Gilley PW, Wang X, Liu H, Zheng B, Qiu Y (2022) Transformers improve breast cancer diagnosis from unregistered multi-view mammograms. Diagnostics 12(7):1549

    Article  Google Scholar 

  • Chen K, Wang J, Pang J, Cao Y, Xiong Y, Li X, Sun S, Feng W, Liu Z, Xu J et al (2019) MMDetection: open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155

  • Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, pp 248–255. Ieee

  • Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S et al (2020) An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929

  • Ferrari RJ, Rangayyan RM, Desautels JL, Borges R, Frere AF (2004) Automatic identification of the pectoral muscle in mammograms. IEEE Trans Med Imaging 23(2):232–245

    Article  Google Scholar 

  • Halling-Brown MD, Warren LM, Ward D, Lewis E, Mackenzie A, Wallis MG, Wilkinson LS, Given-Wilson RM, McAvinchey R, Young KC (2020) OPTIMAM Mammography image database: a large-scale resource of mammography images and clinical data. Radiology 3(1):e200103

    Google Scholar 

  • Heath MD, Bowyer KW (2000) Mass detection by relative image intensity. In: Proceedings of the 5th International Workshop on Digital Mammography (IWDM-2000), pp 219–225

  • He K, Zhang X, Ren S, Sun J (2016a) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  • He K, Zhang X, Ren S, Sun J (2016b) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  • Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  • Hupse R, Karssemeijer N (2009) Use of normal tissue context in computer-aided detection of masses in mammograms. IEEE Trans Med Imaging 28(12):2033–2041

    Article  Google Scholar 

  • Johnson KB, Wei W-Q, Weeraratne D, Frisse ME, Misulis K, Rhee K, Zhao J, Snowdon JL (2021) Precision medicine, AI, and the future of personalized health care. Clin Transl Sci 14(1):86–93

    Article  Google Scholar 

  • Kamran SA, Hossain KF, Tavakkoli A, Bebis G, Baker S (2022) Swin-sftnet: spatial feature expansion and aggregation using swin transformer for whole breast micro-mass segmentation. arXiv preprint arXiv:2211.08717

  • Ke L, Mu N, Kang Y (2010) Mass computer-aided diagnosis method in mammogram based on texture features. In: 2010 3rd International Conference on Biomedical Engineering and Informatics, Volume 1, . 354–357. IEEE

  • Kooi T, Litjens G, van Ginneken B, Gubern-Mérida A, Sánchez CI, Mann R, den Heeten A, Karssemeijer N (2017) Large scale deep learning for computer aided detection of mammographic lesions. Med Image Anal 35:303–312

    Article  Google Scholar 

  • Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Article  Google Scholar 

  • Kwok SM, Chandrasekhar R, Attikiouzel Y, Rickard MT (2004) Automatic pectoral muscle segmentation on mediolateral oblique view mammograms. IEEE Trans Med Imaging 23(9):1129–1140

    Article  Google Scholar 

  • Lbachir IA, Daoudi I, Tallal S (2021) Automatic computer-aided diagnosis system for mass detection and classification in mammography. Multimed Tools Appl 80(6):9493–9525

    Article  Google Scholar 

  • Lin T-Y, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988

  • Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: common objects in context. In: European conference on computer vision, pp 740–755. Springer

  • Liu Z, Hu H, Lin Y, Yao Z, Xie Z, Wei Y, Ning J, Cao Y, Zhang Z, Dong L et al (2021) Swin transformer V2: scaling up capacity and resolution. arXiv preprint arXiv:2111.09883

  • Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B (2021) Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 10012–10022

  • Li F, Zhang H, Liu S, Zhang L, Ni LM, Shum H-Y et al (2022) Mask dino: towards a unified transformer-based framework for object detection and segmentation. arXiv preprint arXiv:2206.02777

  • Loshchilov I, Hutter F (2017) Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101

  • Ma H, Bandos AI, Rockette HE, Gur D (2013) On use of partial area under the roc curve for evaluation of diagnostic performance. Stat Med 32(20):3449–3458

    Article  MathSciNet  Google Scholar 

  • Malliori A, Pallikarakis N (2022) Breast cancer detection using machine learning in digital mammography and breast tomosynthesis: a systematic review. Health Technol 1–18

  • Méndez AJ, Tahoces PG, Lado MJ, Souto M, Correa J, Vidal JJ (1996) Automatic detection of breast border and nipple in digital mammograms. Comput Methods Programs Biomed 49(3):253–262

    Article  Google Scholar 

  • Molinara M, Marrocco C, Tortorella F (2013) Automatic segmentation of the pectoral muscle in mediolateral oblique mammograms. In: Proceedings of the 26th IEEE International Symposium on Computer-Based Medical Systems, pp 506–509. IEEE

  • Mordang J-J, Janssen T, Bria A, Kooi T, Gubern-Mérida A, Karssemeijer N (2016) Automatic microcalcification detection in multi-vendor mammography using convolutional neural networks. In: International Workshop on Digital Mammography, pp 35–42. Springer

  • Mughal B, Sharif M, Muhammad N (2017) Bi-model processing for early detection of breast tumor in CAD system. Eur Phys J Plus 132(6):1–14

    Article  Google Scholar 

  • Patel BC, Sinha G, Soni D (2019) Detection of masses in mammographic breast cancer images using modified histogram based adaptive thresholding (MHAT) method. Int J Biomed Eng Technol 29(2):134–154

    Article  Google Scholar 

  • Petrick N, Chan H-P, Sahiner B, Wei D (1996) An adaptive density-weighted contrast enhancement filter for mammographic breast mass detection. IEEE Trans Med Imaging 15(1):59–67

    Article  Google Scholar 

  • Petrick N, Chan H-P, Sahiner B, Helvie MA (1999) Combined adaptive enhancement and region-growing segmentation of breast masses on digitized mammograms. Med Phys 26(8):1642–1654

    Article  Google Scholar 

  • Punitha S, Amuthan A, Joseph KS (2018) Benign and malignant breast cancer segmentation using optimized region growing technique. Future Comput Inform J 3(2):348–358

    Article  Google Scholar 

  • Rajpurkar P, Chen E, Banerjee O, Topol EJ (2022) Ai in health and medicine. Nat Med 28(1):31–38

    Article  Google Scholar 

  • Redmon J, Divvala S, Girshick R, Farhadi A (2016a) You only look once: Unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788

  • Redmon J, Divvala S, Girshick R, Farhadi A (2016b) You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788

  • Ren S, He K, Girshick R, Sun J(2015) Faster r-cnn: Towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst 28

  • Ribli D, Horváth A, Unger Z, Pollner P, Csabai I (2018) Detecting and classifying lesions in mammograms with deep learning. Sci Rep 8(1):1–7

    Article  Google Scholar 

  • Salim M, Dembrower K, Eklund M, Lindholm P, Strand F (2020) Range of radiologist performance in a population-based screening cohort of 1 million digital mammography examinations. Radiology 297(1):33–39

    Article  Google Scholar 

  • Samuelson FW, Petrick N (2006) Comparing image detection algorithms using resampling. In: IEEE Int. Symp. Biomed. Imag., pp 1312–1315

  • Sankatsing VD, van Ravesteyn NT, Heijnsdijk EA, Looman CW, van Luijt PA, Fracheboud J, den Heeten GJ, Broeders MJ, de Koning HJ (2017) The effect of population-based mammography screening in Dutch municipalities on breast cancer mortality: 20 years of follow-up. Int J Cancer 141(4):671–677

    Article  Google Scholar 

  • Shamshad F, Khan S, Zamir SW, Khan MH, Hayat M, Khan FS, Fu H (2022) Transformers in medical imaging: a survey. arXiv preprint arXiv:2201.09873

  • Solovyev R, Wang W, Gabruseva T (2021) Weighted boxes fusion: Ensembling boxes from different object detection models. Image Vis Comput 107:104117

    Article  Google Scholar 

  • Su Y, Liu Q, Xie W, Hu P (2022) Yolo-logo: a transformer-based yolo segmentation model for breast mass detection and segmentation in digital mammograms. Comput Methods Programs Biomed 106903

  • Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, Bray F (2021) Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. Cancer J Clin 71(3):209–249

    Article  Google Scholar 

  • Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J (2016) Convolutional neural networks for medical image analysis: Full training or fine tuning? IEEE Trans Med Imaging 35(5):1299–1312

    Article  Google Scholar 

  • Te Brake GM, Karssemeijer N (1999) Single and multiscale detection of masses in digital mammograms. IEEE Trans Med Imaging 18(7):628–639

    Article  Google Scholar 

  • Valanarasu JMJ, Oza P, Hacihaliloglu I, Patel VM (2021) Medical transformer: gated axial-attention for medical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp 36–46. Springer

  • Wang Z, Yu G, Kang Y, Zhao Y, Qu Q (2014) Breast tumor detection in digital mammography based on extreme learning machine. Neurocomputing 128:175–184

    Article  Google Scholar 

  • Wang W, Xie E, Li X, Fan D-P, Song K, Liang D, Lu T, Luo P, Shao L (2021) Pyramid vision transformer: a versatile backbone for dense prediction without convolutions. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 568–578

  • Wang J, Yang Y (2019) A hierarchical learning approach for detection of clustered microcalcifications in mammograms. In : 2019 IEEE International Conference on Image Processing (ICIP), pp 804–808

  • Wei Y, Hu H, Xie Z, Zhang Z, Cao Y, Bao J, Chen D, Guo B (2022) Contrastive learning rivals masked image modeling in fine-tuning via feature distillation. arXiv preprint arXiv:2205.14141

  • Yan Y, Conze P-H, Lamard M, Quellec G, Cochener B, Coatrieux G (2021) Towards improved breast mass detection using dual-view mammogram matching. Med Image Anal 71:102083

    Article  Google Scholar 

  • Yang Z, Liu S, Hu H, Wang L, Lin S(2019) Reppoints: point set representation for object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 9657–9666

  • Yu X, Wang S-H, Zhang Y-D(2022) Multiple-level thresholding for breast mass detection. J King Saud Univ-Comput Inf Sci

  • Zhang L, Li Y, Chen H, Wu W, Chen K, Wang S (2022) Anchor-free yolov3 for mass detection in mammogram. Expert Syst Appl 191:116273

    Article  Google Scholar 

  • Zhang Z, Zhang H, Zhao L, Chen T, Arik SÖ, Pfister T (2022) Nested hierarchical transformer: towards accurate, data-efficient and interpretable visual understanding. Proc AAAI Conf Artif Intell 36:3417–3425

    Google Scholar 

  • Zhang H, Li F, Liu S, Zhang L, Su H, Zhu J, Ni LM, Shum H-Y (2022) Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605

  • Zhou Z, Shin J, Zhang L, Gurudu S, Gotway M, Liang J (2017) Fine-tuning convolutional neural networks for biomedical image analysis: actively and incrementally. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7340–7351

  • Zhu C, He Y, Savvides M (2019) Feature selective anchor-free module for single-shot object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 840–849

  • Zhu X, Su W, Lu L, Li B, Wang X, Dai J (2020) Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159

Download references

Acknowledgements

This work was supported by MIUR (Minister for Education, University and Research, Law 232/216, Department of Excellence). Amparo S. Betancourt T. holds an EACEA Erasmus+ grant for the master in Medical Imaging and Applications (MAIA).

Author information

Authors and Affiliations

  1. Department of Electrical and Information Engineering, University of Cassino and Southern Latium, Via G. Di Biasio 43, Cassino, 03043, Italy

    Amparo S. Betancourt Tarifa, Claudio Marrocco, Mario Molinara & Alessandro Bria

  2. Department of Information and Electrical Engineering and Applied Mathematics, University of Salerno, Via Giovanni Paolo II 132, Fisciano, 84084, Italy

    Francesco Tortorella

Authors
  1. Amparo S. Betancourt Tarifa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Claudio Marrocco
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mario Molinara
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Francesco Tortorella
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alessandro Bria
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alessandro Bria.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Betancourt Tarifa, A.S., Marrocco, C., Molinara, M. et al. Transformer-based mass detection in digital mammograms. J Ambient Intell Human Comput 14, 2723–2737 (2023). https://doi.org/10.1007/s12652-023-04517-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-023-04517-9

Keywords

Search

Navigation