Skip to main content

Advertisement

SpringerLink
Log in

Performance evaluation of breast lesion detection systems with expert delineations: a comparative investigation on mammographic images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Performance of computerized diagnostic systems yearning to be approved by medical regulatory bodies must meet the expectations of human experts. Highly accurate lesion segmentation techniques have thus turned out to be an essential part for clinical acceptability of mammography based computer-aided diagnosis systems. The objective of this study is to evaluate the performance of six popular breast tumor detection techniques with manual delineations provided by two experienced radiologists on the mammographic images. In our study, 20 mammographic images from the mini-MIAS database are utilized. For the analysis, input mammographic images are first manually cropped to generate the region of interest (ROI). The ROI images are then pre-processed and segmentation is performed using different techniques, namely: expected maximization, K-means, Fuzzy c-Means (FCM), multilevel thresholding, region growing, and particle swarm optimization. The results were compared against the manual tracings. Among the other five segmentation techniques, FCM achieves the highest Jaccard Index (0.73 ± 0.06) and Dice Similarity Coefficient (0.82 ± 0.08) values. Statistical analysis (t-test, Mann Whitney U test, Wilcoxon test, Chi-Square test, and Kolmogorov–Smirnov test) and graphical analysis (Bland Altman and Regression plots) further prove the stability and reliability of the segmentation methods. Segmentation using FCM demonstrates the most accurate results and can be employed for the detection of breast cancer in the mammographic images. Further, it is concluded that computer-aided lesion detection systems can be used to assist Radiologists in routine clinical practice for the detection of breast tumors in mammographic images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Al-Faris AQ, Umi Kalthum N, MatIsa NA, Shuaib IL (2014) Computer-aided segmentation system for breast MRI tumour using modified automatic seeded region growing (BMRI-MASRG). J Digit Imaging 27:133–144

    Article  Google Scholar 

  2. Arodź T, Kurdziel M, Popiela TJ, Sevre EO, Yuen DA (2006) Detection of clustered microcalcifications in small field digital mammography. Comput Methods Prog Biomed 81(1):56–65

    Article  Google Scholar 

  3. Banchhor SK, Londhe ND, Saba L, Radeva P, Laird JR, Suri JS (2017) Relationship between automated coronary calcium volumes and a set of manual coronary lumen volume, vessel volume and atheroma volume in Japanese diabetic cohort. J Clin Diagn Res 11(6):TC09

    Google Scholar 

  4. Breast cancer statistics, How common is breast cancer? American cancer society. Online document available at https://www.cancer.org/cancer/breast-cancer/about/how-common-is-breastcancer.html. Accessed 4 Nov 2019

  5. Chen Z, Zwiggelaar R A modified fuzzy c-means algorithm for breast tissue density segmentation in mammograms. In: Information technology and applications in biomedicine (ITAB), 2010 10th IEEE international conference on 2010 Nov 3. IEEE, pp 1–4

  6. Choi JY, Kim DH, Plataniotis KN, Ro YM (2016) Classifier ensemble generation and selection with multiple feature representations for classification applications in computer-aided detection and diagnosis on mammography. Expert Syst Appl 46:106–121

    Article  Google Scholar 

  7. Ciecholewski M (2017) Malignant and benign mass segmentation in mammograms using active contour methods. Symmetry 9(11):277

    Article  Google Scholar 

  8. Costa DD, Campos LF, Barros AK (2011) Classification of breast tissue in mammograms using efficient coding. Biomed Eng Online 10:55

    Article  Google Scholar 

  9. Dheeba J, Singh NA, Selvi ST (2014) Computer-aided detection of breast cancer on mammograms: a swarm intelligence optimized wavelet neural network approach. J Biomed Inform 49:45–52

    Article  Google Scholar 

  10. Duijm LE, Louwman MW, Groenewoud JH, Van de Poll-Franse LV, Fracheboud J, Coebergh JW (2009) Inter-observer variability in mammography screening and effect of type and number of readers on screening outcome. Br J Cancer 100(6):901

    Article  Google Scholar 

  11. DuPrel JB, Röhrig B, Hommel G, Blettner M (2010) Choosing statistical tests: part 12 of a series on evaluation of scientific publications. Dtsch Arztebl Int 107(19):343

    Google Scholar 

  12. Elmore JG, Wells CK, Lee CH, Howard DH, Feinstein AR (1994) Variability in Radiologists' interpretations of mammograms. N Engl J Med 331(22):1493–1499

    Article  Google Scholar 

  13. Elmoufidi A, El Fahssi K, Jai-Andaloussi S, Sekkaki A (2015) Automatically density based breast segmentation for mammograms by using dynamic K-means algorithm and seed based region growing. In: Instrumentation and measurement technology conference (I2MTC), 2015 IEEE international. IEEE, pp 533–538

  14. Fan J, Zeng G, Body M, Hacid MS (2005) Seeded region growing: an extensive and comparative study. Pattern Recogn Lett 26(8):1139–1156

    Article  Google Scholar 

  15. GokilaDeepa G (2012) Mammogram image segmentation using fuzzy hybrid with particle swarm optimization (PSO). International Journal of Engineering and Innovative Technology (IJEIT) 2(6)

  16. Guliato D, Rangayyan RM, Carnielli WA, Zuffo JA, Desautels JL (2003) Segmentation of breast tumors in mammograms using fuzzy sets. J Electronic Imaging 12(3):369–379

    Article  Google Scholar 

  17. Gumaei A, El-Zaart A, Hussien M, Berbar M Breast segmentation using k-means algorithm with a mixture of gamma distributions. In: Broadband networks and fast internet (RELABIRA), 2012 symposium on 2012 May 28. IEEE, pp 97–102

  18. Harrabi R, Braiek EB (2012) Color image segmentation using multi-level thresholding approach and data fusion techniques: application in the breast cancer cells images. E J Image Video Proc 2012(1):11

    Article  Google Scholar 

  19. He W, Hogg P, Juette A, Denton ER, Zwiggelaar R (2015) Breast image pre-processing for mammographic tissue segmentation. Comput Biol Med 67:61–73

    Article  Google Scholar 

  20. Jalalian A, Mashohor S, Mahmud R, Karasfi B, Saripan MI, Ramli AR (2017) Foundation and methodologies in computer-aided diagnosis systems for breast cancer detection. EXCLI J 16:113

    Google Scholar 

  21. Jen CC, Yu SS (2015) Automatic detection of abnormal mammograms in mammographic images. Expert Syst Appl 42(6):3048–3055

    Article  Google Scholar 

  22. Karmilasari SW, Hermita M, Agustiyani NP, Hanum Y, Lussiana ET (2014) Sample k-means clustering method for determining the stage of breast cancer malignancy based on cancer size on mammogram image basis. IJACSA. Int J Adv Comput Sci Appl 5(3):86–90

    Google Scholar 

  23. Keller B, Nathan D, Wang Y, Zheng Y, Gee J, Conant E, Kontos D (2011) Adaptive multi-cluster fuzzy C-means segmentation of breast parenchymal tissue in digital mammography. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 562–569

  24. Keller BM, Nathan DL, Gavenonis SC, Chen J, Conant EF, Kontos D (2013) Reader variability in breast density estimation from full-field digital mammograms: the effect of image postprocessing on relative and absolute measures. Acad Radiol 20(5):560–568

    Article  Google Scholar 

  25. Li Y, Brennan PC, Lee W, Nickson C, Pietrzyk MW, Ryan EA (2015) An investigation into the consistency in mammographic density identification by radiologists: effect of radiologist expertise and mammographic appearance. J Digit Imaging 28(5):626–632

    Article  Google Scholar 

  26. Malvia S, Bagadi SA, Dubey US, Saxena S (2017) Epidemiology of breast cancer in Indian women. Asia Pac J Clin Oncol 13(4):289–295

    Article  Google Scholar 

  27. MedCalc:- https://www.medcalc.org/manual/. Accessed 3 July 2019

  28. Melouah A (2015) Comparison of automatic seed generation methods for breast tumor detection using region growing technique. In: IFIP international conference on computer science and its Applications_x000D_. Springer, Cham, pp 119–128

  29. Neto OP, Carvalho O, Sampaio W, Corrêa A, Paiva A Automatic segmentation of masses in digital mammograms using particle swarm optimization and graph clustering. In: Systems, signals and image processing (IWSSIP), 2015 international conference on 2015 Sep 10. IEEE, pp 109–112

  30. Ng KH, Muttarak M (2003) Advances in mammography have improved early detection of breast cancer. J HK Coll Radiol 6:126–131

    Google Scholar 

  31. Nurhasanah, Sampurno J, Faryuni ID, Ivansyah O (2016) Automated analysis of image mammogram for breast cancer diagnosis. In: AIP conference proceedings, vol. 1719, no. 1. AIP Publishing, pp 030036

  32. Oliver A, Freixenet J, Marti J, Perez E, Pont J, Denton ER, Zwiggelaar R (2010) A review of automatic mass detection and segmentation in mammographic images. Med Image Anal 14(2):87–110

    Article  Google Scholar 

  33. Pompe E, de Jong PA, De Jong WU, Takx RA, Eikendal AL, Willemink MJ, Oudkerk M, Budde RP, Lammers JW, Hoesein FA (2016) Inter-observer and inter-examination variability of manual vertebral bone attenuation measurements on computed tomography. Eur Radiol 26(9):3046–3053

    Article  Google Scholar 

  34. Raja NS, Sukanya SA, Nikita Y (2015) Improved PSO based multi-level thresholding for cancer infected breast thermal images using Otsu. Procedia Computer Science 48:524–529

    Article  Google Scholar 

  35. Raju NG, Rao NC (2013) Particle swarm optimization methods for image segmentation applied in mammography. Journal of Engineering Research and Applications 3(6):1572–1579. ISSN: 2248-9622

    Google Scholar 

  36. Ramani R, Valarmathy S, Vanitha NS (2013) Breast cancer detection in mammograms based on clustering techniques-a survey. Int J Comput Appl 62(11)

  37. Redondo A, Comas M, Macia F, Ferrer F, Murta-Nascimento C, Maristany MT, Molins E, Sala M, Castells X (2012) Inter-and intra radiologist variability in the BI-RADS assessment and breast density categories for screening mammograms. Br J Radiol 85(1019):1465–1470

    Article  Google Scholar 

  38. Rejani Y, Selvi ST (2009) Breast Cancer detection using multilevel thresholding. arXiv preprint arXiv:0911.0490

    Google Scholar 

  39. Saba L, Than JC, Noor NM, Rijal OM, Kassim RM, Yunus A, Ng CR, Suri JS (2016) Inter-observer variability analysis of automatic lung delineation in normal and disease patients. J Med Syst 40(6):142

    Article  Google Scholar 

  40. Saba L, Banchhor SK, Araki T, Suri HS, Londhe ND, Laird JR, Viskovic K, Suri JS (2018) Intra-and inter-operator reproducibility analysis of automated cloud-based carotid intima media thickness ultrasound measurement. J Clin Diagn Res 12(2):KC01-KC11

  41. Saba L, Banchhor SK, Araki T, Viskovic K, Londhe ND, Laird JR, Suri HS, Suri JS (2018) Intra-and inter-operator reproducibility of automated cloud-based carotid lumen diameter ultrasound measurement. Indian Heart J 70:649–664

    Article  Google Scholar 

  42. Sadad T, Munir A, Saba T, Hussain A (2018) Fuzzy C-means and region growing based classification of tumor from mammograms using hybrid texture feature. J Comput Sci 29:34–45

    Article  MathSciNet  Google Scholar 

  43. Saha A, Grimm LJ, Harowicz M, Ghate SV, Kim C, Walsh R, Mazurowski MA (2016) Interobserver variability in identification of breast tumors in MRI and its implications for prognostic biomarkers and radiogenomics. Med Phys 43(8 Part1):4558–4564

    Article  Google Scholar 

  44. Sathish A, Sundaram JM (2004) A comparative study on K-means and fuzzy C-means algorithm for breast cancer analysis. International Journal of Computational Intelligence and Informatics. H. Simpson, Dumb robots, 3rd edn. UOS Press, Springfield, pp 6–9

  45. Satyendra SA, Pawar MM (2017) Segmentation of breast images using Gaussian mixture models. Int J Adv Res Ideas Innov Technol 3(3):437–441. ISSN: 2454-132X

    Google Scholar 

  46. Senthilkumar B, Umamaheswari G, Karthik J A novel region growing segmentation algorithm for the detection of breast cancer. In: Computational intelligence and computing research (ICCIC), 2010 IEEE international conference on 2010 Dec 28. IEEE, pp 1–4

  47. Sheshadri HS, Kandaswamy A (2005) Detection of breast cancer by mammogram image segmentation. J Cancer Res Ther 1(4):232

    Article  Google Scholar 

  48. Siegel R, Naishadham D, Jemal A (2013a) Cancer statistics. CA Cancer J Clin 63:11–30

    Article  Google Scholar 

  49. Spandana P, Rao KM (2013) Novel image processing techniques for early detection of breast cancer, mat lab and lab view implementation. In: Point-of-Care Healthcare Technologies (PHT), pp 105–108

  50. Suckling J et al (1994) The Mammographic Image Analysis Society Digital Mammogram Database Exerpta Medica. Int Congr Ser 1069:375–378

    Google Scholar 

  51. Sujata RB, Dhiman R, Chugh TS (2012) An evaluation of two mammography segmentation techniques. International Journal of Advanced Computer Research 2 Number-4(7):2277–7970. (ISSN (print): 2249-7277 ISSN (online)

  52. Survey by Indian cancer society (2013) Indian cancer society

  53. Thyagarajan R, Murugavalli S (2012) Segmentation of digital breast tomograms using clustering techniques. In: India conference (INDICON), 2012 annual IEEE. IEEE, pp 1090–1094

  54. Valarmathie P, Sivakrithika V, Dinakaran K (2016) Classification of mammogram masses using selected texture, shape and margin features with multilayer perceptron classifier. Biomedical research India, Special Issue page no. s310–s313

  55. Vedanarayanan V (2017) Advanced image segmentation techniques for accurate isolation of abnormality to enhance breast cancer detection in digital mammographs. Biomed Res 28(6):2753–2757

    Google Scholar 

  56. Vesal S, Ravikumar N, Ellman S, Maier A (2018) Comparative analysis of unsupervised algorithms for breast MRI lesion segmentation. In: Bildverarbeitung für die Medizin 2018. Springer Vieweg, Berlin, pp 257–262

  57. Wang Y, Li J, Gao X (2014) Latent feature mining of spatial and marginal characteristics for mammographic mass classification. Neurocomputing 144:107–118

    Article  Google Scholar 

  58. Wang J, Jing H, Wernick MN, Nishikawa RM, Yang Y (2014) Analysis of perceived similarity betweenpairs of microcalcification clusters in mammograms. Med Phys 41(5):051904

    Article  Google Scholar 

  59. Yuvaraj K, Ragupathy US (2013) Automatic mammographic mass segmentation based on region growing technique. In: 3rd international conference on electronics, biomedical engineering and its applications (ICEBEA'2013), pp 29–30

  60. Zheng Y, Keller BM, Ray S, Wang Y, Conant EF, Gee JC, Kontos D (2015) Parenchymal texture analysis in digital mammography: a fully automated pipeline for breast cancer risk assessment. Med Phys 42(7):4149–4160

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Chhatisgarh Council of Science & Technology, Raipur, India (grant number 2481/CCOST/MRP/2016). The authors of this article are extremely grateful to them.

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, National Institute of Technology Raipur, Raipur, CG, 492001, India

    Bikesh K. Singh, Pankaj Jain & Sumit K. Banchhor

  2. Department of Computer Applications, National Institute of Technology Raipur, Raipur, CG, India

    Kesari Verma

Authors
  1. Bikesh K. Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pankaj Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sumit K. Banchhor
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kesari Verma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bikesh K. Singh.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1

Table 6 Description of all the statistical and performance tests
Full size table

Appendix 2

Fig. 7
figure 7

Bland Altman plots between all the segmentation techniques (expected maximization (EM), Fuzzy c-Means (FCM), K-means (KM), multilevel thresholding (MT), region growing (RG), particle swarm optimization (PSO), and Radiologist 1 (R1)

Full size image
Fig. 8
figure 8

Bland Altman plots between all the segmentation techniques (expected maximization (EM), Fuzzy c-Means (FCM), K-means (KM), multilevel thresholding (MT), region growing (RG), particle swarm optimization (PSO), and Radiologist 2 (R2)

Full size image
Fig. 9
figure 9

Regression plots between all the segmentation techniques (expected maximization (EM), Fuzzy c-Means (FCM), K-means (KM), multilevel thresholding (MT), region growing (RG), particle swarm optimization (PSO), and Radiologist 1 (R1)

Full size image
Fig. 10
figure 10

Regression plots between all the segmentation techniques (expected maximization (EM), Fuzzy c-Means (FCM), K-means (KM), multilevel thresholding (MT), region growing (RG), particle swarm optimization (PSO), and Radiologist 2 (R2)

Full size image

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Singh, B.K., Jain, P., Banchhor, S.K. et al. Performance evaluation of breast lesion detection systems with expert delineations: a comparative investigation on mammographic images. Multimed Tools Appl 78, 22421–22444 (2019). https://doi.org/10.1007/s11042-019-7570-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7570-z

Keywords

Search

Navigation