Skip to main content
SpringerLink
Log in

Medical ultrasound image segmentation using Multi-Residual U-Net architecture

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Advances in medical imaging modalities facilitate the early and accurate detection of tumors of various types. A preferred imaging modality for diagnosis and identification of tumors is the B-mode ultrasound imaging, but due to the noise and artifacts present, correct interpretation of lesions region becomes a difficult task for an inexperienced radiologist. In this context, an efficient and reliable computer-aided segmentation system is preferred for extracting regions of interest. Recently, conventional methods of segmentation have been replaced by deep learning methods. In this article, a novel Multi-Residual U-Net model is proposed for the segmentation of ultrasound medical images. This architecture adopts residual blocks to improve the performance of deep convolutional networks and a loss function that addresses the class imbalance issue. To improve the quality and reduce Speckle noise, input images are pre-processed using an optimized Non-Local Means filter. Three benchmark B-mode Ultrasound image datasets of 200 Breast lesion images, 504 Skeletal images, and 647 Breast Lesion images are used for experimentation. Experimental results demonstrate that the proposed model performs more accurate segmentation in comparison to the five deep models chosen for the study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig.4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The data sets analyzed during the study are available as open access data. http://bluebox.ippt.gov.pl/~hpiotrzk; https://www.mdpi.com/2313-433X/4/2/29; https://www.kaggle.com/datasets/aryashah2k/breast-ultrasound-images-dataset

References

  1. Abraham N, Khan NM (2019) A Novel Focal Tversky Loss Function With Improved Attention U-Net for Lesion Segmentation. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). pp. 683–687, https://doi.org/10.1109/isbi.2019.8759329

  2. Adams R, Bischof L (1994) Seeded region growing. IEEE Trans Pattern Anal Machine Vision 16:641–647. https://doi.org/10.1109/34.295913

    Article  Google Scholar 

  3. Al-Dhabyani W, Gomaa M, Khaled H, Fahmy A (2020) Dataset of breast ultrasound images. Data Brief 28:104863. https://doi.org/10.1016/j.dib.2019.104863

    Article  PubMed  Google Scholar 

  4. Almajalid R, Shan J, Du Y, Zhang M (2018). Development of a Deep-Learning-Based Method for Breast Ultrasound Image Segmentation, 17th IEEE Int. Conf. on Machine Learning and Applications (ICMLA), Orlando, FL, 2018, pp. 1103–1108, https://doi.org/10.1109/ICMLA.2018.00179

  5. Asma K, Asma B, Antoine M, Mohamed HB (2016) A coronary artery segmentation method based on multiscale analysis and region growing. Comp Med Imag Graph 48:49–61. https://doi.org/10.1016/j.compmedimag.2015.12.004

    Article  Google Scholar 

  6. Azad R, Asadi-Aghbolaghi M, Fathy M, Escalera S (2019) Bi-Directional ConvLSTM U-Net with Densley Connected Convolutions, https://doi.org/10.48550/arXiv.1909.00166

  7. Cao Z, Duan L, Yang G, Yue T, Chen Q (2019) An experimental study on breast lesion detection and classification from ultrasound images using deep learning architectures. BMC Med Imaging 19:51. https://doi.org/10.1186/s12880-019-0349-x. Springer

    Article  PubMed  PubMed Central  Google Scholar 

  8. Chan V, Perlas A (2011), Basics of Ultrasound Imaging, S.N. Narouze (ed.). Atlas of Ultrasound-Guided Procedures in Interventional Pain Management, Springer Science+Business Media, LLC. 13–19, https://doi.org/10.1007/978-1-4419-1681-5_2

  9. Chen Y, Jiang H, Li C, Jia X, Ghamisi P (2016) Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks. IEEE Trans Geosci Remote Sens 54(10):6232–6251. https://doi.org/10.1109/TGRS.2016.2584107

    Article  ADS  Google Scholar 

  10. Chen X, Zheng C, Yao H, Wang B (2017) Image segmentation using a unified Markov random field model. IET Image Process 11(10):860–869. https://doi.org/10.1049/iet-ipr.2016.1070

    Article  Google Scholar 

  11. Clarke L, Velthuizen R, Camacho M, Heine J, Vaidyanathan M, Hall L, Thatcher R, Silbiger M (1995) MRI segmentation: methods and applications. Magn Reson Imaging 13(3):343–368. https://doi.org/10.1016/0730-725X(94)00124-L

    Article  CAS  PubMed  Google Scholar 

  12. Cunningham R, Sánchez MB, May G, Loram I (2018) Estimating Full Regional Skeletal Muscle Fibre Orientation from B-Mode Ultrasound Images Using Convolutional, Residual, and Deconvolutional Neural Networks. J Imaging 4:29. https://doi.org/10.3390/jimaging4020029

    Article  Google Scholar 

  13. Dhanachandra N, Manglem K, Chanu YJ (2015) Image Segmentation using K-means Clustering Algorithm and Subtractive Clustering Algorithm. Procedia Comput Sci 54:764–771. https://doi.org/10.1016/j.procs.2015.06.090338-

    Article  Google Scholar 

  14. Dokur Z, Olmez T (2002) Segmentation of ultrasound images by using a hybrid neural network. Pattern Recog Lett 23:1825–1836. https://doi.org/10.1016/S0167-8655(02)00155-1

    Article  ADS  Google Scholar 

  15. Fourcade A, Khonsari RH (2019) Deep learning in medical image analysis: A third eye for doctors. J Stomatol Oral Maxillofac Surg 120(4):279–288. https://doi.org/10.1016/j.jormas.2019.06.002

    Article  CAS  PubMed  Google Scholar 

  16. Ghosh D, Kumar A, Ghosal P, Chowdhury T, Sadhu A, Nandi D (2020) Breast Lesion Segmentation in Ultrasound Images Using Deep Convolutional Neural Networks. 2020 IEEE Calcutta Conference (CALCON). https://doi.org/10.1109/calcon49167.2020.9106568

  17. Gonzalez RC, Woods RE (2008) Digital Image Processing, Edition 3. Pearson Education, Inc., Prentice Hall Publications

    Google Scholar 

  18. Guo Y, Duan X, Wang C, Guo H (2021) Segmentation and recognition of breast ultrasound images based on an expanded U-Net. PLoS ONE, 16(6), https://doi.org/10.1371/journal.pone.0253202

  19. Han B, Wu Y (2018). Active contours driven by global and local weighted SPF for image segmentation. Pattern Recog (2018), https://doi.org/10.1016/j.patcog.2018.12.028

  20. Hesamian MH, Jia W, He X, Kennedy P (2019) Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges. J Digit Imaging. https://doi.org/10.1007/s10278-019-00227-x

    Article  PubMed  PubMed Central  Google Scholar 

  21. Ibtehaz N, Rahman MS (2020) MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Netw 121(2020):74–87. https://doi.org/10.1016/j.neunet.2019.08.025

    Article  PubMed  Google Scholar 

  22. Isunuri BV, Kakarla J (2020) Fast brain tumour segmentation using optimized U-Net and adaptive thresholding. Automatika 61(3):352–360. https://doi.org/10.1080/00051144.2020.1760590

    Article  Google Scholar 

  23. Jain N, Kumar V (2016) IFCM Based Segmentation Method for Liver Ultrasound Images, J. Med Syst (2016) Springer, https://doi.org/10.1007/s10916-016-0623-1

  24. Johannes S, Rueden CT, Hiner MC, Eliceiri KW (2015) The imageJ ecosystem: an open platform for biomedical image analysis. Mol Reprod Dev 82(7–8):518–529. https://doi.org/10.1002/mrd.22489

    Article  CAS  Google Scholar 

  25. Kaltenbach TE, Engler P, Kratzer W, Oeztuerk S, Seufferlein T, Haenle MM, Graeter T (2016) Prevalence of benign focal liver lesions: ultrasound investigation of 45,319 hospital patients. Abdom Radiol 41:25–32. https://doi.org/10.1007/s00261-015-0605-7

    Article  PubMed Central  Google Scholar 

  26. Kingma DP, Ba J (2015) Adam: A Method for stochastic Optimization. San Diego: The International Conference on Learning Representations (ICLR), https://doi.org/10.48550/arXiv.1412.6980

  27. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25. Available online at: https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-onvolutional-neuralnetworks. Accessed on 22 Jan 2018

  28. Kumar SN, Lenin Fred A, Ajay Kumar H, Varghese PS (2018) ‘Performance Metric Evaluation of Segmentation Algorithms for Gold Standard Medical Images’, Springer Nature Singapore, Sa PK et al. (eds.), Recent Findings in Intelligent Computing Techniques, Advances in Intelligent Systems and Computing, 709, https://doi.org/10.1007/978-981-10-8633-5_45

  29. Kumar SN, Lenin Fred A, Muthukumar S, Haridhas AK, Varghese S (2018), A voyage on medical image segmentation algorithms, Biomedical Research; Special Issue: S75-S87, https://doi.org/10.4066/biomedicalresearch.29-16-1785

  30. Kumar D, Verma H, Mehra A (2019) A modified intuitionistic fuzzy c-means clustering approach to segment human brain MRI image. Multimed Tools Appl 78:12663–12687. https://doi.org/10.1007/s11042-018-5954-0

    Article  Google Scholar 

  31. Lee W-L (2013) An ensemble-based data fusion approach for characterizing ultrasonic liver tissue. Appl Soft Comput 13(8):3683–3692. https://doi.org/10.1016/j.asoc.2013.03.009

    Article  Google Scholar 

  32. Litjens G, Kooi T, Bejnordi BE, Setio AA, Adiyoso C, Francesco G, Mohsen et al (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88

    Article  PubMed  Google Scholar 

  33. Liu G, Li H (2018) Robust Evolution Method of Active Contour Models and Application in Segmentation of Image Sequence, Hindawi J Electr Comput Eng Vol 2018, https://doi.org/10.1155/2018/3493070

  34. Liu S, Wang Y, Yang X, Lei B, Liu L, Li SX, Ni D, Wang T (2019) Deep Learning in Medical Ultrasound Analysis: A Review. Engineering 5(2):261–275. https://doi.org/10.1016/j.eng.2018.11.020

    Article  Google Scholar 

  35. Mehnert A, Jackway P (1997) An improved seeded region growing algorithm. Pattern Recog Lett 18:1065–1071. https://doi.org/10.1016/S0167-8655(97)00131-1

    Article  ADS  Google Scholar 

  36. Mishra D, Chaudhury S, Sarkar M, Soin AS (2017) Ultrasound Image Segmentation: A Deeply Supervised Network with Attention to Boundaries. IEEE Trans Biomed Eng, https://doi.org/10.1109/TBME.2018.2877577

  37. Piotrzkowska-Wróblewska H (2017) Open access database of raw ultrasonic signals acquired from malignant and benign breast lesions. Med Phys. 44(11):6105–6109. https://doi.org/10.1002/mp.12538

    Article  CAS  PubMed  Google Scholar 

  38. Refaeilzadeh P, Tang L, Liu H (2009) Cross-Validation. In: LIU L., ÖZSU M.T. (eds) Encyclopedia of Database Systems. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-39940-9_565

  39. Ronneberger O, Fischer P, Brox T (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation, Vol. 1505.04597. https://arxiv.org/abs/1505.04597. Accessed 20 Mar 2021

  40. Ruan S, Lebonvallet S, Merabet A, Constans J-M (2007) Tumor segmentation from a multispectral MRI images by using support vector machine classification. In: Biomedical Imaging: From Nano to Macro. ISBI 2007. 4th IEEE International Symposium on, pp. 1236–1239. https://doi.org/10.1109/ISBI.2007.357082

  41. Santoro GA, Falco G (2004) Basic Principles of ultrasonography. Springer-Verlag Italia, Atlas of endoanal and endorectal ultrasonography. https://doi.org/10.1007/978-88-470-2129-7

    Book  Google Scholar 

  42. Shereena VB, Raju G (2021) Modified Non-Local Means Model for Speckle Noise Reduction in Ultrasound Images, 2nd Congress on Intelligent Systems, CIS 2021, Organized by Soft Computing Research Society and CHRIST (Deemed to be University), Bengaluru, September 04–05, 2021.

  43. Sokolova M, Japkowicz N, Szpakowicz S (2006) Beyond Accuracy, F-Score and ROC: A Family of Discriminant Measures for Performance Evaluation. Am Assoc Artif Intell. https://doi.org/10.1007/11941439_114

    Article  Google Scholar 

  44. Taha AA, Hanbury A (2015) Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool, BMC Medical Imaging, https://doi.org/10.1186/s12880-015-0068-x

  45. Targ S, Almeida D, Lyman K (2016) Resnet in Resnet: Generalizing Residual Architectures. https://arxiv.org/abs/1603.08029. Accessed 10-01-2021

  46. Thakur A, Radhey SA (2007) A local statistics based region growing segmentation method for ultrasound medical images. World Academy of Science, Engineering and Technology Int J Medical and Health Sciences 1(10):564–569

    Google Scholar 

  47. Vincent L, Soille P (1991) Watersheds in digital spaces: An efficient algorithm based on immersion simulation. IEEE Trans Pattern Anal Mach Intell 13:583–598. https://doi.org/10.1109/34.87344

    Article  Google Scholar 

  48. Wang CM, Lin GC (2014) A study on the application of fuzzy information seeded region growing in brain MRI tissue segmentation. Math Problems in Eng (Hindawi): https://doi.org/10.1155/2014/290607

  49. Wu Z, Shen C, Hengel AVD (2019) Wider or Deeper: Revisiting the ResNet Model for Visual Recognition. Pattern Recogn 90:119–133. https://doi.org/10.1016/j.patcog.2019.01.006

    Article  ADS  Google Scholar 

  50. Yuan J, Wang J (2017) Active contour based on Local Statistic Information and an Attractive Force for Ultrasound Image Segmentation, Advances in Intelligent Systems Research, Volume 132, https://doi.org/10.2991/msam-17.2017.23

  51. Zheng Y, Qin L, Qiu T, Zhou A, Xu P, Xue Z (2021) Automated detection and recognition of thyroid nodules in ultrasound images using Improve Cascade Mask R-CNN. Multimed Tools Appl 81(10):13253–13273. https://doi.org/10.1007/s11042-021-10939-4

    Article  Google Scholar 

  52. Zhuang Z, Li N, Joseph Raj AN, Mahesh VGV, Qiu S (2019) An RDAU-NET model for lesion segmentation in breast ultrasound images. PLoS ONE 14(8):e0221535. https://doi.org/10.1371/journal.pone.0221535

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of ComputerApplications, MES College Marampally, Kochi, India

    Shereena V. B.

  2. Department of Computer Science and Engineering, Alliance University, Bengaluru, India

    Raju G.

Authors
  1. Shereena V. B.
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Raju G.
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The first author (Shereena) identified the basic model used in the research and proposed modifications. The model is critically analyzed and approved by both the authors. The coding and experimentation are carried out by Shereena (Research scholar) which is validated by Raju (Supervisor). A draft paper is prepared by Shereena, which is verified and refined by Raju. The final draft is read and approved by both the authors.

Corresponding author

Correspondence to Raju G..

Ethics declarations

Conflicts of interests/Competing interests

The authors have no relevant financial or non-financial interests to disclose. The authors have no competing interests to declare that are relevant to the content of this article. All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript. The authors have no financial or proprietary interests in any material discussed in this article.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

V. B., S., G., R. Medical ultrasound image segmentation using Multi-Residual U-Net architecture. Multimed Tools Appl 83, 27067–27088 (2024). https://doi.org/10.1007/s11042-023-16461-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16461-z

Keywords

Search

Navigation