Skip to main content

Advertisement

SpringerLink
Log in

Fully multi-target segmentation for breast ultrasound image based on fully convolutional network

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Ultrasound image segmentation plays an important role in computer-aided diagnosis of breast cancer. Existing approaches focused on extracting the tumor tissue to characterize the tumor class. However, other tissues are also helpful for providing the references. In this paper, a multi-target semantic segmentation approach is proposed based on the fully convolutional network for segmenting the breast ultrasound image into different target tissue regions. For handling the uncertain affiliation of pixels in blurry boundaries, the certain outputs of pixel characteristics in AlexNet are transformed into the fuzzy decision expression. For improving the image detail representation, the AlexNet network structure of fully convolutional network is optimized with fully connected skip structure. In addition, the output of net model is optimized with fully connected conditional random field to improve the characterization of spatial consistency and pixels’ correlation of the image. Moreover, a data training optimization method is developed for improving the efficiency of network training. In the experiment, 325 ultrasound images and four error metrics are utilized for validating the segmentation performance. Comparing with existing methods, experimental results show that the proposed approach is effective for handling the breast ultrasound images accurately and reliably.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Cheng HD, Shan J, Ju W, Guo YH, Zhang L (2010) Automated breast cancer detection and classification using ultrasound images: a survey [J]. Pattern Recogn 43(1):299–317

    Google Scholar 

  2. Liu Y, Cheng HD, Huang JH, Zhang YT, Tang XL (2012) An effective approach of lesion segmentation within the breast ultrasound image based on cellular automata. J Digit Imaging 25(5):580–590

    PubMed  PubMed Central  Google Scholar 

  3. Cheng HD, Shi XJ, Min R, Hu LM, Cai XR, Du HN (2006) Approaches for automated detection and classification of masses in mammograms. Pattern Recogn 39:646–668

    Google Scholar 

  4. Liu Y, Cheng HD, Huang JH, Zhang YT, Tang XL, Tian JW, Wang H (2012) Computer-aided diagnosis system for breast cancer using B-mode and color Doppler flow images. Opt Eng 51(4):1–9

    CAS  Google Scholar 

  5. Liu B, Cheng HD, Huang JH, Tian JW, Tang XL, Liu JF (2010) Fully automatic and segmentation-robust classification of breast tumors based on local texture analysis of ultrasound images. Pattern Recogn 43:280–298

    Google Scholar 

  6. Zhang Y, Cheng HD, Tian J, Huang J, Tang X (2010) Fractional subpixel diffusion and fuzzy logic approach for ultrasound speckle reduction. Pattern Recogn 43(8):2962–2970

    Google Scholar 

  7. Zhang Y, Cheng HD, Chen Y, Huang J (2010) A novel noise removal method based on fractional anisotropic diffusion and subpixel approach. New Math Nat Comput 07(1):173–185

    Google Scholar 

  8. Liu Y, Cheng HD, Huang JH, Zhang YT, Tang XL, Tian JW, Wang Y (2012) Computer aided diagnosis system for breast cancer based on color doppler flow imaging. J Med Syst 36(6):3975–3982

    PubMed  Google Scholar 

  9. Liu Y, Cheng HD, Huang JH, Zhang YT, Tang XL, Tian YW (2013) An effective non-rigid registration approach for ultrasound image based on demons algorithm. J Digit Imaging 26(3):521–529

    CAS  PubMed  Google Scholar 

  10. Liu B, Cheng HD, Huang JH, Tian JW, Tang XL, Liu JF (2010) Probability density difference-based active contour for ultrasound image segmentation. Pattern Recogn 43:2028–2042

    Google Scholar 

  11. Xian M, Huang J, Zhang Y, Tang X (2012) Multiple-domain knowledge based MRF model for tumor segmentation in breast ultrasound images. IEEE ICIP:2021–2024

  12. Xian M, Zhang Y, Cheng HD (2015) Fully automatic segmentation of breast ultrasound images based on breast characteristics in space and frequency domains. Pattern Recogn 48:485–497

    Google Scholar 

  13. Chen CM, Lu HH, Lin YC (2000) An early vision-based snake model for ultrasound image segmentation. Ultrasound Med Biol 26(2):273–285

    CAS  PubMed  Google Scholar 

  14. Hua QH, Luo YZ, Zhang QZ (2017) Breast ultrasound image segmentation: a survey. Int J Comput Assist Radiol Surg 12(3):493–507

    Google Scholar 

  15. Noble JA (2010) Ultrasound image segmentation and tissue characterization. Proc Inst Mech Eng H J Eng Med 224(2):307–316

    CAS  Google Scholar 

  16. Moschidis E, Graham J (2010) A systematic performance evaluation of interactive image segmentation methods based on simulated user interaction. 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp 928–931

  17. Bakas S, Chatzimichail K, Hoppe A, Galariotis V, Hunter G, Makris D (2012) Histogram-based motion segmentation and characterisation of focal liver lesions in CEUS. Annals of the BMVA 2012(7):1–4

    Google Scholar 

  18. Bakas S, Chatzimichail K, Hunter G, Labbé B, Sidhu PS, Makris D (2017) Fast semi-automatic segmentation of focal liver lesions in contrast-enhanced ultrasound, based on a probabilistic model. Comput Methods Biomech Biomed Eng Imaging Vis 5(5):329–338

    Google Scholar 

  19. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. NIPS, pp:1097–1105

  20. Robinson AE, Hammon PS (2007) Explaining brightness illusions using spatial filtering and local response normalization. Vis Res 47(12):1631–1644

    PubMed  Google Scholar 

  21. Shao H, Zhang Y, Xian M, Cheng HD, Xu F, Ding J (2015) A saliency model for automated tumor detection in breast ultrasound images. IEEE ICIP, pp 1424–1428

  22. Shalev S, Singer YP (2011) Primal estimated sub-gradient solver for SVM. Math Program 127(1):3–30

    Google Scholar 

  23. Madabhushi A, Metaxas DN (2003) Combining low-, high-level and empirical domain knowledge for automated segmentation of ultrasonic breast lesions. IEEE Trans Med Imaging 22(2):155–169

    PubMed  Google Scholar 

  24. Shan J, Cheng HD, Wang YX (2012) Completely automated segmentation approach for breast ultrasound images using multiple-domain features. Ultrasound Med Biol 38:262–275

    PubMed  Google Scholar 

  25. Liu B, Cheng HD, Huang JH, Tian JW, Liu JF, Tang XL (2009) Automated segmentation of ultrasonic breast lesions using statistical texture classification and active contour based on probability distance. Ultrasound Med Biol 35(8):1309–1324

    PubMed  Google Scholar 

  26. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. IEEE Conference on Computer Vision and Pattern Recognition, IEEE Computer Society, pp 3431–3440

    Google Scholar 

  27. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. International Conference on Neural Information Processing Systems. Curran Associates Inc, pp 1097–1105

  28. Stavros AT, Thickman D, Rapp CL, Dennis MA, Parker SH, Sisney GA (1995) Solid breast nodules: use of sonography to distinguish between benign and malignant lesions. Radiology. 196(1):123–134

    CAS  PubMed  Google Scholar 

  29. Li J, Zhang Y, Tang X (2017) Automatic tumor segmentation via saliency detection in breast ultrasound images. ICPCSEE

    Google Scholar 

  30. Kaiming H, Georgia G, Piotr D, Ross G (2017) Mask R-CNN. Comput Vis Pattern Recognit:1–12

Download references

Funding

This work was supported by the program for Innovation Research of Harbin City (Grant No. 2017RALXJ006), Key Project of National Natural Science Foundation (Grant No. 81630048), and Open Project from the Key Laboratory of Intelligent Perception and Advanced Control of State Ethnic Affairs Commission (MD-IPAC-2019101).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Harbin Institute of Technology, No. 92, Xidazhi Street, Harbin, 150001, China

    Yingtao Zhang

  2. Department of Mathematics, College of Science, Harbin Institute of Technology, No. 92, Xidazhi Street, Harbin, 150001, China

    Yan Liu

  3. Department of Computer Science, Utah State University, Logan, UT, 84322, USA

    Hengda Cheng

  4. Second Affiliated Hospital of Harbin Medical University, Nangang, Harbin, China

    Ziyao Li & Cong Liu

Authors
  1. Yingtao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hengda Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ziyao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Cong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yan Liu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Highlights

For overcoming the deficiency of sample set, this paper chooses the AlexNet network structure as the basic network structure of the full convolution network and proposes to enhance the image and enlarge the size of dataset by using the wavelet transform and expanding the dataset by rotating and turning operation. In addition, this paper utilizes the idea of jumping structure in the convolution network to improve the original AlexNet structure, which is beneficial for representing the image details more accurately. Furthermore, the fully connected conditional random field is utilized to optimize convolutional network, which is beneficial for considering the spatial consistency and the relationship of pixels.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Liu, Y., Cheng, H. et al. Fully multi-target segmentation for breast ultrasound image based on fully convolutional network. Med Biol Eng Comput 58, 2049–2061 (2020). https://doi.org/10.1007/s11517-020-02200-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-020-02200-1

Keywords

Search

Navigation