Skip to main content
SpringerLink
Log in

Feature decomposition and enhancement for unsupervised medical ultrasound image denoising and instance segmentation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In this paper, we study how to denoise medical ultrasound images and improve the performance of instance segmentation using deep learning technology. Since medical ultrasound images usually contain a lot of noises, we first propose a novel unsupervised learning approach called Dual Image (DI) for denoising of medical ultrasound images. DI consists of three main features. Firstly, unlike many existing supervised denoising methods, it does not need clean medical ultrasound images for denoising. Instead, it uses Computed Tomography (CT) images and the noise patches extracted from medical ultrasound images for denoising. Secondly, to effectively select noise patches from medical ultrasound images, a patch selection algorithm based on entropy is formulated. Thirdly, to minimize structure variation of denoised medical ultrasound images, a new reconstruction block is designed for combining the structural information from the structural enhancement block. After denoising, since medical ultrasound images are usually poor in features, to further improve the instance segmentation performance, we extend SOLOv2 to Segmenting on Ultrasound Image (SOUI) by proposing the Double Feature Pyramid Network (D-FPN) and mask fusion branch to strengthen the communication and fusion of different feature layers. Extensive experiments have been performed to study the performance of DI and SOUI using practical medical ultrasound images. We demonstrate that DI can greatly improve the quality of medical ultrasound images and minimize structure variation of denoised medical ultrasound images. SOUI gives 53.7% AP(Average Precision) on practical medical ultrasound images, and outperforms most the state-of-the-art instance segmentation methods including SOLOv2. Code is available at: https://github.com/ztt0821/SOUI.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. A noisy-clean pair consists of two images which have the same contents but one of the images has no noise while the other image contains noises.

References

  1. Alkhatib M, Hafiane A, Tahri O, Vieyres P, Delbos A (2018) Adaptive median binary patterns for fully automatic nerves tracking in ultrasound images. Comput Methods Prog Biomed 160:129–140

    Article  Google Scholar 

  2. Hadjerci O, Hafiane A, Morette N, Novales C, Vieyres P, Delbos A (2016) Assistive system based on nerve detection and needle navigation in ultrasound images for regional anesthesia. Expert Syst Appl 61:64–77

    Article  Google Scholar 

  3. Horng M-H, Yang C-W, Sun Y-N, Yang T-H (2020) Deepnerve: a new convolutional neural network for the localization and segmentation of the median nerve in ultrasound image sequences. Ultrasound Med Biol 46(9):2439–2452

    Article  Google Scholar 

  4. He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 2961–2969

  5. Huang Z, Huang L, Gong Y, Huang C, Wang X (2019) Mask scoring r-cnn. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6409–6418

  6. Chen K, Pang J, Wang J, Xiong Y, Li X, Sun S, Feng W, Liu Z, Shi J, Ouyang W, Loy C C, Lin D (2019) Hybrid task cascade for instance segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4974–4983

  7. Bolya D, Zhou C, Xiao F, Lee Y J (2019) Yolact: Real-time instance segmentation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 9157–9166

  8. Chen H, Sun K, Tian Z, Shen C, Huang Y, Yan Y (2020) Blendmask: top-down meets bottom-up for instance segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8573–8581

  9. Tian Z, Shen C, Chen H (2020) Conditional convolutions for instance segmentation. In: Computer Vision–ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16. Springer, pp 282–298

  10. Wang X, Kong T, Shen C, Jiang Y, Li L (2020) Solo: Segmenting objects by locations. In: Proceedings of the European conference on computer vision, pp 649–665

  11. Wang X, Zhang R, Kong T, Li L, Shen C (2020) Solov2: dynamic and fast instance segmentation. Adv Neural Inf Process Syst 33:17721–17732

    Google Scholar 

  12. Yousuf MA, Nobi MN (2011) A new method to remove noise in magnetic resonance and ultrasound images. J Sci Res 3(1): 81–81

    Article  Google Scholar 

  13. Zhu L, Fu C-W, Brown M S, Heng P-A (2017) A non-local low-rank framework for ultrasound speckle reduction. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5650–5658

  14. González J G, Alvarez M A, Orozco A A (2015) Automatic segmentation of nerve structures in ultrasound images using graph cuts and gaussian processes. In: Proceedings of the 37th annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 3089–3092

  15. Liu C, Liu F, Wang L, Ma L, Lu Z-M (2018) Segmentation of nerve on ultrasound images using deep adversarial network. Int J Innov Comput Inf Control 14:53–64

    Google Scholar 

  16. Wang R, Shen H, Zhou M (2019) Ultrasound nerve segmentation of brachial plexus based on optimized resu-net. In: 2019 IEEE international conference on imaging systems and techniques (IST), pp 1–6

  17. Lu X, Wang W, Shen J, Crandall D, Van Gool L (2021) Segmenting objects from relational visual data. IEEE Trans Pattern Anal Mach Intell 1. https://doi.org/10.1109/TPAMI.2021.3115815

  18. Lu X, Wang W, Ma C, Shen J, Shao L, Porikli F (2019) See more, know more: unsupervised video object segmentation with co-attention siamese networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3623–3632

  19. Liu S, Jia J, Fidler S, Urtasun R (2017) Sgn: Sequential grouping networks for instance segmentation. In: Proceedings of the IEEE international conference on computer vision, pp 3496–3504

  20. Gao N, Shan Y, Wang Y, Zhao X, Yu Y, Yang M, Huang K (2019) Ssap: Single-shot instance segmentation with affinity pyramid. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 642–651

  21. Michailovich O V, Tannenbaum A (2006) Despeckling of medical ultrasound images. IEEE Trans Ultrason Ferroelectr Freq Control 53(1):64–78

    Article  Google Scholar 

  22. Gai S, Zhang B, Yang C, Yu L (2018) Speckle noise reduction in medical ultrasound image using monogenic wavelet and laplace mixture distribution. Digit Signal Process 72:192–207

    Article  Google Scholar 

  23. Chen Z, Zeng Z, Shen H, Zheng X, Dai P, Ouyang P (2020) Dn-gan: Denoising generative adversarial networks for speckle noise reduction in optical coherence tomography images. Biomed Signal Process Control 55:101632

    Article  Google Scholar 

  24. Dong Z, Liu G, Ni G, Jerwick J, Duan L, Zhou C (2020) Optical coherence tomography image denoising using a generative adversarial network with speckle modulation. J Biophotonics 13 (4):e201960135

    Article  Google Scholar 

  25. Huang Y, Xia W, Lu Z, Liu Y, Chen H, Zhou J, Fang L, Zhang Y (2020) Noise-powered disentangled representation for unsupervised speckle reduction of optical coherence tomography images. IEEE Trans Med Imaging 40(10):2600–2614

    Article  Google Scholar 

  26. Parrilli S, Poderico M, Angelino C V, Verdoliva L (2011) A nonlocal sar image denoising algorithm based on llmmse wavelet shrinkage. IEEE Trans Geosci Remote Sens 50(2):606–616

    Article  Google Scholar 

  27. Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27(3):379–423

    Article  MathSciNet  MATH  Google Scholar 

  28. Feng X, Ji H, Jiang B, Pei W, Chen F, Lu G (2021) Contrastive feature decomposition for image reflection removal. In: Proceedings of the IEEE international conference on multimedia and expo (ICME), pp 1–6

  29. Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, Qiao Y, Change Loy C (2018) Esrgan: enhanced super-resolution generative adversarial networks. In: Proceedings of the European conference on computer vision (ECCV) workshops

  30. Wada K (2016) labelme: image polygonal annotation with python. https://github.com/wkentaro/labelme

  31. Leuschner J, Schmidt M, Baguer D O, Maass P (2021) Lodopab-ct, a benchmark dataset for low-dose computed tomography reconstruction. Sci Data 8(1):1–12

    Article  Google Scholar 

  32. Chen K, Wang J, Pang J, Cao Y, Xiong Y, Li X, Sun S, Feng W, Liu Z, Xu J, Zhang Z, Cheng D, Zhu C, Cheng T, Zhao Q, Li B, Lu X, Zhu R, Wu Y, Dai J, Wang J, Shi J, Ouyang W, Loy C C, Lin D (2019) MMDetection: open mmlab detection toolbox and benchmark. arXiv:1906.07155

  33. Zou Z, Lei S, Shi T, Shi Z, Ye J (2020) Deep adversarial decomposition: a unified framework for separating superimposed images. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12806–12816

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, City University of Hong Kong, Tat chee Avenue, Hong Kong, 999077, Hong Kong SAR, China

    Tian-Tian Zhang & Kam-Yiu Lam

  2. Department of Ultrasound, The First Affiliated Hospital of Nanjing Medical University, Guangzhou Road, Nan Jing, 210029, Jiang Su, China

    Hua Shu & Ao Li

  3. Social Mind Analytics (Research and Technology) Limited, Hoi Shing Road, Hong Kong, 999077, Hong Kong SAR, China

    Chi-Yin Chow

Authors
  1. Tian-Tian Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hua Shu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kam-Yiu Lam
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chi-Yin Chow
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Kam-Yiu Lam, Chi-Yin Chow or Ao Li.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, TT., Shu, H., Lam, KY. et al. Feature decomposition and enhancement for unsupervised medical ultrasound image denoising and instance segmentation. Appl Intell 53, 9548–9561 (2023). https://doi.org/10.1007/s10489-022-03857-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03857-x

Keywords

Search

Navigation