Skip to main content
SpringerLink
Log in

Segmentation of ultrasound image sequences by combing a novel deep siamese network with a deformable contour model

  • S.I.: ML4BD SHS
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Deformable contours are widely applied in medical image segmentation, which are usually derived from appearance cues in medical images. However, the performance of deformed contour is suppressed in ultrasonic image segmentation by the weak, misleading boundaries and the complex shapes of lesion regions. In this paper, a novel deformable contour model is proposed for segmenting ultrasound image sequences, which aims to utilize the powerful ability of deep learning network in learning of image features to help the deformable contour model resist weaknessses of ultrasound images. The deep learning network is designed as a densely connected siamese architecture. It trains a contrastive loss that serves as a boundary searching metric of a deformable contour to segment ultrasound image sequences. In this network, the densely residual blocks and the attention focused blocks are designed to make the network efficiently propagate features and focus on the lesion region, and the feature memory module stores and generates the prior features to aid the evolution of a deformable contour. Moreover, for resisting the impact of misleading or weak boundary, the shape similarity of lesion regions is used to as a shape prior and integrated into the framework of deformable contour to constrain the change of contours. The experimental results for the clinical ultrasound image sequences demonstrate that compared to the state-of-the-art methods, the proposed method can provide more accurate results in HIFU ultrasound images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Kim J, Choi W, Park EY, Kang Y, Lee KJ, Kim HH, Kim WJ, Kim C (2019) Real-time photoacoustic thermometry combined with clinical ultrasound imaging and high-intensity focused ultrasound. IEEE Trans Biomed Eng 66(12):3330–3338

    Article  Google Scholar 

  2. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp 3431–3440

  3. Hu Y, Soltoggio A, Lock R, Carter S (2019) A fully convolutional two-stream fusion network for interactive image segmentation. Neural Netw:Off J Int Neural Netw Soc 109:31–42

    Article  Google Scholar 

  4. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing & Computer-Assisted Intervention

  5. Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M, Misawa K, Mori K, McDonagh S, Hammerla NY, Kainz B, Glocker B, Rueckert D (2018) Attention u-net: Learning where to look for the pancreas. arXiv:1804.03999

  6. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2019) Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE Transactions on Medical Imaging

  7. Li H, Fang J, Liu S, Liang X, Yang X, Mai Z, Van MT, Wang T, Chen Z, Ni D (2019) Cr-unet: A composite network for ovary and follicle segmentation in ultrasound images. IEEE J Biomed Health Inform 24:974–983

    Article  Google Scholar 

  8. Marcos D, Tuia D, Kellenberger B, Zhang L, Bai M, Liao R, Urtasun R (2018) Learning deep structured active contours end-to-end. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition pp 8877–8885

  9. Chen X, Williams BM, Vallabhaneni SR, Czanner G, Williams RS, Zheng Y (2019) Learning active contour models for medical image segmentation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp 11624–11632

  10. Fang Z, Qiao M, Guo Y, Wang Y, Li J, Zhou S, Chang C (2019) Combining a fully convolutional network and an active contour model for automatic 2d breast tumor segmentation from ultrasound images. J Med Imaging Health Informatics 9:1510–1515

    Article  Google Scholar 

  11. Hu Y, Guo Y, Wang Y, Yu J, Li J, Zhou S, Chang C (2019) Automatic tumor segmentation in breast ultrasound images using a dilated fully convolutional network combined with an active contour model. Med Phys 46:215–228

    Article  Google Scholar 

  12. Xu C, Prince JL (1998) Snakes, shapes, and gradient vector flow. IEEE Trans Image Process 7(3):359–69

    Article  MathSciNet  MATH  Google Scholar 

  13. Paragios N, Deriche R (2000) Geodesic active contours and level sets for the detection and tracking of moving objects. IEEE Trans Pattern Anal Mach Intell 22(3):266–280

    Article  Google Scholar 

  14. Guo Y, Şengür A, Tian JW (2016) A novel breast ultrasound image segmentation algorithm based on neutrosophic similarity score and level set. Comput Methods Programs Biomed 123:43–53

    Article  Google Scholar 

  15. Zhao Y, Zhao J, Yang J, Liu Y, Zhao Y, Zheng Y, Xia L, Wang Y (2017) Saliency driven vasculature segmentation with infinite perimeter active contour model. Neurocomputing 259:201–209

    Article  Google Scholar 

  16. Radoglou-Grammatikis P, Robolos K, Sarigiannidis P, Argyriou V, Lagkas T, Sarigiannidis A, Goudos SK, Wan S (2021) Modelling, detecting and mitigating threats against industrial healthcare systems: a combined sdn and reinforcement learning approach. IEEE Transactions on Industrial Informatics

  17. Wan S, Xia Y, Qi L, Yang YH, Atiquzzaman M (2020) Automated colorization of a grayscale image with seed points propagation. IEEE Transactions on Multimedia

  18. Zhou S, Wang J, Zhang S, Liang Y, Gong Y (2016) Active contour model based on local and global intensity information for medical image segmentation. Syst Eng Electron 186((C)):107–118

    Google Scholar 

  19. Yu H, He F, Pan Y (2018) A novel region-based active contour model via local patch similarity measure for image segmentation. Multimedia Tools Appl 3:1–23

    Google Scholar 

  20. Wang, L., Li, M., Fang, X., Nappi, M., & Wan, S. (2022). Improving random walker segmentation using a nonlocal bipartite graph. Biomedical Signal Processing and Control, 71, 103154.

  21. Gao Y, Bouix S, Shenton ME, Tannenbaum AR (2013) Sparse texture active contour. IEEE Trans Image Process 22:3866–3878

    Article  MathSciNet  MATH  Google Scholar 

  22. Yu J, Tan M, Zhang H, Tao D, Rui Y (2019) Hierarchical deep click feature prediction for fine-grained image recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence

  23. Zhao Y, Li H, Wan S, Sekuboyina A, Hu X, Tetteh G, Piraud M, Menze B (2019) Knowledge-aided convolutional neural network for small organ segmentation. IEEE J Biomed Health Inform 23(4):1363–1373

    Article  Google Scholar 

  24. Fang J, Liu H, Liu J, Zhou H, Liu H (2021) Fuzzy region-based active contour driven by global and local fitting energy for image segmentation. Appl Soft Comput 100:106982

    Article  Google Scholar 

  25. Subudhi P, Mukhopadhyay S (2021) A statistical active contour model for interactive clutter image segmentation using graph cut optimization. Signal Process 4:108056

    Article  Google Scholar 

  26. Van Ginneken B, Frangi AF, Staal JJ, Haar TBM, Romeny VMA (2002) Active shape model segmentation with optimal features. IEEE Trans Med Imaging 21(8):924–933

    Article  Google Scholar 

  27. Zhang S, Zhan Y, Dewan M, Huang J, Metaxas DN, Zhou XS (2012) Towards robust and effective shape modeling: Sparse shape composition. Med Image Anal 16(1):265–277

    Article  Google Scholar 

  28. Huang X, Dione DP, Compas CB, Papademetris X, Lin BA, Bregasi A, Sinusas AJ, Staib LH, Duncan JS (2014) Contour tracking in echocardiographic sequences via sparse representation and dictionary learning. Med Image Anal 18(2):253–271

    Article  Google Scholar 

  29. Korez R, Ibragimov B, Likar B, Pernuš F, Vrtovec T (2015) A framework for automated spine and vertebrae interpolation-based detection and model-based segmentation. IEEE Trans Med Imaging 34(8):1649–1662

    Article  Google Scholar 

  30. Ni B, He F, Yuan ZY (2015) Segmentation of uterine fibroid ultrasound images using a dynamic statistical shape model in hifu therapy. Comput Med Imaging Graph 46(3):302–314

    Article  Google Scholar 

  31. Wang, H., Zhang, D., Ding, S. et al. Rib segmentation algorithm for X-ray image based on unpaired sample augmentation and multi-scale network. Neural Comput & Applic (2021). https://doi.org/10.1007/s00521-021-06546-x

    Google Scholar 

  32. Wachinger C, Yigitsoy M, Rijkhorst EJ, Navab N (2012) Manifold learning for image-based breathing gating in ultrasound and mri. Med Image Anal 16(4):806–818

    Article  Google Scholar 

  33. Gao Z, Li Y, Wan S (2020) Exploring deep learning for view-based 3d model retrieval. ACM Trans Multimed Comput Commun Appl 16(1):1–21

    Article  Google Scholar 

  34. Zhou X, Huang X, Duncan JS, Yu W (2013) Active contours with group similarity. 2013 IEEE Conference on Computer Vision and Pattern Recognition pp 2969–2976

  35. Ni B, He F, teng Pan Y, Yuan Z (2016) Using shapes correlation for active contour segmentation of uterine fibroid ultrasound images in computer-aided therapy. Appl Math-A J Chin Univ 31:37–52

    Article  MathSciNet  MATH  Google Scholar 

  36. Gamarra M (2020) Convexity shape constraints for retinal blood vessel segmentation and foveal avascular zone detection. Computers in Biology and Medicine 127

  37. Yan S, Tai XC, Liu J, Huang HY (2020) Convexity shape prior for level set based image segmentation method. IEEE Transactions on Image Processing PP(99):1–1

  38. Milletari F, Navab N, Ahmadi SA (2016) V-net: Fully convolutional neural networks for volumetric medical image segmentation. 2016 Fourth International Conference on 3D Vision (3DV) pp 565–571

  39. Alom MZ, Yakopcic C, Hasan M, Taha T, Asari V (2019) Recurrent residual u-net for medical image segmentation. J Med Imaging 6:014006–014006

    Article  Google Scholar 

  40. Yu J, Li J, Yu Z, Huang Q (2019) Multimodal transformer with multi-view visual representation for image captioning. IEEE Transactions on Circuits and Systems for Video Technology PP(99):1–1

  41. Ding S, Qu S, Xi Y, Wan S (2019) Stimulus-driven and concept-driven analysis for image caption generation. Neurocomputing

  42. Huang G, Liu Z, Weinberger KQ (2017) Densely connected convolutional networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp 2261–2269

  43. Yang W, Zhou Q, Lu J, Wu X, Zhang S, Latecki L (2018) Dense deconvolutional network for semantic segmentation. 2018 25th IEEE International Conference on Image Processing (ICIP) pp 1573–1577

  44. Li H, He X, Zhou F, Yu Z, Ni D, Chen S, Wang T, Lei B (2019) Dense deconvolutional network for skin lesion segmentation. IEEE J Biomed Health Inform 23:527–537

    Article  Google Scholar 

  45. Park B, Yu S, Jeong J (2019) Densely connected hierarchical network for image denoising. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) pp 2104–2113

  46. Yuan Y, Qin W, Guo X, Buyyounouski M, Hancock SH, Han B, Xing L (2019) Prostate segmentation with encoder-decoder densely connected convolutional network (ed-densenet). 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) pp 434–437

  47. Minnema J, Wolff J, Koivisto J, Lucka F, Batenburg KJ, Forouzanfar T, Eijnatten M (2021) Comparison of convolutional neural network training strategies for cone-beam ct image segmentation. Comput Methods Prog Biomed 207:106192

    Article  Google Scholar 

  48. Fu J, Liu J, Tian H, Fang Z, Lu H (2019) Dual attention network for scene segmentation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp 3141–3149

  49. Wang G, Li W, Zuluaga MA, Pratt R, Patel PA, Aertsen M, Doel T, David AL, Deprest J, Ourselin S et al (2018) Interactive medical image segmentation using deep learning with image-specific fine tuning. IEEE Trans Med Imaging 37(7):1562–1573

    Article  Google Scholar 

  50. Shan H, Zhang Y, Yang Q, Kruger U, Kalra MK, Sun L, Cong W, Wang G (2018) 3-d convolutional encoder-decoder network for low-dose ct via transfer learning from a 2-d trained network. IEEE Trans Med Imaging 37(6):1522

    Article  Google Scholar 

  51. Opbroek AV, Achterberg HC, Vernooij MW, Bruijne MD (2018) Transfer learning for image segmentation by combining image weighting and kernel learning. IEEE Transactions on Medical Imaging PP(99):1–1

  52. Han D, Liu Q, Fan W (2018) A new image classification method using cnn transfer learning and web data augmentation. Expert Syst Appl 95:43–56

    Article  Google Scholar 

  53. Yu J, Zhu C, Zhang J, Huang Q, Tao D (2019) Spatial pyramid-enhanced netvlad with weighted triplet loss for place recognition. IEEE Transactions on Neural Networks and Learning Systems

  54. Dong N, Trullo R, Lian J, Li W, Petitjean C, Su R, Qian W, Shen D (2018) Medical image synthesis with deep convolutional adversarial networks. IEEE Transactions on Biomedical Engineering PP(99):1–1

  55. Wolterink JM, Kamnitsas K, Ledig C, Išgum I (2018) Generative adversarial networks and adversarial methods in biomedical image analysis. arXiv preprint arXiv:1810.10352

  56. Yu B, Zhou L, Wang L, Shi Y, Fripp J, Bourgeat P (2019) Ea-gans: Edge-aware generative adversarial networks for cross-modality mr image synthesis. IEEE Transactions on Medical Imaging PP(99):1–1

  57. Li Y, Chen Y, Shi Y (2020) Brain tumor segmentation using 3d generative adversarial networks. International Journal of Pattern Recognition and Artificial Intelligence

  58. Lei B, Xia Z, Jiang F, Jiang X, Wang S (2020) Skin lesion segmentation via generative adversarial networks with dual discriminators. Med Image Anal 64:101716

    Article  Google Scholar 

  59. Kugelman J, Alonso-Caneiro D, Read SA, Vincent SJ, Collins MJ (2021) Data augmentation for patch-based oct chorio-retinal segmentation using generative adversarial networks. Neural Computing and Applications (4)

  60. Zagoruyko S, Komodakis N (2015) Learning to compare image patches via convolutional neural networks. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp 4353–4361

  61. Cai Q, Pan Y, Yao T, Yan CC, Mei T (2018) Memory matching networks for one-shot image recognition. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition pp 4080–4088

  62. Li Z, Kamnitsas K, Glocker B (2021) Analyzing overfitting under class imbalance in neural networks for image segmentation. IEEE Trans Med Imaging 40(3):1065–1077. https://doi.org/10.1109/TMI.2020.3046692

    Article  Google Scholar 

  63. Kamran SA, Hossain KF, Tavakkoli A, Zuckerbrod SL, Sanders KM, Baker SA (2021) Rv-gan : Retinal vessel segmentation from fundus images using multi-scale generative adversarial networks

  64. Petit O, Thome N, Rambour C, Soler L (2021) U-net transformer: Self and cross attention for medical image segmentation

  65. Valanarasu JMJ, Oza P, Hacihaliloglu I, Patel VM (2021) Medical transformer: Gated axial-attention for medical image segmentation. arXiv:2102.10662

  66. He K, Zhang X, Ren S, Sun J (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–16

    Article  Google Scholar 

  67. Wold S, Esbensen KH, Geladi P (1987) Principal component analysis

  68. Ye Y, He C (2012) Adaptive active contours without edges. Math Comput Model 55(5–6):1705–1721

    MathSciNet  MATH  Google Scholar 

  69. Huang Y, Yan HY, Wen YW, Yang X (2018) Rank minimization with applications to image noise removal. Inf Sci 429:147–163

    Article  MathSciNet  MATH  Google Scholar 

  70. Beck A, Teboulle M (2009) A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imaging Sci 2:183–202

    Article  MathSciNet  MATH  Google Scholar 

  71. Cai J, Candès E, Shen Z (2010) A singular value thresholding algorithm for matrix completion. SIAM J Optim 20:1956–1982

    Article  MathSciNet  MATH  Google Scholar 

  72. Hermans A, Beyer L, Leibe B (2017) In defense of the triplet loss for person re-identification. arXiv:1703.07737

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (No.62172438), the fundamental research funds for the central universities (31412111303, 31512111310) and by the open project from the State Key Laboratory for Novel Software Technology, Nanjing University, under Grant No.KFKT2019B17.

Author information

Authors and Affiliations

  1. School of Computer Science, Wuhan University, Wuhan, 430072, People’s Republic of China

    Bo Ni & Xiantao Cai

  2. School of Computer Science, Hubei Polytechnic University, Huangshi, 435003, People’s Republic of China

    Bo Ni & Zhiyuan Liu

  3. The University of Salerno, Fisciano, Italy

    Michele Nappi

  4. Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen, 518110, People’s Republic of China

    Shaohua Wan

Authors
  1. Bo Ni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhiyuan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiantao Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michele Nappi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shaohua Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shaohua Wan.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interests regarding the publication of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ni, B., Liu, Z., Cai, X. et al. Segmentation of ultrasound image sequences by combing a novel deep siamese network with a deformable contour model. Neural Comput & Applic 35, 14535–14549 (2023). https://doi.org/10.1007/s00521-022-07054-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07054-2

Keywords

Search

Navigation