Skip to main content
SpringerLink
Log in

Automatic coarse-to-refinement-based ultrasound prostate segmentation using optimal polyline segment tracking method and deep learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Automatic segmentation of the prostate in transrectal ultrasound (TRUS) images provides useful information for prostate cancer diagnosis and treatment. However, boundaries between the prostate and other tissues are often absent or ill defined in TRUS images, which means that automatic segmentation of the prostate in TRUS images is highly challenging. In this study, we attempted to overcome these challenges by developing a novel method we termed “automatic prostate segmentation” (Auto-ProSeg) that is capable of effectively segmenting the prostate in TRUS images. Auto-ProSeg comprises two steps: the first step is a preprocessing step that uses attention U-Net to extract approximate prostate contours automatically; then, in the second step, the approximate prostate contours are optimized via a modified principal curve-based method linked to an evolutionary neural network, whereby a mathematical mapping formula based on the parameters of an enhanced evolutionary neural network is used to generate smooth prostate contours. Our results illustrate that Auto-ProSeg exhibits better prostate segmentation performance than other recently developed methods: the average Dice similarity coefficient and Jaccard similarity coefficient (Ω) of Auto-ProSeg-generated prostate contours against ground truths were 94.2% ± 3.2% and 93% ± 3.7%, whereas those for other state-of-the-art fully automatic segmentation methods were approximately 90% ± 5% and 89% ± 6%, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

Data will be made available on reasonable request.

Notes

  1. Unlike the K-segments polyline segment tracking (KPST) method [27], the closed polyline segment tracking (CPST) method that we have previously devised [28,29,30] has several constraint conditions and is the first method to do so. Moreover, we developed the hybrid prostate segmentation (H-ProSeg) [23] method, which uses vertex cleaning conditions to control the influence of abnormal data.

References

  1. Goldenberg SL, Nir G, Salcudean SE (2019) A new era: artificial intelligence and machine learning in prostate cancer. Nat Rev Urol 16:391–403

    Google Scholar 

  2. Xu X, Sanford T, Turkbey B et al (2022) Shadow-consistent semi-supervised learning for prostate ultrasound segmentation. IEEE Trans Med Imaging 41:1331–1345

    Google Scholar 

  3. Jiang J, Guo Y, Bi Z et al (2023) Segmentation of prostate ultrasound images: the state of the art and the future directions of segmentation algorithms. Artif Intell Rev 56:615–651

    Google Scholar 

  4. Chen G, Dai Y, Li R et al (2021) SDFNet: automatic segmentation of kidney ultrasound images using multi-scale low-level structural feature. Expert Syst Appl 185:115619

    Google Scholar 

  5. Peng T, Zhao J, Wang J (2021) Interpretable mathematical model-guided ultrasound prostate contour extraction using data mining techniques. In: IEEE 15th International Conference on Bioinformatics and Biomedicine (BIBM), pp 1037–1044

  6. He K, Lian C, Adeli E et al (2021) MetricUNet: synergistic image- and voxel-level learning for precise prostate segmentation via online sampling. Med Image Anal 71:102039

    Google Scholar 

  7. Chen G, Li L, Dai Y et al (2022) AAU-net: an adaptive attention U-net for breast lesions segmentation in ultrasound images. IEEE Trans Med Imaging:1–1

  8. Liu Z, Yang C, Huang J et al (2021) Deep learning framework based on integration of S-mask R-CNN and inception-v3 for ultrasound image-aided diagnosis of prostate cancer. Futur Gener Comput Syst 114:358–367

    Google Scholar 

  9. Chen G, Dai Y, Zhang J (2023) RRCNet: refinement residual convolutional network for breast ultrasound images segmentation. Eng Appl Artif Intell 117:105601

    Google Scholar 

  10. Peng T, Gu Y, Ye Z et al (2022) A-LugSeg: automatic and explainability-guided multi-site lung detection in chest X-ray images. Expert Syst Appl 198:116873

    Google Scholar 

  11. Peng T, Wang C, Zhang Y, Wang J (2022) H-SegNet: hybrid segmentation network for lung segmentation in chest radiographs using mask region-based convolutional neural network and adaptive closed polyline searching method. Phys Med Biol 67:075006

    Google Scholar 

  12. Shahedi M, Halicek M, Guo R et al (2018) A semiautomatic segmentation method for prostate in CT images using local texture classification and statistical shape modeling. Med Phys 45:2527–2541

    Google Scholar 

  13. Anas EMA, Mousavi P, Abolmaesumi P (2018) A deep learning approach for real time prostate segmentation in freehand ultrasound guided biopsy. Med Image Anal 48:107–116

    Google Scholar 

  14. Kervadec H, Dolz J, Tang M et al (2019) Constrained-CNN losses for weakly supervised segmentation. Med Image Anal 54:88–99

    Google Scholar 

  15. Zhang T-T, Shu H, Lam K-Y et al (2023) Feature decomposition and enhancement for unsupervised medical ultrasound image denoising and instance segmentation. Appl Intell 53:9548–9561

  16. Bi H, Jiang Y, Tang H et al (2020) Fast and accurate segmentation method of active shape model with Rayleigh mixture model clustering for prostate ultrasound images. Comput Methods Prog Biomed 184:105097

    Google Scholar 

  17. Jia X, Ren L, Cai J (2020) Clinical implementation of AI technologies will require interpretable AI models. Med Phys 47:1–4

    Google Scholar 

  18. Xing L, Krupinski EA, Cai J (2018) Artificial intelligence will soon change the landscape of medical physics research and practice. Med Phys 45:1791–1793

    Google Scholar 

  19. Orlando N, Gyacskov I, Gillies DJ et al (2022) Effect of dataset size, image quality, and image type on deep learning-based automatic prostate segmentation in 3D ultrasound. Phys Med Biol 67:074002

    Google Scholar 

  20. Vesal S, Gayo I, Bhattacharya I et al (2022) Domain generalization for prostate segmentation in transrectal ultrasound images: a multi-center study. Med Image Anal 82:102620

    Google Scholar 

  21. Lei Y, Wang T, Roper J et al (2021) Male pelvic multi-organ segmentation on transrectal ultrasound using anchor-free mask CNN. Med Phys 48:3055–3064

    Google Scholar 

  22. Liu Q, Dou Q, Heng P-A (2020) Shape-aware meta-learning for generalizing prostate MRI segmentation to unseen domains. In: International conference on medical image computing and computer-assisted intervention, pp 475–485

  23. Peng T, Wu Y, Qin J et al (2022) H-ProSeg: hybrid ultrasound prostate segmentation based on explainability-guided mathematical model. Comput Methods Prog Biomed 219:106752

    Google Scholar 

  24. Xu X, Sanford T, Turkbey B et al (2022) Polar transform network for prostate ultrasound segmentation with uncertainty estimation. Med Image Anal 78:102418

    Google Scholar 

  25. Oktay O, Schlemper J, Folgoc LL, et al (2018) Attention U-Net: learning where to look for the pancreas. In: Medical Imaging with Deep Learning (MIDL)

  26. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention – MICCAI 2015. Springer International Publishing, Cham, pp 234–241

    Google Scholar 

  27. Kégl B, Linder T, Zeger K (2000) Learning and design of principal curves. IEEE Trans Pattern Anal Machine Intell 22:281–297

    Google Scholar 

  28. Peng T, Wang Y, Xu TC et al (2018) Detection of lung contour with closed principal curve and machine learning. J Digit Imaging 31:520–533

    Google Scholar 

  29. Peng T, Xu TC, Wang Y et al (2020) Hybrid automatic lung segmentation on chest CT scans. IEEE Access 8:73293–73306

    Google Scholar 

  30. Peng T, Xu TC, Wang Y, Li F (2020) Deep belief network and closed polygonal line for lung segmentation in chest radiographs. Comput J 65:1107–1128

  31. Zhou H, Zhao H, Zhang Y (2020) Nonlinear system modeling using self-organizing fuzzy neural networks for industrial applications. Appl Intell 50:1657–1672

    Google Scholar 

  32. Ali MZ, Awad NH, Suganthan PN, Reynolds RG (2017) An adaptive multipopulation differential evolution with dynamic population reduction. IEEE Trans Cybern 47:2768–2779

    Google Scholar 

  33. Laredo JLJ, Fernandes C, Merelo JJ, Gagné C (2009) Improving genetic algorithms performance via deterministic population shrinkage. In: Proceedings of the 11th Annual conference on Genetic and evolutionary computation, Montreal, Canada, p 819

  34. He K, Gkioxari G, Dollar P, Girshick R (2017) Mask R-CNN. In: Proceedings of the IEEE international conference on computer Vision. Venice, Italy, pp 2961–2969

  35. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2020) Unet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imaging 39:1856–1867

    Google Scholar 

  36. Gao Y, Zhou M, Metaxas D (2021) UTNet: a hybrid transformer architecture for medical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, pp 61–71

    Google Scholar 

  37. Peng T, Tang C, Wang J (2022) Prostate segmentation of ultrasound images based on interpretable-guided mathematical model. In: International conference on multimedia modeling (MMM). Springer, pp 166–177

    Google Scholar 

  38. Nam H, Kim H-E (2018) Batch-instance normalization for adaptively style-invariant neural networks. In: Advances in neural information processing systems

  39. Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In: ICML workshop on deep learning for audio, speech and language processing

  40. Hara K, Saito D, Shouno H (2015) Analysis of function of rectified linear unit used in deep learning. In: 2015 International Joint Conference on Neural Networks (IJCNN), pp 1–8

  41. Zhang J, Chen D, Kruger U (2008) Adaptive constraint K-segment principal curves for intelligent transportation systems. IEEE Trans Intell Transport Syst 9:666–677

    Google Scholar 

  42. Kabir W, Ahmad MO, Swamy MNS (2015) A novel normalization technique for multimodal biometric systems. In: 2015 IEEE 58th International Midwest Symposium on Circuits and Systems (MWSCAS). IEEE, Fort Collins, pp 1–4

  43. Chen P (2019) Effects of normalization on the entropy-based TOPSIS method. Expert Syst Appl 136:33–41

    Google Scholar 

  44. Storn R (1997) Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11:341

  45. Parouha RP, Verma P (2022) A systematic overview of developments in differential evolution and particle swarm optimization with their advanced suggestion. Appl Intell 52:10448–10492

    Google Scholar 

  46. Zeng Y-R, Zeng Y, Choi B, Wang L (2017) Multifactor-influenced energy consumption forecasting using enhanced back-propagation neural network. Energy 127:381–396

    Google Scholar 

  47. Peng T, Zhao J, Gu Y et al (2022) H-ProMed: ultrasound image segmentation based on the evolutionary neural network and an improved principal curve. Pattern Recogn 131:108890

    Google Scholar 

  48. Peng T, Tang C, Wu Y, Cai J (2022) H-SegMed: a hybrid method for prostate segmentation in TRUS images via improved closed principal curve and improved enhanced machine learning. Int J Comput Vis 130:1896–1919

    Google Scholar 

  49. Peng T, Gu Y, Wang J (2021) Lung contour detection in chest X-ray images using mask region-based convolutional neural network and adaptive closed polyline searching method. In: 2021 43rd Annual International Conference of the IEEE Engineering in Medicine Biology Society (EMBC), pp 2839–2842

  50. Kegl B, Krzyzak A (2002) Piecewise linear skeletonization using principal curves. IEEE Trans Pattern Anal Machine Intell 24:59–74

    Google Scholar 

  51. Zhu Q, Du B, Yan P (2020) Boundary-weighted domain adaptive neural network for prostate MR image segmentation. IEEE Trans Med Imaging 39:753–763

    Google Scholar 

  52. Lei Y, Tian S, He X et al (2019) Ultrasound prostate segmentation based on multidirectional deeply supervised V-net. Med Phys 46:3194–3206

    Google Scholar 

  53. Peng T, Wang Y, Xu TC, Chen X (2019) Segmentation of lung in chest radiographs using Hull and closed polygonal line method. IEEE Access 7:137794–137810

    Google Scholar 

  54. Gu R, Wang G, Song T et al (2021) CA-net: comprehensive attention convolutional neural networks for explainable medical image segmentation. IEEE Trans Med Imaging 40:699–711

    Google Scholar 

  55. Floridi L (2019) Establishing the rules for building trustworthy AI. Nat Mach Intell 1:261–262

    Google Scholar 

  56. Liang W, Tadesse GA, Ho D et al (2022) Advances, challenges and opportunities in creating data for trustworthy AI. Nat Mach Intell 4:669–677

    Google Scholar 

Download references

Acknowledgments

Professional English language editing support is provided by AsiaEdit (asiaedit.com). The author would like to thank Dr. Jin Wang for checking and editing the manuscript.

Author information

Author notes

  1. Tao Peng, Daqiang Xu, Caiyin Tang, Jing Zhao and Yuntian Shen contributed equally to this work.

Authors and Affiliations

  1. School of Future Science and Engineering, Soochow University, Suzhou, China

    Tao Peng & Cong Yang

  2. Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Hong Kong, China

    Tao Peng & Jing Cai

  3. Department of Radiation Oncology, UT Southwestern Medical Center, Dallas, TX, USA

    Tao Peng

  4. Department of Radiology, Suzhou Municipal Hospital, Suzhou, Jiangsu, China

    Daqiang Xu

  5. Department of Radiology, Taizhou People’s Hospital Affiliated to Nanjing Medical University, Taizhou, China

    Caiyin Tang

  6. Department of Ultrasound, Tsinghua University Affiliated Beijing Tsinghua Changgung Hospital, Beijing, China

    Jing Zhao

  7. Department of Radiotherapy & Oncology, The Second Affiliated Hospital of Soochow University, Suzhou, China

    Yuntian Shen

Authors
  1. Tao Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daqiang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Caiyin Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jing Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuntian Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Cong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jing Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by Daqiang Xu, Caiyin Tang, Jing Zhao, and Yuntian Shen. The first draft of the manuscript was written by Tao Peng, and writing checking and review, and supervision were performed by Cong Yang and Jing Cai. All authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Tao Peng or Jing Cai.

Ethics declarations

Ethical and informed consent for data used

It is the retrospective study, and the clinicians have obtained patients’ agreement before the ultrasound examination, which is an item covered by the medical insurance program. In summary, there is no need for patient consent in our study.

Conflicts of interest/competing interests

The authors declare that they have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peng, T., Xu, D., Tang, C. et al. Automatic coarse-to-refinement-based ultrasound prostate segmentation using optimal polyline segment tracking method and deep learning. Appl Intell 53, 21390–21406 (2023). https://doi.org/10.1007/s10489-023-04676-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04676-4

Keywords

Search

Navigation