Skip to main content
Log in

Coarse-to-fine tuning knowledgeable system for boundary delineation in medical images

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Medical ultrasound image segmentation is crucial to the clinical diagnosis of planning for medical diseases. However, this task is challenging because of the missing/ambiguous edges and inhomogeneous intensity distribution of organs usually noted in ultrasound images. In this study, we devised a new coarse-to-refined architecture for different organ segmentation tasks. Our presented model has four merits: first, our work exploits the benefit of artificial intelligence algorithm to intelligently locate the target area and the feature of the principal curve to intelligently approach the center of data points in the refinement step. Second, we designed an enhanced polygon tracking model to increase our algorithm’s accuracy and efficiency. Third, to ensure population diversity and achieve optimal model initialization, we improved the traditional quantum evolution network both the numerous operator and global optimum search algorithm. Fourth, we devised an interpretable mathematical mapping function to smoothen the contour of the region of interest, which is expressed through the neural network model parameters. Outcomes on different types of datasets indicate that our developed model achieves excellent segmentation capability, yielding an average Dice similarity coefficient, Jaccard similarity coefficient, and accuracy of 94.1% ± 3.9%, 92.4% ± 4.7%, and 93.6% ± 4.1%, respectively.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Algorithm 1
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

Data will be made available on reasonable request. 

References

  1. Wang K, Zhang X, Zhang X et al (2022) EANet: iterative edge attention network for medical image segmentation. Pattern Recogn 127:108636

    Article  Google Scholar 

  2. Rahman A, Valanarasu JMJ, Hacihaliloglu I, Patel VM (2023) Ambiguous Medical Image Segmentation Using Diffusion Models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp 11536–11546

  3. Vesal S, Gayo I, Bhattacharya I et al (2022) Domain generalization for prostate segmentation in transrectal ultrasound images: a multi-center study. Med Image Anal 82:102620

    Article  Google Scholar 

  4. Xu X, Sanford T, Turkbey B et al (2022) Polar transform network for prostate ultrasound segmentation with uncertainty estimation. Med Image Anal 78:102418

    Article  Google Scholar 

  5. Xu X, Lian C, Wang S et al (2021) Asymmetric multi-task attention network for prostate bed segmentation in computed tomography images. Med Image Anal 72:102116

    Article  Google Scholar 

  6. Monkam P, Jin S, Lu W (2023) Annotation cost minimization for ultrasound image segmentation using cross-domain transfer learning. IEEE J Biomedical Health Inf 27:2015–2025

    Google Scholar 

  7. Sharifzadeh M, Benali H, Rivaz H (2022) Investigating shift variance of convolutional neural networks in ultrasound image segmentation. IEEE Trans Ultrason Ferroelectr Freq Control 69:1703–1713

  8. Peng T, Zhao J, Gu Y et al (2022) H-ProMed: Ultrasound image segmentation based on the evolutionary neural network and an improved principal curve. Pattern Recogn 131:108890

    Article  Google Scholar 

  9. Peng T, Wu Y, Qin J et al (2022) H-ProSeg: hybrid ultrasound prostate segmentation based on explainability-guided mathematical model. Comput Methods Programs Biomed 219:106752

    Article  Google Scholar 

  10. Xiao G, Tian S, Yu L et al (2023) Siamese few-shot network: a novel and efficient network for medical image segmentation. Appl Intell 53:17952–17964

    Article  Google Scholar 

  11. Li X, Wu Y, Dai S (2023) Semi-supervised medical imaging segmentation with soft pseudo-label fusion. Appl Intell 53:20753–20765

  12. Ghose S, Oliver A, Mitra J et al (2013) A supervised learning framework of statistical shape and probability priors for automatic prostate segmentation in ultrasound images. Med Image Anal 17:587–600

    Article  Google Scholar 

  13. Zhou Q, Wang Q, Bao Y et al (2022) LAEDNet: a lightweight attention encoder–decoder network for ultrasound medical image segmentation. Comput Electr Eng 99:107777

    Article  Google Scholar 

  14. Amiri M, Brooks R, Rivaz H (2020) Fine-tuning U-Net for Ultrasound Image Segmentation: different layers, different outcomes. IEEE Trans Ultrason Ferroelectr Freq Control 67:2510–2518

    Article  Google Scholar 

  15. Xia C, Li J, Chen X et al (2017) What is and what is not a salient object? learning salient object detector by ensembling linear exemplar regressors. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Honolulu, HI, pp 4399–4407

    Chapter  Google Scholar 

  16. Ni B, Liu Z, Cai X et al (2022) Segmentation of ultrasound image sequences by combing a novel deep siamese network with a deformable contour model. Neural Comput Appl 35:14535–14549

  17. Cai J, Zhang Z, Cui L et al (2019) Towards cross-modal organ translation and segmentation: a cycle- and shape-consistent generative adversarial network. Med Image Anal 52:174–184

    Article  Google Scholar 

  18. Gupta D, Anand RS, Tyagi B (2015) A hybrid segmentation method based on gaussian kernel fuzzy clustering and region based active contour model for ultrasound medical images. Biomed Signal Process Control 16:98–112

    Article  Google Scholar 

  19. Fang L, Qiu T, Liu Y, Chen C (2018) Active contour model driven by global and local intensity information for ultrasound image segmentation. Comput Math Appl 75:4286–4299

    Article  MathSciNet  Google Scholar 

  20. Lafci B, Mercep E, Morscher S et al (2021) Deep learning for automatic segmentation of hybrid Optoacoustic Ultrasound (OPUS) images. IEEE Trans Ultrason Ferroelectr Freq Control 68:688–696

    Article  Google Scholar 

  21. Ronneberger O, Fischer P, Brox T (2015) U-Net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-assisted intervention – MICCAI 2015. Springer International Publishing, Cham, pp 234–241

    Chapter  Google Scholar 

  22. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2020) Unet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imaging 39:1856–1867

    Article  Google Scholar 

  23. Lei Y, Tian S, He X et al (2019) Ultrasound prostate segmentation based on multidirectional deeply supervised V-Net. Med Phys 46:3194–3206

    Article  Google Scholar 

  24. Peng T, Wang Y, Xu TC et al (2018) Detection of lung contour with closed principal curve and machine learning. J Digit Imaging 31:520–533

    Article  Google Scholar 

  25. Kegl B, Krzyzak A (2002) Piecewise linear skeletonization using principal curves. IEEE Trans Pattern Anal Mach Intell 24:59–74

    Article  Google Scholar 

  26. Kegl B, Krzyzak A, Linder T, Zeger K (2000) Learning and design of principal curves. IEEE Trans Pattern Anal Mach Intell 22:281–297

    Article  Google Scholar 

  27. Su H, Yang Y (2011) Differential evolution and quantum-inquired differential evolution for evolving Takagi–Sugeno fuzzy models. Expert Syst Appl 38:6447–6451

    Article  Google Scholar 

  28. Han J, Moraga C (1995) The influence of the sigmoid function parameters on the speed of backpropagation learning. From Natural to Artificial neural computation. Springer Berlin Heidelberg, Berlin, Heidelberg, pp 195–201

    Book  Google Scholar 

  29. Clevert D, Unterthiner T, Hochreiter S (2016) Fast and accurate deep network learning by exponential linear units (ELUs). In: International Conference on Learning Representations (ICLR)

  30. Peng T, Wu Y, Zhao J et al (2022) Explainability-guided mathematical model-based segmentation of transrectal ultrasound images for prostate brachytherapy. In: IEEE 16th International Conference on Bioinformatics and Biomedicine (BIBM). pp 1126–1131

  31. Gao Y, Zhou M, Metaxas D (2021) UTNet: A hybrid transformer architecture for medical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp 61–71

  32. Peng T, Wang C, Zhang Y, Wang J (2022) H-SegNet: hybrid segmentation network for lung segmentation in chest radiographs using mask region-based convolutional neural network and adaptive closed polyline searching method. Phys Med Biol 67:075006

    Article  Google Scholar 

  33. He K, Gkioxari G, Dollar P, Girshick R (2017) Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision. Venice, Italy, pp 2961–2969

  34. Dogan RO, Dogan H, Bayrak C, Kayikcioglu T (2021) A two-phase approach using mask R-CNN and 3D U-Net for high-accuracy automatic segmentation of pancreas in CT imaging. Comput Methods Programs Biomed 207:106141

    Article  Google Scholar 

  35. Kabir W, Ahmad MO, Swamy MNS (2015) A novel normalization technique for multimodal biometric systems. 2015 IEEE 58th International Midwest Symposium on Circuits and Systems (MWSCAS). IEEE, Fort Collins, CO, USA, pp 1–4

    Google Scholar 

  36. Chen P (2019) Effects of normalization on the entropy-based TOPSIS method. Expert Syst Appl 136:33–41

    Article  Google Scholar 

  37. Packwood DM, Pattanasattayavong P (2020) Disorder-robust bands from anisotropic orbitals in a coordination polymer semiconductor. J Phys: Condens Matter 32:275701

    Google Scholar 

  38. Zeng Y-R, Zeng Y, Choi B, Wang L (2017) Multifactor-influenced energy consumption forecasting using enhanced back-propagation neural network. Energy 127:381–396

    Article  Google Scholar 

  39. Leema N, Nehemiah HK, Kannan A (2016) Neural network classifier optimization using Differential Evolution with Global Information and Back Propagation algorithm for clinical datasets. Appl Soft Comput 49:834–844

    Article  Google Scholar 

  40. Eelbode T, Bertels J, Berman M et al (2020) Optimization for medical image segmentation: theory and practice when evaluating with dice score or Jaccard Index. IEEE Trans Med Imaging 39:3679–3690

    Article  Google Scholar 

  41. Ma J, Chen J, Ng M et al (2021) Loss odyssey in medical image segmentation. Med Image Anal 71:102035

    Article  Google Scholar 

  42. Abdel-Basset M, Chang V, Mohamed R (2021) A novel equilibrium optimization algorithm for multi-thresholding image segmentation problems. Neural Comput Appl 33:10685–10718

    Article  Google Scholar 

  43. Ali S, Madabhushi A (2012) An integrated region-, boundary-, shape-based active contour for multiple object overlap resolution in histological imagery. IEEE Trans Med Imaging 31:1448–1460

    Article  Google Scholar 

  44. Ionescu C, Papava D, Olaru V, Sminchisescu C (2014) Human3.6 M: large scale datasets and predictive methods for 3D human sensing in natural environments. IEEE Trans Pattern Anal Mach Intell 36:1325–1339

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed to the study conception and design. Material preparation, data collection, and clinical background analysis were performed by Yiyun Wu, Jing Zhao, Caishan Wang, Wenjie Wang, and Yuntian Shen. The first draft of the manuscript was completed by Dr. Tao Peng, and writing checking and review, and supervision were performed by Dr. Jing Cai. All authors commented on previous versions of the manuscript, and read and approved the final manuscript.

Corresponding authors

Correspondence to Tao Peng or Jing Cai.

Ethics declarations

Ethical and informed consent for data used

It is the retrospective work, and the clinicians have received patients’ agreement before the ultrasound inspection, which is an item covered by the medical insurance program. Overall, there is no requirement for patient consent in our research work. 

Conflicts of interest

All the authors announce that they have no conflicts of interest that are related to this work.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

1.1 Quantum computing

The most important theory of quantum computing is to transform the relative phase between quanta using the ground states to compensate for the interference of the superposition state through the quantum gate. Here are some important concepts of quantum computing.

Chromosome coding based on real numbers

A quantum bit is the smallest unit of information saved in a quantum computer with two states (“0” and “1” states). Let α and β be complex numbers that denote the probability amplitudes of the “0” and “1” states, respectively. Meanwhile, the likelihood of measuring |0〉 and |1〉 are |α|2 and |β|2, respectively. The quantum bit is denoted as follows:

$$\left| \psi \right\rangle =\alpha \left| 0 \right\rangle +\beta \left| 1 \right\rangle$$
(10)

where it satisfies the rule that: |α|2 + |β|2 = 1.

Q-bit representation

The main purpose of Q-bit representation is to indicate a linear superposition of states. Composed of a string of m Q-bits, the Q-bit individual set is denoted as follows.

$$q=\left[ {\left. {\begin{array}{*{20}{c}} {{\alpha _1}} \\ {{\beta _1}} \end{array}} \right|\left. {\begin{array}{*{20}{c}} {{\alpha _2}} \\ {{\beta _2}} \end{array}} \right|\left. {\begin{array}{*{20}{c}} \cdots \\ \cdots \end{array}} \right|\begin{array}{*{20}{c}} {{\alpha _m}} \\ {{\beta _m}} \end{array}} \right]$$
(11)

To better understand this, suppose that in the two-Q-bit system, there exists an individual \(\left[ {\left. {\begin{array}{*{20}{c}} {1/\sqrt 2 } \\ {1/\sqrt 2 } \end{array}} \right|\begin{array}{*{20}{c}} {1/2} \\ {\sqrt 3 /2} \end{array}} \right]\) whose states are shown as \(\frac{{\sqrt 2 }}{4}\left| {00} \right\rangle +\frac{{\sqrt 6 }}{4}\left| {01} \right\rangle +\frac{{\sqrt 2 }}{4}\left| {10} \right\rangle +\frac{{\sqrt 6 }}{4}\left| {11} \right\rangle\). Therefore, we can calculate the corresponding probabilities of each state \(\left| {00} \right\rangle ,\left| {01} \right\rangle ,\left| {10} \right\rangle ,\left| {11} \right\rangle\) with 1/8, 3/8, 1/8, and 3/8, respectively. Overall, this individual includes information on four states.

Quantum rotation gate

The quantum rotation gate is very important to update the quantum gate operation, as it has a great impact on improving the performance of QEN. Using the rotation angle\(\Delta {\theta _i},i=1,\dots,m\), the quantum rotation gate \(R({\theta _i})\) in QEN is defined as \(R({\theta _i})=\left[ {\begin{array}{cc} {\cos (\Delta {\theta _i})}& -{\sin (\Delta {\theta _i})} \\ {\sin (\Delta {\theta _i})}&{\cos (\Delta {\theta _i})} \end{array}} \right]\).

Quantum rotation space transformation

Using the quantum rotation gate \(R({\theta _i})\), the Q-bit of an individual is calculated as \(\left[ {\begin{array}{*{20}{c}} {\alpha _{i}^{\prime}} \\ {\beta _{i}^{\prime}} \end{array}} \right]=U\left( {\Delta {\theta _i}} \right)\left[ {\begin{array}{*{20}{c}} {{\alpha _i}} \\ {{\beta _i}} \end{array}} \right]=\left[ {\begin{array}{*{20}{c}} {\cos (\Delta {\theta _i})}&{ - \sin (\Delta {\theta _i})} \\ {\sin (\Delta {\theta _i})}&{\cos (\Delta {\theta _i})} \end{array}} \right]\bullet \left[ {\begin{array}{*{20}{c}} {{\alpha _i}} \\ {{\beta _i}} \end{array}} \right]\).

1.2 QEN

We suppose that QEN has a population of Q-bit individuals, \(Q(g)=(q_{1}^{g},q_{2}^{g},\dots,q_{n}^{g})\) at generation g, n is the population size, and \(q=\{ q_{{i1}}^{g},q_{{i2}}^{g},..,q_{{im}}^{g}\} =\left[ {\left. {\begin{array}{c} {\alpha _{{i1}}^{g}} \\ {\beta _{{i1}}^{g}} \end{array}} \right|\left. {\begin{array}{c} {\alpha _{{i2}}^{g}} \\ {\beta _{{i2}}^{g}} \end{array}} \right|\left. {\begin{array}{*{20}{c}} \cdots \\ \cdots \end{array}} \right|\begin{array}{*{20}{c}} {\alpha _{{im}}^{g}} \\ {\beta _{{im}}^{g}} \end{array}} \right]\) is a Q-bit individual set.

(Step 1) Quantum mutation operation

The mutation operator is expressed as follows.

$$v_{{i,j}}^{g}=\alpha _{{z1,j}}^{g}+F(\alpha _{{z2,j}}^{g} - \alpha _{{z3,j}}^{g})$$
(12)

where both i and j are integers and are in the range of [1, n] and [1, m], respectively. The integers z1, z2, and z3 are selected randomly and are in the range of [1, n]. The control variable F, which controls the amplification of the differential variation, is determined within the range of [0, 1].

(Step 2) Quantum crossover operation

The crossover operator is denoted as follows:

$$u_{{i,j}}^{{g+1}}=\left\{ {\begin{array}{*{20}{c}} {v_{{i,j}}^{g}{\text{ }}if(rand[0,1] \leq CR)} \\ {\alpha _{{i,j}}^{g}{\text{ }}if(rand[0,1]>CR)} \end{array}} \right.$$
(13)

where CR is the crossover probability and lies within the range of [0, 1]. If CR is too large, the QEN turns to fast convergence. If CR is too small, the model needs to spend more time looking for the minimum problem in the QEN.

(Step 3) Quantum selection operation

This step is adopted to decide whether or not the outcome in this step should become the best Q-bit individual of generation g + 1. A selection scheme is implemented between a Q-bit (or binary) individual and its crossover vector. In this stage, a greedy selection operator is adopted, which is denoted as follows.

$$\alpha _{i}^{{g+1}}=\left\{ {\begin{array}{*{20}{c}} {u_{i}^{g}{\text{ }}if{\text{ }}f(u_{i}^{g})<f(\alpha _{i}^{g})} \\ {\alpha _{i}^{g}{\text{ }}otherwise} \end{array}} \right.$$
(14)

and then,

$$q_{i}^{{g+1}}=\left\{ {\begin{array}{*{20}{c}} {u_{i}^{{{\text{g+}}1}}{\text{ }}if{\text{ }}f(u_{i}^{{g+1}})<f(\alpha _{i}^{{g+1}})} \\ {\alpha _{i}^{{g+1}}{\text{ }}otherwise} \end{array}} \right.$$
(15)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peng, T., Wu, Y., Zhao, J. et al. Coarse-to-fine tuning knowledgeable system for boundary delineation in medical images. Appl Intell 53, 30642–30660 (2023). https://doi.org/10.1007/s10489-023-05143-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-05143-w

Keywords

Navigation