Skip to main content
SpringerLink
Log in

A two-stage segmentation of sublingual veins based on compact fully convolutional networks for Traditional Chinese Medicine images

  • Research
  • Published:
Health Information Science and Systems Aims and scope Submit manuscript

Abstract

As one of the key methods of Traditional Chinese Medicine inspection, tongue diagnosis manifests the advantages of simplicity and directness. Sublingual veins can provide essential information about human health. In order to automate tongue diagnosis, sublingual veins segmentation has become one important issue in the field of Chinese medicine medical image processing. At present, the primary methods for sublingual veins segmentation are traditional feature engineering methods and the feature representation methods represented by deep learning. The former, which mainly based on colour space, belongs to unsupervised classification method. The latter, which includes U-Net and other deep neural network models, belongs to supervised classification method. Current feature engineering methods can only capture low dimensional information, which makes it difficult to extract efficient features for sublingual veins. On the other hand, current deep learning methods use down-sampling structures, which manifest weak robustness and low accuracy. So, it is difficult for current segmentation approaches to recognize tiny branches of sublingual veins. To overcome the above limits, this paper proposes a novel two-stage semantic segmentation method for sublingual veins. In the first stage, a fully convolutional network without down-sampling is used to realize the accurate segmentation of the tongue that includes the sublingual veins to be segmented in the next stage. During the tongue segmentation, the proposed networks can effectively reduce the loss of medical images spatial feature information. At the same time, in order to expand the receptive field, the dilated convolution has been introduced to the proposed networks, which can capture multi-scale information of segmentation images. In the second stage, another fully convolutional network has been used to segment the sublingual veins on the base of the results from the first stage. In this model, proper dilated convolutional rates have been selected to avoid gridding issue. In order to keep the quality of the images to be segmented, several particular data pre-processing and post-processing have been used, which includes specular highlight removal, data augmentation, erosion and dilation. Finally, in order to evaluate the performance of the proposed model, segmentation results have been compared with the state-of-the-art methods on the base of the dataset from Shanghai University of Traditional Chinese Medicine. The effectiveness of sublingual veins segmentation has been proved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Li N, Zhang D, Wang K. Tongue diagnostics. Beijing: Xueyuan Press; 2006.

    Google Scholar 

  2. Li N. The handbook of Chinese tongue diagnosis. Beijing: Xueyuan Press; 1994.

    Google Scholar 

  3. Wang FW, Liu Y, Lin M-X. Observation and analysis of 112 cases on subglossal collateral vessels of coronary heart disease patients. Chin J Inf Tradit Chin Med. 2004;4:323–4.

    Google Scholar 

  4. Yang Y, Qian J, Zhan Y, Wang Y, Xu H, Yang W. Observation and characteristics analysis of sublingual veins of diabetic vascular disease. J Nanjing Univ Tradit Chin Med. 2008;24(06):370–2.

    Google Scholar 

  5. Pang B, Zhang D, Li N, Wang K. Computerized tongue diagnosis based on Bayesian networks. IEEE Trans Biomed Eng. 2004;51(10):1803–10.

    Article  Google Scholar 

  6. Zhang D, Pang B, Li N, Wang K, Zhang H. Computerized diagnosis from tongue appearance using quantitative feature classification. Am J Chin Med. 2005;33(06):859–66.

    Article  Google Scholar 

  7. Lin H-J, Chen Y-J, Damdinsuren N, Tan T-H, Liu T-Y, Chiang JY. Automatic sublingual vein feature extraction system. In: Proceedings of international conference on medical biometrics. IEEE; 2014. p. 55–62.

  8. Ma J, Wen G, Wang C, Jiang L. Complexity perception classification method for tongue constitution recognition. Artif Intell Med. 2019;96:123–33.

    Article  Google Scholar 

  9. Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In: Proceedings of international conference on medical image computing and computer-assisted intervention. Springer; 2015. p. 234–41.

  10. Yang T, Yoshimura Y, Morita A, Namiki T, Nakaguchi T. Fully automatic segmentation of sublingual veins from retrained u-net model for few near infrared images. 2018. http://arxiv.org/abs/1812.09477.

  11. Yan Z, Wang K, Li N. Sublingual vein segmentation from near infrared sublingual images. In: Proceedings of the 8th IEEE international conference on bioinformatics and bioengineering, BIBE; 2008. October 8–10, Athens, Greece.

  12. Yan Z, Wang K, Li N. Computerized feature quantification of sublingual veins from color sublingual images. Comput Methods Programs Biomed. 2009;93(2):192–205.

    Article  Google Scholar 

  13. Li Q, Wang Y, Liu H, Guan Y, Xu L. Sublingual vein extraction algorithm based on hyperspectral tongue imaging technology. Comput Med Imaging Graph. 2011;35(3):179–85.

    Article  Google Scholar 

  14. Sun D, Wu J, Zhang Y, Bai J, Weng W, Wu Y. Sublingual veins automatic extraction method based on feature clustering. Chin J Biomed Eng. 2008;27(02):265–9.

    Google Scholar 

  15. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2015. p. 3431–40.

  16. Zhao H, Shi J, Qi X, Wang X, Jia J. Pyramid scene parsing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2017. p. 2881–90.

  17. He K, Gkioxari G, Dollár P, Girshick R. Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision; 2017. p. 2961–69.

  18. Ren S, He K, Girshick R, Sun J. Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell. 2015;39(6).

  19. Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL. Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFS. IEEE Trans Pattern Anal Mach Intell. 2017;40(4):834–48.

    Article  Google Scholar 

  20. Peng C, Zhang X, Yu G, Luo G, Sun J. Large kernel matters–improve semantic segmentation by global convolutional network. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2017. p. 4353–61.

  21. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA. Inception-v4, inception-resnet and the impact of residual connections on learning. In: Proceedings of thirty-first AAAI conference on artificial intelligence; 2017.

  22. Kirillov A, Wu Y, He K, Girshick R. Pointrend: image segmentation as rendering. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; 2020. p. 9799–08.

  23. Wong SC, Gatt A, Stamatescu V, McDonnell MD. Understanding data augmentation for classification: when to warp? In: Proceedings of international conference on digital image computing: techniques and applications (DICTA). IEEE; 2016. p. 1–6.

  24. Taghanaki SA, Abhishek K, Cohen JP, Cohen-Adad J, Hamarneh G. Deep semantic segmentation of natural and medical images: a review. Artif Intell Rev. 2020;54:1–42.

    Google Scholar 

  25. Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL. Semantic image segmentation with deep convolutional nets and fully connected crfs. 2014. http://arxiv.org/abs/1412.7062.

  26. Li W, Wang G, Fidon L, Ourselin S, Cardoso MJ, Vercauteren T. On the compactness, efficiency, and representation of 3d convolutional networks: brain parcellation as a pretext task. In: Proceedings of international conference on information processing in medical imaging. Springer; 2017. p. 348–60.

  27. Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions. 2015. http://arxiv.org/abs/1511.07122.

  28. Sevilla-Lara L, Sun D, Jampani V, Black MJ. Optical flow with semantic segmentation and localized layers. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016. p. 3889–98.

  29. Oord Avd, Dieleman S, Zen H, Simonyan K, Vinyals O, Graves A, Kalchbrenner N, Senior A, Kavukcuoglu K. Wavenet: a generative model for raw audio. 2016. http://arxiv.org/abs/1609.03499.

  30. Wang P, Chen P, Yuan Y, Liu D, Huang Z, Hou X, Cottrell G. Understanding convolution for semantic segmentation. In: 2018 IEEE winter conference on applications of computer vision (WACV). IEEE; 2018. p. 1451–60.

  31. Tan RT, Ikeuchi K. Separating reflection components of textured surfaces using a single image. In: Ikeuchi K, Miyazaki D, editors. Digitally archiving cultural objects. New York: Springer; 2008. p. 353–84.

    Chapter  Google Scholar 

  32. He K, Sun J, Tang X. Single image haze removal using dark channel prior. IEEE Trans Pattern Anal Mach Intell. 2010;33(12):2341–53.

    Google Scholar 

  33. He K, Sun J, Tang X. Guided image filtering. IEEE Trans Pattern Anal Mach Intell. 2012;35(6):1397–409.

    Article  Google Scholar 

  34. Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd international conference on international conference on machine learning; 2015. p. 448–56.

  35. Tang Y. Deep learning using linear support vector machines. 2013. http://arxiv.org/abs/1306.0239.

  36. Haralick RM, Sternberg SR, Zhuang X. Image analysis using mathematical morphology. IEEE Trans Pattern Anal Mach Intell. 1987;4:532–50.

    Article  Google Scholar 

  37. Fawcett T. An introduction to roc analysis. Pattern Recogn Lett. 2006;27(8):861–74.

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

A heartfelt thanks to all those who participated in this paper (Lijuan Wang, Huadong Li, Fanyang Meng, Wenmeng Yu, Junhui Deng, Yingyan Hou).

Funding

This work was supported by the National Natural Science Foundation of China (Grant No. 62173195); and the National Key R &D Program Projects of China (Grant No: 2018YFC1707605).

Author information

Authors and Affiliations

  1. State Key Laboratory of Intelligent Technology and Systems, Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China

    Hua Xu & Xiaofei Chen

  2. School of Information Science and Engineering, Hebei University of Science and Technology, Shijiazhuang, 050018, China

    Xiaofei Chen

  3. Shanghai University of Traditional Chinese Medicine, Shanghai, 201203, China

    Peng Qian & Fufeng Li

Authors
  1. Hua Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaofei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peng Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fufeng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Hua Xu or Peng Qian.

Ethics declarations

Conflict of interest

The authors confirm that there are no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, H., Chen, X., Qian, P. et al. A two-stage segmentation of sublingual veins based on compact fully convolutional networks for Traditional Chinese Medicine images. Health Inf Sci Syst 11, 19 (2023). https://doi.org/10.1007/s13755-023-00214-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13755-023-00214-1

Keywords

Search

Navigation