Skip to main content
SpringerLink
Log in

Natural tongue physique identification using hybrid deep learning methods

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Traditional Chinese Medicine (TCM) illustrates that the physique determines the susceptibility of human to certain diseases and treatment programs for illness. Tongue diagnosis is an important way to identify the physique, but now it is performed by the doctor’s professional experience and the design of a questionnaire. Consequently, accurate physique identification cannot be obtained easily. In this paper, we propose a new method to identify the physique through wild tongue images using hybrid deep learning methods. It begins with constructing a large number of tongue images that are taken in natural conditions, instead of in a controlled environment. Based on the resulting database, a new method of tongue coating detection is put forward that applies a rapid deep learning method to complete the initial tongue coating detection, and then utilizes another deep learning method, a calibration neural network, to further improve the accuracy of tongue detection. Finally, an effective deep learning method is applied to identify the tongue physique. Experiments validate the proposed method, illustrating that physique identification can be performed well using hybrid deep learning methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Deng T (2002) Treatment based on syndrome differentiation is the soul of clinical medicine of traditional Chinese medicine. Chin Arch Tradit Chin Med 20 (4):394–395. https://doi.org/10.3969/j.issn.1673-7717.2002.04.003

    Google Scholar 

  2. Du JQ, Lu YS, Zhang K, Zhu MF, Ding CH (2008) A novel approach of tongue body and tongue coating separation based on FCM. In: 2008 2nd international conference on bioinformatics and biomedical engineering, Shanghai, 16–18 May, 2008, pp 2499–2503. https://doi.org/10.1109/ICBBE.2008.958

  3. He K, Zhang X, Ren S, Sun J (2014) Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) Computer vision – ECCV 2014. Lecture notes in computer science, vol 8691. Springer, Cham, pp 346–361. https://doi.org/10.1007/978-3-319-10578-9_23

    Google Scholar 

  4. Hu MC, Cheng MH, Lan KC (2016) Color correction parameter estimation on the smartphone and its application to automatic tongue diagnosis. J Med Syst Jan 40(1):18. https://doi.org/10.1007/s10916-015-0387-z

    Article  Google Scholar 

  5. Huang B, Wu J, Zhang D, Li N (2010) Tongue shape classification by geometric features. Inf Sci (Ny) 180(2):312–324. https://doi.org/10.1016/j.ins.2009.09.016

    Article  Google Scholar 

  6. Ini R, Siio I (2013) TongueDx: a tongue diagnosis system for personal health care on smartphone. Human Interface (Japan). http://hdl.handle.net/10083/57050

  7. Institute of Chinese Medicine (2009) ZYYXH/T157-2009 Chinese medicine constitution classification and decision. Chinese Traditional Chinese Medicine Press, Beijing (In Chinese)

  8. Kamarudin ND, Ooi CY, Kawanabe T, Mi X (2016) Tongue’s substance and coating recognition analysis using HSV color threshold in tongue diagnosis. In: Proceedings of SPIE 10011, first international workshop on pattern recognition, 100110J (11 July 2016). https://doi.org/10.1117/12.2242404

  9. Kanawong R (2012) Computer-aided tongue image diagnosis and analysis. Dissertation, University of Missouri at Columbia

  10. Kanawong R, Xu W, Xu D, Li S, Ma T, Duan Y (2012) An automatic tongue detection and segmentation framework for computer-aided tongue imag+e analysis. Int J Funct Inform Personal Med 4(1):56–68. https://doi.org/10.1504/IJFIPM.2012.050420

    Google Scholar 

  11. Kim KH, Do JH, Ryu H, Kim JY (2008) Tongue diagnosis method for extraction of effective region and classification of tongue coating. In: 2008 first workshops on image processing theory, tools and applications, Sousse, 23–26 Nov. 2008, pp 1–7. https://doi.org/10.1109/IPTA.2008.4743772

  12. Ko MM, Park TY, Lee JA, Kang BK, Lee J, Lee MS (2013) A study of tongue and pulse diagnosis in traditional Korean medicine for stroke patients based on quantification theory type II. Evid Based Complement Alternat Med 2013:508918. https://doi.org/10.1155/2013/508918

    Google Scholar 

  13. Lee J, Ko M, Park T, Kang B, Moon T, Lee M (2012) P02.79. Study on the basic tongue diagnosis indicators for pattern identifications in stroke using a decision tree method. BMC Complement Altern Med 12(S1):P135. https://doi.org/10.1186/1472-6882-12-S1-P135

    Article  Google Scholar 

  14. Li Q, Liu Z (2009) Tongue color analysis and discrimination based on hyperspectral images. Comput Med Imaging Graph 33(3):217–221. https://doi.org/10.1016/j.compmedimag.2008.12.004

    Article  Google Scholar 

  15. Li FF, Zhao C, Xia Z et al (2012) Computer-assisted lip diagnosis on traditional Chinese medicine using multi-class support vector machines. BMC Complementary Alternative Med 12:127. http://www.biomedcentral.com/1472-6882/12/127

    Article  Google Scholar 

  16. Li H, Lin Z, Shen X, Brandt J, Hua G (2015) A convolutional neural network cascade for face detection. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), Boston, MA, 7–12 June, 2015, pp 5325–5334. https://doi.org/10.1109/CVPR.2015.7299170

  17. Lin F, Xiahou J, Xu Z (2016) TCM Clinic records data mining approaches based on weighted-LDA and multi-relationship LDA model. Multimed Tools Appl 75 (22):14203–14232. https://doi.org/10.1007/s11042-016-3363-9

    Article  Google Scholar 

  18. Liu B, Zhang Z (2004) The ancient method of syndrome differentiation of thinking. Chin J Basic Med Tradit Chin Med 10(5):325–331. https://doi.org/10.3969/j.issn.1006-3250.2004.05.002

    Google Scholar 

  19. Lo LC, Chen YF, Chen WJ, Cheng TL, Chiang JY (2012) The study on the agreement between automatic tongue diagnosis system and Traditional Chinese Medicine practitioners. Evid Based Complement Alternat Med 2012:505063. https://doi.org/10.1155/2012/505063

    Google Scholar 

  20. Luo D, Wen G, Li D, Hu Y, Huan E (2018) Iterative regression based Face Detection using deep learning. Multimed Tools Appl 2018(6):1–18

  21. Ning J, Zhang L, Zhang D, Wu C (2010) Interactive image segmentation by maximal similarity based region merging. Pattern Recognit 43 (2):445–456. https://doi.org/10.1016/j.patcog.2009.03.004

    Article  Google Scholar 

  22. Pang B, Zhang D, Li N, Wang K (2004) Computerized tongue diagnosis based on bayesian networks. IEEE Trans Biomed Eng 51(10):1803–1810. https://doi.org/10.1109/TBME.2004.831534

    Article  Google Scholar 

  23. Pang B, Zhang D, Wang K (2005) The bi-elliptical deformable contour and its application to automated tongue segmentation in Chinese medicine. IEEE Trans Med Imaging 24(8):946–956. https://doi.org/10.1109/TMI.2005.850552

    Article  Google Scholar 

  24. Ren S, He K, Girshick R, Sun J (2017) Faster r-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031 https://doi.org/10.1109/TPAMI.2016.2577031

    Article  Google Scholar 

  25. Shi M, Li G, Li F (2013) C2g2FSnake: automatic tongue image segmentation utilizing prior knowledge. Sci China Inform Sci 56(9):1–14. https://doi.org/10.1007/s11432-011-4428-z

    MathSciNet  Google Scholar 

  26. Shi MJ, Li GZ, Li FF, Xu C (2014) Computerized tongue image segmentation via the double geo-vector flow. Chin Med 9:7. https://doi.org/10.1186/1749-8546-9-7

    Article  Google Scholar 

  27. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:http://arxiv.org/abs1409.1556v1 [cs.CV]

  28. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), Boston, MA, 7–12 June 2015. https://doi.org/10.1109/CVPR.2015.7298594, pp 1–9

  29. Wang X, Zhang D (2010) An optimized tongue image color correction scheme. IEEE Trans Inf Technol Biomed 14(6):1355–1364. https://doi.org/10.1109/TITB.2010.2076378

    Article  Google Scholar 

  30. Wang X, Zhang D (2013) A high quality color imaging system for computerized tongue image analysis. Expert Syst Appl 40(15):5854–5866. https://doi.org/10.1016/j.eswa.2013.04.031

    Article  Google Scholar 

  31. Wang X, Zhang B, Yang Z, Wang H, Zhang D (2013) Statistical analysis of tongue images for feature extraction and diagnostics. IEEE Trans Image Process 22(12):5336–5347. https://doi.org/10.1109/TIP.2013.2284070 https://doi.org/10.1109/TIP.2013.2284070

    Article  Google Scholar 

  32. Wu K, Zhang D (2015) Robust tongue segmentation by fusing region-based and edge-based approaches. Expert Syst Appl 42(21):8027–8038. https://doi.org/10.1016/j.eswa.2015.06.032

    Article  Google Scholar 

  33. You M, Ge L, Li GZ, Xu L, Huang S (2009) A TCM platform for maters’ experience sharing. In: International joint conference on bioinformatics, systems biology and intelligent computing (IJCBS09). IEEE Computer Society, Washington, DC, pp 388–391

  34. Zhang J, Hu G, Zhang X (2015) Extraction of tongue feature related to TCM physique based on image processing. In: 2015 12th international computer conference on wavelet active media technology and information processing (ICCWAMTIP), Chengdu, China, 18–20 December 2015, pp 251–255. https://doi.org/10.1109/ICCWAMTIP.2015.7493986

  35. Zhang B, Kumar BV, Zhang D (2014) Detecting diabetes mellitus and nonproliferative diabetic retinopathy using tongue color, texture, and geometry features. IEEE Trans Biomed Eng 61(2):491–501. https://doi.org/10.1109/TBME.2013.2282625

    Article  Google Scholar 

  36. Zhang Q, Shang HL, Zhu JJ, Jin MM, Wang WX, Kong QS (2013) A new tongue diagnosis application on Android platform. In: 2013 IEEE international conference on bioinformatics and biomedicine (BIBM), 18–21 December 2013, Shanghai, China, pp 334–327. https://doi.org/10.1109/BIBM.2013.6732705

  37. Zhuo L, Zhang J, Dong P, Zhao Y, Peng B (2014) An SA–GA–BP neural network-based color correction algorithm for TCM tongue images. Neurocomputing 134:111–116. https://doi.org/10.1016/j.neucom.2012.12.080

    Article  Google Scholar 

  38. Zhuo L, Zhang P, Qu P, Peng Y, Zhang J, Li X (2016) A k-PLSR-based color correction method for TCM tongue images under different illumination conditions. Neurocomputing 174:815–821. https://doi.org/10.1016/j.neucom.2015.10.008

    Article  Google Scholar 

Download references

Acknowledgements

This study was supported by China National Science Foundation (Grant Nos. 60973083 and 61273363), Science and Technology Planning Project of Guangdong Province (Grant Nos. 2014A010103009 and 2015A020217002), and Guangzhou Science and Technology Planning Project (Grant No. 201504291154480,201803010088). We also thank LetPub (www.letpub.com) for its linguistic assistance during the preparation of this manuscript.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, South China University of Technology, Guangzhou, China

    Huihui Li, Guihua Wen & Haibin Zeng

  2. Guangdong Artificial Intelligence Engineering Research Center for Traditional Chinese Medicine, Guangzhou, China

    Huihui Li & Guihua Wen

  3. Guangdong Artificial Intelligence Engineering Research Center for Traditional Chinese Medicine, Shenzhen, China

    Haibin Zeng

Authors
  1. Huihui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guihua Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haibin Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guihua Wen.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, H., Wen, G. & Zeng, H. Natural tongue physique identification using hybrid deep learning methods. Multimed Tools Appl 78, 6847–6868 (2019). https://doi.org/10.1007/s11042-018-6279-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6279-8

Keywords

Search

Navigation