Abstract
Traditional Chinese Medicine (TCM) illustrates that the physique determines the susceptibility of human to certain diseases and treatment programs for illness. Tongue diagnosis is an important way to identify the physique, but now it is performed by the doctor’s professional experience and the design of a questionnaire. Consequently, accurate physique identification cannot be obtained easily. In this paper, we propose a new method to identify the physique through wild tongue images using hybrid deep learning methods. It begins with constructing a large number of tongue images that are taken in natural conditions, instead of in a controlled environment. Based on the resulting database, a new method of tongue coating detection is put forward that applies a rapid deep learning method to complete the initial tongue coating detection, and then utilizes another deep learning method, a calibration neural network, to further improve the accuracy of tongue detection. Finally, an effective deep learning method is applied to identify the tongue physique. Experiments validate the proposed method, illustrating that physique identification can be performed well using hybrid deep learning methods.










Similar content being viewed by others
References
Deng T (2002) Treatment based on syndrome differentiation is the soul of clinical medicine of traditional Chinese medicine. Chin Arch Tradit Chin Med 20 (4):394–395. https://doi.org/10.3969/j.issn.1673-7717.2002.04.003
Du JQ, Lu YS, Zhang K, Zhu MF, Ding CH (2008) A novel approach of tongue body and tongue coating separation based on FCM. In: 2008 2nd international conference on bioinformatics and biomedical engineering, Shanghai, 16–18 May, 2008, pp 2499–2503. https://doi.org/10.1109/ICBBE.2008.958
He K, Zhang X, Ren S, Sun J (2014) Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) Computer vision – ECCV 2014. Lecture notes in computer science, vol 8691. Springer, Cham, pp 346–361. https://doi.org/10.1007/978-3-319-10578-9_23
Hu MC, Cheng MH, Lan KC (2016) Color correction parameter estimation on the smartphone and its application to automatic tongue diagnosis. J Med Syst Jan 40(1):18. https://doi.org/10.1007/s10916-015-0387-z
Huang B, Wu J, Zhang D, Li N (2010) Tongue shape classification by geometric features. Inf Sci (Ny) 180(2):312–324. https://doi.org/10.1016/j.ins.2009.09.016
Ini R, Siio I (2013) TongueDx: a tongue diagnosis system for personal health care on smartphone. Human Interface (Japan). http://hdl.handle.net/10083/57050
Institute of Chinese Medicine (2009) ZYYXH/T157-2009 Chinese medicine constitution classification and decision. Chinese Traditional Chinese Medicine Press, Beijing (In Chinese)
Kamarudin ND, Ooi CY, Kawanabe T, Mi X (2016) Tongue’s substance and coating recognition analysis using HSV color threshold in tongue diagnosis. In: Proceedings of SPIE 10011, first international workshop on pattern recognition, 100110J (11 July 2016). https://doi.org/10.1117/12.2242404
Kanawong R (2012) Computer-aided tongue image diagnosis and analysis. Dissertation, University of Missouri at Columbia
Kanawong R, Xu W, Xu D, Li S, Ma T, Duan Y (2012) An automatic tongue detection and segmentation framework for computer-aided tongue imag+e analysis. Int J Funct Inform Personal Med 4(1):56–68. https://doi.org/10.1504/IJFIPM.2012.050420
Kim KH, Do JH, Ryu H, Kim JY (2008) Tongue diagnosis method for extraction of effective region and classification of tongue coating. In: 2008 first workshops on image processing theory, tools and applications, Sousse, 23–26 Nov. 2008, pp 1–7. https://doi.org/10.1109/IPTA.2008.4743772
Ko MM, Park TY, Lee JA, Kang BK, Lee J, Lee MS (2013) A study of tongue and pulse diagnosis in traditional Korean medicine for stroke patients based on quantification theory type II. Evid Based Complement Alternat Med 2013:508918. https://doi.org/10.1155/2013/508918
Lee J, Ko M, Park T, Kang B, Moon T, Lee M (2012) P02.79. Study on the basic tongue diagnosis indicators for pattern identifications in stroke using a decision tree method. BMC Complement Altern Med 12(S1):P135. https://doi.org/10.1186/1472-6882-12-S1-P135
Li Q, Liu Z (2009) Tongue color analysis and discrimination based on hyperspectral images. Comput Med Imaging Graph 33(3):217–221. https://doi.org/10.1016/j.compmedimag.2008.12.004
Li FF, Zhao C, Xia Z et al (2012) Computer-assisted lip diagnosis on traditional Chinese medicine using multi-class support vector machines. BMC Complementary Alternative Med 12:127. http://www.biomedcentral.com/1472-6882/12/127
Li H, Lin Z, Shen X, Brandt J, Hua G (2015) A convolutional neural network cascade for face detection. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), Boston, MA, 7–12 June, 2015, pp 5325–5334. https://doi.org/10.1109/CVPR.2015.7299170
Lin F, Xiahou J, Xu Z (2016) TCM Clinic records data mining approaches based on weighted-LDA and multi-relationship LDA model. Multimed Tools Appl 75 (22):14203–14232. https://doi.org/10.1007/s11042-016-3363-9
Liu B, Zhang Z (2004) The ancient method of syndrome differentiation of thinking. Chin J Basic Med Tradit Chin Med 10(5):325–331. https://doi.org/10.3969/j.issn.1006-3250.2004.05.002
Lo LC, Chen YF, Chen WJ, Cheng TL, Chiang JY (2012) The study on the agreement between automatic tongue diagnosis system and Traditional Chinese Medicine practitioners. Evid Based Complement Alternat Med 2012:505063. https://doi.org/10.1155/2012/505063
Luo D, Wen G, Li D, Hu Y, Huan E (2018) Iterative regression based Face Detection using deep learning. Multimed Tools Appl 2018(6):1–18
Ning J, Zhang L, Zhang D, Wu C (2010) Interactive image segmentation by maximal similarity based region merging. Pattern Recognit 43 (2):445–456. https://doi.org/10.1016/j.patcog.2009.03.004
Pang B, Zhang D, Li N, Wang K (2004) Computerized tongue diagnosis based on bayesian networks. IEEE Trans Biomed Eng 51(10):1803–1810. https://doi.org/10.1109/TBME.2004.831534
Pang B, Zhang D, Wang K (2005) The bi-elliptical deformable contour and its application to automated tongue segmentation in Chinese medicine. IEEE Trans Med Imaging 24(8):946–956. https://doi.org/10.1109/TMI.2005.850552
Ren S, He K, Girshick R, Sun J (2017) Faster r-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031 https://doi.org/10.1109/TPAMI.2016.2577031
Shi M, Li G, Li F (2013) C2g2FSnake: automatic tongue image segmentation utilizing prior knowledge. Sci China Inform Sci 56(9):1–14. https://doi.org/10.1007/s11432-011-4428-z
Shi MJ, Li GZ, Li FF, Xu C (2014) Computerized tongue image segmentation via the double geo-vector flow. Chin Med 9:7. https://doi.org/10.1186/1749-8546-9-7
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:http://arxiv.org/abs1409.1556v1 [cs.CV]
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), Boston, MA, 7–12 June 2015. https://doi.org/10.1109/CVPR.2015.7298594, pp 1–9
Wang X, Zhang D (2010) An optimized tongue image color correction scheme. IEEE Trans Inf Technol Biomed 14(6):1355–1364. https://doi.org/10.1109/TITB.2010.2076378
Wang X, Zhang D (2013) A high quality color imaging system for computerized tongue image analysis. Expert Syst Appl 40(15):5854–5866. https://doi.org/10.1016/j.eswa.2013.04.031
Wang X, Zhang B, Yang Z, Wang H, Zhang D (2013) Statistical analysis of tongue images for feature extraction and diagnostics. IEEE Trans Image Process 22(12):5336–5347. https://doi.org/10.1109/TIP.2013.2284070 https://doi.org/10.1109/TIP.2013.2284070
Wu K, Zhang D (2015) Robust tongue segmentation by fusing region-based and edge-based approaches. Expert Syst Appl 42(21):8027–8038. https://doi.org/10.1016/j.eswa.2015.06.032
You M, Ge L, Li GZ, Xu L, Huang S (2009) A TCM platform for maters’ experience sharing. In: International joint conference on bioinformatics, systems biology and intelligent computing (IJCBS09). IEEE Computer Society, Washington, DC, pp 388–391
Zhang J, Hu G, Zhang X (2015) Extraction of tongue feature related to TCM physique based on image processing. In: 2015 12th international computer conference on wavelet active media technology and information processing (ICCWAMTIP), Chengdu, China, 18–20 December 2015, pp 251–255. https://doi.org/10.1109/ICCWAMTIP.2015.7493986
Zhang B, Kumar BV, Zhang D (2014) Detecting diabetes mellitus and nonproliferative diabetic retinopathy using tongue color, texture, and geometry features. IEEE Trans Biomed Eng 61(2):491–501. https://doi.org/10.1109/TBME.2013.2282625
Zhang Q, Shang HL, Zhu JJ, Jin MM, Wang WX, Kong QS (2013) A new tongue diagnosis application on Android platform. In: 2013 IEEE international conference on bioinformatics and biomedicine (BIBM), 18–21 December 2013, Shanghai, China, pp 334–327. https://doi.org/10.1109/BIBM.2013.6732705
Zhuo L, Zhang J, Dong P, Zhao Y, Peng B (2014) An SA–GA–BP neural network-based color correction algorithm for TCM tongue images. Neurocomputing 134:111–116. https://doi.org/10.1016/j.neucom.2012.12.080
Zhuo L, Zhang P, Qu P, Peng Y, Zhang J, Li X (2016) A k-PLSR-based color correction method for TCM tongue images under different illumination conditions. Neurocomputing 174:815–821. https://doi.org/10.1016/j.neucom.2015.10.008
Acknowledgements
This study was supported by China National Science Foundation (Grant Nos. 60973083 and 61273363), Science and Technology Planning Project of Guangdong Province (Grant Nos. 2014A010103009 and 2015A020217002), and Guangzhou Science and Technology Planning Project (Grant No. 201504291154480,201803010088). We also thank LetPub (www.letpub.com) for its linguistic assistance during the preparation of this manuscript.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Li, H., Wen, G. & Zeng, H. Natural tongue physique identification using hybrid deep learning methods. Multimed Tools Appl 78, 6847–6868 (2019). https://doi.org/10.1007/s11042-018-6279-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-018-6279-8