Skip to main content
SpringerLink
Log in

Recommending prescription via tongue image to assist clinician

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Traditional Chinese Medicine often use the prescription composed of herbs to cure the disease, which requires doctors with the rich professional knowledge and experience. It is much expected that the prescription can be generated automatically to assist doctors in prescribing using such as machine learning on the tongue images. However, it is confronted with two challenges. First, there is not a larger tongue image database available for machine learning. Second, there is no such machine learning method available for generating prescription according to the given tongue image. This paper begins with constructing a larger tongue image database, where each image corresponds to a prescription. It then uses auto-encoder to extract features for the tongue image, on which the recommendation neural network is proposed to recommend herbs for the prescription. Finally, a new prescription generation method is proposed to select optimal herbs from the recommended herbs to form the final prescription. Experimental results on our constructed databases validate the effectiveness and the superior performance of the proposed methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Castells P (2011) Rank and relevance in novelty and diversity metrics for recommender systems. In: ACM Conference on Recommender Systems, pp 109–116

  2. Chen H-Y, Chen J-Q, Li J-Y, et al. (2019) Deep learning and random forest approach for finding the optimal traditional chinese medicine formula for treatment of alzheimers disease. J Chem Inf Model 59:1605–1623

    Article  Google Scholar 

  3. Cheung F (2011) Tcm: Made in china. Nature 480:S82–S83

    Article  Google Scholar 

  4. Cyranoski D (2018) Why chinese medicine is heading for clinics around the world. Nature 561:448–450

    Article  Google Scholar 

  5. Diwakar M, Kumar M (2018) A review on ct image noise and its denoising. Biomed Signal Process Control 42:73–88

    Article  Google Scholar 

  6. Diwakar M, Singh P (2020) Ct image denoising using multivariate model and its method noise thresholding in non-subsampled shearlet domain. Biomed Signal Process Control 57

  7. Fu M, Qu H, Yi Z (2018) A novel deep learning-based collaborative filtering model for recommendation system. IEEE Trans Cybern PP(99):1–13

    Google Scholar 

  8. haohui Liang, Liu J, Ou A, Zhang H, Li Z, Huang J X (2019) Deep generative learning for automated ehr diagnosis of traditional chinese medicine. Comput Methods Prog Biomed 174:17–23

    Article  Google Scholar 

  9. He X, Liao L, Zhang H et al (2017) Neural collaborative filtering. arXiv:1708.05031

  10. He X, Zhang H, Kan M Y et al (2017) Fast matrix factorization for online recommendation with implicit feedback. arXiv:1708.05024

  11. Hu Q, Yu T, Li J, Yu Q, Zhu L, Gu Y (2019) End-to-end syndrome differentiation of yin deficiency and yang deficiency in traditional chinese medicine. Comput Methods Prog Biomed 174:9–15

    Article  Google Scholar 

  12. Hu Y, Wen G, Liao H et al (2019) Automatic construction of chinese herbal prescription from tongue image via cnns and auxiliary latent therapy topics. IEEE Transaction on Cybernetics, in press

  13. Jiang Z, Zhou X, Zhang X, Chen S (2012) Using link topic model to analyze traditional chinese medicine clinical symptom-herb regularities. Proc. IEEE 14th Int. Conf. E-Health Netw., Appl. Serv., pp 15–18

  14. Kamarudin N D, Ooi C Y, Kawanabe T, Mi X (2016) Tongues substance and coating recognition analysis using hsv color threshold in tongue diagnosis. Proc of SPIE

  15. Ko M M, Park T Y, Lee J A (2013) A study of tongue and pulse diagnosis in traditional korean medicine for stroke patients based on quantification theory type ii. Evidence-Based Complementary and Alternative Medicine

  16. Li S, Zhang B, Jiang D et al (2010) Herb network construction and co-module analysis for uncovering the combination rule of traditional chinese herbal formulae. BMC Bioinf 11(11)

  17. Li S, Kawale J, Fu Y (2015) Deep collaborative filtering via marginalized denoising auto-encoder. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 811–820

  18. Li H, Xu B, Wang N et al (2016) Deep convolutional neural networks for classifying body constitution. Proceedings of the Springer International Conference on Artificial Neural Networks, pp 128–135

  19. Li W, Yang Z (2017) Distributed representation for traditional chinese medicine herb via deep learning models. arXiv:1711.01701

  20. Li W, Yang Z, Sun X (2018) Exploration on generating traditional chinese medicine prescription from symptoms with an end-to-end method. arXiv:1801.09030

  21. Li H, Wen G, Zeng H (2019) Natural tongue physique identification using hybrid deep learning methods. Multimed Tools Appl 78:6847–6868

    Article  Google Scholar 

  22. Li X, Zhang Y, Cui Q et al (2019) Tooth-marked tongue recognition using multiple instance learning and cnn features. IEEE Trans Cybern 49 (2):380–387

    Article  Google Scholar 

  23. Liang Y, Yin Z, Baogang W et al (2018) A topic modeling approach for traditional chinese medicine prescriptions. IEEE Trans Knowl Data Eng 30(6):1007–1021

    Article  Google Scholar 

  24. Liao H, Wen G, Hu Y, Wang C (2019) Convolutional herbal prescription building method from multi-scale facial features. Multimed Tools Appl 78 (24):35665–35688

    Article  Google Scholar 

  25. Liu P, Wang X, Sun X et al (2016) Hkdp: A hybrid knowledge graph based pediatric disease prediction system. In: International Conference on Smart Health, pp 78–90

  26. Lu G, Huang Y, Zhang Q, Huang Z (2019) The study of auxiliary tcm constitution identification model based on tongue image and physical features (in chinese). Lishizhen Med Mater Med Res 30(1):244–246

    Google Scholar 

  27. Ma J, Wen G, Wang C, Jiang L (2019) Complexity perception classification method for tongue constitution recognition. Artif Intell Med 96:123–133

    Article  Google Scholar 

  28. Ping D, Liu L (2009) Core prescription recommending system based on integrated reasoning. In: Fourth International Conference on Computer Sciences and Convergence Information Technology, pp 279–282

  29. Qiu J (2007) Traditional medicine: A culture in the balance. Nature 448(7150):126–128

    Article  Google Scholar 

  30. Ruan C, Ma J, Wang Y, Zhang Y, Yang Y (2019) Discovering regularities from traditional chinese medicine prescriptions via bipartite embedding model. In: IJCAI International Joint Conference on Artificial Intelligence, pp 3346–3352

  31. Ruan C, Wang Y, Zhang Y, Yang Y (2019) Exploring regularity in traditional chinese medicine clinical data using heterogeneous weighted networks embedding. In: Li G et al (eds) DASFAA 2019, LNCS 11448, pp 310–313

  32. Shu Z, Liu W, Wu H et al (2019) Symptom-based network classification identifies distinct clinical subgroups of liver diseases with common molecular pathways. Comput Methods Prog Biomed 174:41–50

    Article  Google Scholar 

  33. Tajima A et al (2017) Embedding-based news recommendation for millions of users. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 1933–1942

  34. Ting SL, Wang WM, Kwok SK et al (2010) Racer: Rule-associated case-based reasoning for supporting general practitioners in prescription making. Expert Syst Appl 37:8079–8089

    Article  Google Scholar 

  35. Vocaturo E, Zumpano E, Veltri. P (2019) On discovering relevant features for tongue colored image analysis. In: 23rd International Database Engineering and Applications Symposium, Athens

  36. Wang J, Wang Q, Li L et al (2013) Phlegm-dampness constitution: genomics, susceptibility, adjustment and treatment with traditional chinese medicine. Amer J Chin Med 41(2):253–262

    Article  Google Scholar 

  37. Wang H, Wang H, Wu X, Liu Q (2015) Relationship prediction of drug-disease: A recommendation system model. Chin Pharmacol Bullet 31(12):1770–1774

    Google Scholar 

  38. Wang H, Wang N, Yeung D Y (2015) Collaborative deep learning for recommender systems. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 1235–1244

  39. Wang J, Wong Y-K, Liao F (2018) What has traditional chinese medicine delivered for modern medicine? Expert Rev Mol Med

  40. Wang R (2019) A chinese medicine formula homology algorithm. J Phys:1168

  41. Wang X, Zhang Y, Wang X, Chen J (2019) A knowledge graph enhanced topic modeling approach for herb recommendation. In: Li G et al (eds) DASFAA 2019, LNCS 11446, pp 709–724

  42. Wei J, He J, Chen K et al (2017) Collaborative filtering and deep learning based recommendation system for cold start items. Expert Syst Appl 69:29–39

    Article  Google Scholar 

  43. Wu C-H, Chen T-C, Hsieh Y-C, Tsao H-L (2019) A hybrid rule mining approach for cardiovascular disease detection in traditional chinese medicine. J Intell Fuzzy Syst:36

  44. Wu G, Zhang W, Li H (2019) Application of metabolomics for unveiling the therapeutic role of traditional chinese medicine in metabolic diseases. J Ethnopharmacol 242:112057

    Article  Google Scholar 

  45. Yan E, Song J, Liu C, Luan J, Hong W (2019) Comparison of support vector machine,backpropagation neural network and extreme learning machine for syndrome element differentiation. Artif Intell Rev

  46. Yang J, Yu K, Gong Y, Huang T (2009) Linear spatial pyramid matching using sparse coding for image classification. In: CVPR

  47. Yao L, Zhang Y, Wei B (2014) An evolution system for traditional chinese medicine prescription. In: Knowl Eng Manag:95–106

  48. Ying Z, Wendi J, Yiping Z et al (2017) Auxiliary diagnosis and treatment system of tcm based on latent semantic model. J Comput Appl S1:303–307

    Google Scholar 

  49. Ying Zhang WJ, Wang Xl, Zhou Y (2017) Latent semantic diagnosis in traditional chinese medicine. World Wide Web 20:1071–1087

    Article  Google Scholar 

  50. Yu T, Li J, Yu Q et al (2017) Knowledge graph for tcm health preservation: Design, construction, and applications. Artif Intell Med 77:48–52

    Article  Google Scholar 

  51. Yuan W, Li C, Guan D et al (2018) Socialized healthcare service recommendation using deep learning. Neural Comput Appl 7:1–12

    Google Scholar 

  52. Zhang N L, Zhang R, Chen T (2012) Discovery of regularities in the use of herbs in traditional chinese medicine prescriptions. Front Appl Data Min:353–360

  53. Zhang B, Bhagavatula V, Zhang D (2014) Detecting diabetes mellitus and nonproliferative diabetic retinopathy using tongue color, texture, and geometry features. IEEE Trans Biomed Eng 61(2):491–501

    Article  Google Scholar 

  54. Zhang J, Hu G, Zhang X (2015) Extraction of tongue feature related to tcm physique based on image processing. In: International Computer Conference on Wavelet Active Media Technology and Information Processing, pp 251–255

  55. Zhang F, Yuan N J, Lian D et al (2016) Collaborative knowledge base embedding for recommender systems. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 353–362

  56. Zhang S, Yao L, Sun A (2018) Deep learning based recommender system: A survey and new perspectives. ACM Comput Surv 1:1:35

    Google Scholar 

  57. Zhang Q, Bai C, Chen Z et al (2019) Smart chinese medicine for hypertension treatment with a deep learning model. J Netw Comput Appl 129:1–8

    Article  Google Scholar 

  58. Zhao G, Zhuang X, Wang X et al (2018) Data-driven traditional chinese medicine clinical herb modeling and herb pair recommendation. In: 2018 7th International Conference on Digital Home, pp 160–166

  59. Zheng G, Jiang M, Lu C, Lu A (2014) Prescription analysis and mining. Data Anal Tradition Chin Med Res:97–109

  60. Zhou H, Hu G, Zhang X (2018) Constitution identification of tongue image based on cnn. In: 11th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics

  61. Zhou B, Lib T, Yang M et al (2019) Characterization of the hot and cold medicinal properties of traditional chinese herbs by spontaneous photon emission ratio of mice. J Ethnopharmacol 243:112108

    Article  Google Scholar 

  62. Zhou Y, Qi X, Huang Y, Ju. F (2019) Research on construction and application of tcm knowledge graph based on ancient chinese texts. In: IEEE/WIC/ACM International Conference on Web Intelligence, Thessaloniki

  63. Zhu J, Liu Y, Zhang Y et al (2019) Ihpreten: A novel supervised learning framework with attribute regularization for prediction of incompatible herb pair in traditional chinese medicine. Neurocomputing 338:207–221

    Article  Google Scholar 

  64. Zhuo L, Zhang J, Dong P et al (2014) An sa-ga-bp neural network based color correction algorithm for tcm tongue images. Neurocomputing 134:111–116

    Article  Google Scholar 

Download references

Acknowledgments

This study was supported by China National Science Foundation (Grant Nos. 61273363 and 61976092 ), Guangdong Province Key Area R & D Plan Project (2020B1111120001), and Guangzhou Science and Technology Planning Project (Grant No. 201604020179 and 201803010088).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering,, South China University of Technology, Guangzhou, China

    Guihua Wen, Kewen Wang & Yuhua Huang

  2. School of Computer Science, Guangdong Polytechnic Normal University, Guangzhou, China

    Huihui Li

  3. First Affiliated Hospital of Sun Yat sen University, Guangzhou, China

    Shijun Zhang

Authors
  1. Guihua Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kewen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huihui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuhua Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shijun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huihui Li.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wen, G., Wang, K., Li, H. et al. Recommending prescription via tongue image to assist clinician. Multimed Tools Appl 80, 14283–14304 (2021). https://doi.org/10.1007/s11042-020-10441-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10441-3

Keywords

Search

Navigation