Skip to main content
SpringerLink
Log in

Transfer learning with deep convolutional neural network for constitution classification with face image

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Constitution classification is the basis and core content of constitution research in Traditional Chinese medicine. The convolutional neural networks have successfully established many models for image classification, but it requires a lot of training data. In the field of Traditional Chinese medicine, the available clinical data is very limited. To solve this problem, we propose a method for constitution classification through transfer learning. Firstly, the DenseNet-169 model trained in ImageNet is applied. Secondly, we carefully modify the DenseNet-169 structure according to the constitution characteristics, and then the modified model is trained in the clinical data to obtain the constitution identification network called ConstitutionNet. In order to further improve the accuracy of classification, we integrate the ConstitutionNet with Vgg-16, Inception v3 and DenseNet-121 to test according to the integrated learning idea, and judge the input face image to its constitution type. The experimental results show that transfer learning can achieve better results in small clinical dataset, and the final accuracy of constitution recognition is 66.79%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Ahonen T, Hadid A, Pietikäinen M (2004) Face recognition with local binary patterns. Proceedings of Springer European Conference on Computer Vision, In, pp 469–481

    MATH  Google Scholar 

  2. Bermejo P, Gámez JA, Puerta JM (2014) Speeding up incremental wrapper feature subset selection with Naive Bayes classifier. Knowledge-Based Systems 55:140–147

  3. Burdick J, Marques O, Weinthal J, Furht B (2018) Rethinking skin lesion segmentation in a convolutional classifier. J Digit Imaging 31(4):435–440

    Article  Google Scholar 

  4. Chen KH, Wang KJ, Wang KM et al (2014) Applying particle swarm optimization- based decision tree classifier for cancer classification on gene expression data. Appl Soft Comput 24:773–780

    Article  Google Scholar 

  5. China Association of Chinese Medicine (2009) Classification and identification of constitution theory of TCM (ZYYXH/T157-2009). World Journal of Traditional Chinese Medicine 4(4):303–304

    Google Scholar 

  6. Chollet F (2017) Xception: deep learning with depthwise separable convolutions. arXiv preprint:1610.02357

  7. Chollet F.: Keras (2015). https://github.com/fchollet/keras

  8. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, In, pp 886–893

    Google Scholar 

  9. Deng J, Dong W, Socher R et al (2009) Imagenet: a large-scale hierarchical image database. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, In, pp 248–255

    Google Scholar 

  10. Ding YW (2010) Development of constitution theory in TCM. Yunnan Journal of Traditional Chinese Medicine and Materia Medica 2:71–75

    Google Scholar 

  11. Geng Y, Chen J, Fu R et al (2016) Enlighten wearable physiological monitoring systems: on-body rf characteristics based human motion classification using a support vector machine. IEEE Trans Mob Comput 15(3):656–671

    Article  Google Scholar 

  12. He KM, Zhang X, Sun S et al (2016) Deep residual learning for image recognition. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, In, pp 770–778

    Google Scholar 

  13. He KM, Gkioxari G, Dollár P et al (2017) Mask r-cnn. Proceedings of IEEE International Conference on Computer Vision, In, pp 2980–2988

    Google Scholar 

  14. Hoo-Chang S, Roth HR, Gao M et al (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285

    Article  Google Scholar 

  15. Howard AG, Zhu M, Chen B et al (2017) Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861

  16. Hu X, Zhu H, Xu J, Xu D et al (2014) Wrist pulse signals analysis based on deep convolutional neural networks. Proceedings of IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology, In, pp 1–7

    Google Scholar 

  17. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. Proceedings of the IEEE conference on computer vision and pattern recognition, In, pp 7132–7141

    Google Scholar 

  18. Huan EY, Wen GH, Zhang SJ et al (2017) Deep convolutional neural networks for classifying body constitution based on face image. Computational and Mathematical Methods in Medicine

  19. Huang G, Liu Z, Maaten VD et al (2017) Densely connected convolutional networks. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, In, pp 4700–4708

    Google Scholar 

  20. Huynh BQ, Li H, Giger ML (2016) Digital mammographic tumor classification using transfer learning from deep convolutional neural networks. Journal of Medical Imaging 3(3):034501

    Article  Google Scholar 

  21. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167

  22. Kermany DS, Goldbaum M, Cai W, Valentim CCS, Liang H, Baxter SL, McKeown A, Yang G, Wu X, Yan F, Dong J, Prasadha MK, Pei J, Ting MYL, Zhu J, Li C, Hewett S, Dong J, Ziyar I, Shi A, Zhang R, Zheng L, Hou R, Shi W, Fu X, Duan Y, Huu VAN, Wen C, Zhang ED, Zhang CL, Li O, Wang X, Singer MA, Sun X, Xu J, Tafreshi A, Lewis MA, Xia H, Zhang K (2018) Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172(5):1122–1131

    Article  Google Scholar 

  23. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems pp:1097–1105

  24. Lan X, Ye M (2019) Shao R et al. A Robust RGB-Infrared Tracking System. IEEE Transactions on Industrial Electronics, Learning Modality-Consistency Feature Templates

    Google Scholar 

  25. Lan X, Ma AJ, Yuen PC, Chellappa R (2015) Joint sparse representation and robust feature-level fusion for multi-cue visual tracking. IEEE Trans Image Process 24(12):5826–5841

    Article  MathSciNet  MATH  Google Scholar 

  26. Lan X, Zhang S, Yuen PC et al (2017) Learning common and feature-specific patterns: a novel multiple-sparse-representation-based tracker. IEEE Trans Image Process 27(4):2022–2037

    Article  MathSciNet  MATH  Google Scholar 

  27. Law H, Deng J (2018) Cornernet: detecting objects as paired keypoints. Proceedings of the Springer European Conference on Computer Vision, In, pp 734–750

    Google Scholar 

  28. Li H, Xu B, Wang N et al (2016) Deep convolutional neural networks for classifying body constitution. Proceedings of Springer International Conference on Artificial Neural Networks, In, pp 128–135S

    Google Scholar 

  29. Li HH, Wen GH, Zeng HB (2018) Natural tongue physique identification using hybrid deep learning methods. Multimedia Tools and Applications, pp1–22

  30. Lin JD, Lin JS, Chen LL et al (2012) BCQs: a body constitution questionnaire to assess stasis in traditional Chinese medicine. European Journal of Integrative Medicine 4(4):e379–e391

    Article  Google Scholar 

  31. Lin JS, Chen LL, Lin JD (2012) BCQ-: a body constitution questionnaire to assess Yin-Xu. Part II: Evaluation of reliability and validity. Forschende Komplementarmed 19(6):285–292

    Article  Google Scholar 

  32. Lin JD, Chen LL, Lin JS et al (2012) BCQ-: a body constitution questionnaire to assess Yin-Xu. Part I: establishment of a provisional version through a Delphi process. Forschende Komplementarmedizin 19(5):234–241

    Article  Google Scholar 

  33. Liu X, Wang Q (2013) Suggestion and analysis on revise of standard of classification and determination of constitution in TCM. Beijing University of Chinese Medicine 5:005

    Google Scholar 

  34. Liu S, Qi L, Qin H et al (2018) Path aggregation network for instance segmentation. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, In, pp 8759–8768

    Google Scholar 

  35. Ma JJ, Wen GH, Hu Y et al (2018) Tongue image constitution recognition based on complexity perception method. arXiv preprint arXiv:1803.00219

  36. Masetic Z, Subasi A (2016) Congestive heart failure detection using random forest classifier. Comput Methods Prog Biomed 130:54–64

    Article  Google Scholar 

  37. Rajpurkar P, Irvin J,Zhu K et al (2017) Chexnet: radiologist-level pneumonia detection on chest x-rays with deep learning.arXiv preprint arXiv:1711.05225

  38. Samanthula BK, Elmehdwi Y, Jiang W (2015) K-nearest neighbor classification over semantically secure encrypted relational data. IEEE Trans Knowl Data Eng 27(5):1261–1273

    Article  Google Scholar 

  39. scikit-learn (2018): machine learning in python. http:// scikit-learn.org/

  40. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  41. Su YC (2007) Establishment of traditional Chinese medical constitutional scale and classificatory index (2–1). Yearbook of Chinese Medicine and Pharmacy 25(5): 45–144

  42. Su YC (2008) The creation of traditional Chinese medical constitutional scale and classification index (2–2). Yearbook of Chinese Medicine and Pharmacy 26(5):65–152

    Google Scholar 

  43. Su SY, Yang CH, Chiu CC, Wang Q (2013) Acoustic features for identifying constitutions in traditional Chinese medicine. J Altern Complement Med 19(6):569–576

    Article  Google Scholar 

  44. Szegedy C, Vanhoucke V, Ioffe S et al (2016) Rethinking the inception architecture for computer vision. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, In, pp 2818–2826

    Google Scholar 

  45. Szegedy C, Ioffe S, Vanhoucke V et al (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. AAAI Conference on Artificial Intelligence 4:12

    Google Scholar 

  46. Tajbakhsh N, Shin JY, Gurudu SR et al (2016) Convolutional neural networks for medical image analysis: full training or fine tuning. IEEE transactions on medical imaging 35(5):1299–1312

  47. Tan M, Le QV (2019) EfficientNet: rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:1905.11946

  48. TensorFlow (2018). https://www.tensorflow.org

  49. Van OA, Ikram MA, Vernooij MW et al (2015) Transfer learning improves supervised image segmentation across imaging protocols. IEEE Trans Med Imaging 34(5):1018–1030

    Article  Google Scholar 

  50. Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Article  Google Scholar 

  51. Walt SD, Schönberger JL, Nunez-Iglesias J et al (2014) Scikit-image: image processing in python. PeerJ 2: e453

  52. Wang Q (2005) Classification and diagnosis basis of nine basic constitutions in Chinese medicine. Journal-Beijing University of Traditional Chinese Medicine 28(4):1

    Google Scholar 

  53. Wang Q (2006) Three key issues in the study of TCM constitution (Part I). J Tradit Chin Med 4:250–252

    Google Scholar 

  54. Wang Q (2006) Three key issues in the study of TCM constitution (Part II). J Tradit Chin Med 5:329–332

    Google Scholar 

  55. Wang YC, Bai LN (2014) Classification of body constitution of pulse signal in TCM based on BP neural network. J Tradit Chin Med 55(15)

  56. Wang F, Jiang M, Qian C et al (2017) Residual attention network for image classification. arXiv preprint arXiv:1704.06904

  57. Wong W, Lam CK, Su YC, Lin SJ, Ziea ET, Wong VT, Wai LK, Kwan AK (2014) Measuring body constitution: validation of the body constitution questionnaire (BCQ) in Hong Kong. Complementary therapies in medicine 22(4):670–682

    Article  Google Scholar 

  58. Woo S, Park J, Lee JY et al (2018) Cbam: convolutional block attention module. In: Proceedings of the European Conference on Computer Vision, pp3–19.

  59. Xie S, Girshick R, Dollár P et al (2017) Aggregated residual transformations for deep neural networks. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, In, pp 5987–5995

    Google Scholar 

  60. Yu RX, Wang Q, Wang J et al (2013) An analysis of the status quo of application of constitution identification. Chinese Journal of Information on Traditional Chinese Medicine 2:107–109

    Google Scholar 

  61. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. Proceedings of Springer European Conference on Computer Vision, In, pp 818–833

    Google Scholar 

  62. Zhang X, Li Z, Loy CC et al (2017) Polynet: a pursuit of structural diversity in very deep networks. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, In, pp 3900–3908

    Google Scholar 

  63. Zhang SF, Wen LY, Bian X et al (2017) Single-shot refinement neural network for object detection. arXiv preprint arXiv:1711.06897

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, China Guangdong Artificial Intelligence Engineering Research Center for Traditional Chinese Medicine, South China University of Technology, Guangzhou, China

    Er-Yang Huan & Gui-Hua Wen

Authors
  1. Er-Yang Huan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gui-Hua Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gui-Hua Wen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huan, EY., Wen, GH. Transfer learning with deep convolutional neural network for constitution classification with face image. Multimed Tools Appl 79, 11905–11919 (2020). https://doi.org/10.1007/s11042-019-08376-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08376-5

Keywords

Search

Navigation