Skip to main content

Advertisement

SpringerLink
Log in

Recognition of Chinese food using convolutional neural network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Food recognition is the first step for dietary assessment. Computer vision technology is being viewed as an effective tool for automatic food recognition for monitoring nutrition intake. Of the many food recognition algorithms in the literature, Bag-of-Features model is a proven approach that has shown impressive recognition accuracy. In this paper, we propose a small and efficient convolutional neural network architecture for Chinese food recognition, which is more applicable for resources limited platforms. Our network architecture is designed to model and perform a pipeline of processing similar to the Bag-of-Features approach. The main advantage of the proposed architecture, like other convolutional neural networks, is its ability to unifiedly optimize the entire network through back propagation, which is critical to recognition accuracy. We further compare and correlate our architecture with the traditional Bag-of-Features model in an attempt to investigate the similarities between them and identify factors that influence the recognition accuracy. The proposed architecture with a 5-layer deep convolutional neural network achieves the top-1 accuracy of 97.12% and the top-5 accuracy of 99.86% on a newly created Chinese food image dataset that is composed of 8734 images of 25 food categories. Our experimental result demonstrates the feasibility of applying the proposed compact CNN architecture to a challenging problem and achieve real-time performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Aharon M, Elad M, Bruckstein A (2006) K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 54(11):4311–4322

    Article  Google Scholar 

  2. Andrew P (2013) AMA recognizes obesity as a disease. The New York Times, p.10

  3. Anthimopoulos MM, Gianola L, Scarnato L et al (2014) A food recognition system for diabetic patients based on an optimized bag-of-features model. IEEE Journal of Biomedical and Health Informatics 18(4):1261–1271

    Article  Google Scholar 

  4. Ashkan A, Forouzanfar Mohammad H, Reitsma Marissa B et al (2017) Health effects of overweight and obesity in 195 countries over 25 years. N Engl J Med 377(1):13–27

    Article  Google Scholar 

  5. Bleich S, Cutler D, Murray C et al (2008) Why is the developed world obese? Annu Rev Public Health 29(1):273–295

    Article  Google Scholar 

  6. Chen M, Dhingra K, Wu W, et al (2009) PFID: Pittsburgh fast-food image dataset. IEEE International Conference on Image Processing. pp. 289–292

  7. Chen J, Ngo CW (2016) Deep-based ingredient recognition for cooking recipe retrieval. ACM Multimedia Conference, ACM, 32–41

  8. Chen M, Yang Y, Ho C, et al (2012) Automatic Chinese food identification and quantity estimation. In: SIGGRAPH Asia 2012 Technical Briefs. ACM, pp 29

  9. Ciocc G, Napoletano P, Schettini R (2017) Food Recognition: A new dataset, experiments, and results. IEEE Journal of Biomedical and Health Informatics 21(3):588–598

    Article  Google Scholar 

  10. Dong C, Loy CC, He K et al (2016) Image Super-Resolution Using Deep Convolutional Networks. IEEE Transactions on Pattern Analysis & Machine Intelligence 38(2):295

    Article  Google Scholar 

  11. Farinella GM, Moltisanti M, Battiato S (2014) Classifying food images represented as bag of textons. In: Image Processing (ICIP), 2014 IEEE International Conference. IEEE, pp.5212–5216

  12. Giovany S, Putra A, Hariawan AS et al (2017) Machine learning and sift approach for Indonesian food image recognition. Procedia Computer Science 116:612–620

    Article  Google Scholar 

  13. Haslam DW, James WP (2005) Obesity. Lancet (Review) 366(9492):1197–1209. https://doi.org/10.1016/S0140-6736(05)67483-1

    Article  Google Scholar 

  14. Kawano Y, Yanai K (2013) Real-Time Mobile Food Recognition System. In Computer Vision and Pattern Recognition Workshops. IEEE:1–7

  15. Kawano Y, Yanai K (2014) Food image recognition with deep convolutional features. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication. ACM, pp 589–593

  16. Kawano Y, Yanai K (2015) Foodcam: a real-time food recognition system on a smartphone. Multimedia Tools & Applications 74(14):5263–5287

    Article  Google Scholar 

  17. Lin M, Chen Q, Yan S (2013) Network in network. arXiv preprint arXiv:1312.4400

  18. Luan S, Zhang B, Chen C et al (2017) Gabor Convolutional Networks. IEEE Trans Image Process 27(9):4357–4366

    Article  MathSciNet  Google Scholar 

  19. Luppino FS, de Wit LM, Bouvy PF et al (2010) Overweight, obesity, and depression: a systematic review and meta-analysis of longitudinal studies. Arch Gen Psychiatry 67(3):220–229

    Article  Google Scholar 

  20. Martin CK, Kaya S, Gunturk BK (2009) Quantification of food intake using food image analysis. In: Engineering in Medicine and Biology Society. EMBC 2009. Annual International Conference of the IEEE. pp 6869–6872

  21. Matsuda Y, Hoashi H, Yanai K (2012) Recognition of multiple-food images by detecting candidate regions. In Multimedia and Expo, pp 25–30

  22. Matthew W (2013) The facts about obesity. H&HN. American Hospital Association. Retrieved June 24

  23. Papyan V, Romano Y, Elad M (2016) Convolutional neural networks analyzed via convolutional sparse coding. arXiv preprint arXiv:1607.08194

  24. Shroff G, Smailagic A, Siewiorek DP (2008) Wearable context-aware food recognition for calorie monitoring. In: Proc. 12th IEEE Int. Symp. Wearable Comput, pp 119–120

  25. USDA (2008) Food and Nutrient Database for Dietary Studies, 3.0. Agricultural Research Service, Food Surveys Research Group, Beltsville

    Google Scholar 

  26. Thai Van Phat, Dang Xuan Tien, Quang Pham, et al. (2017) Vietnamese food recognition using convolutional neural networks. International Conference on Knowledge and Systems Engineering, pp 124–129

  27. Wang L, Zhang B, Han J et al (2016) Robust object representation by boosting-like deep learning architecture. Signal Process Image Commun 47:490–499

    Article  Google Scholar 

  28. Yanai K, Kawano Y (2015) In: Proceedings of 2015 IEEE Int. Conf. on Multimedia and Expo Workshops, Trino, pp. 1–6

  29. Yanai K, Tanno R, Okamoto K, et al (2016) Efficient mobile implementation of a CNN-based object recognition system. ACM on Multimedia Conference. ACM, pp 362–366

  30. Yang S, Chen M, Pomerleau D, et al (2010) Food recognition using statistics of pairwise local features. In: Computer Vision & Pattern Recognition, pp 2249–2256

  31. Yang H, Zhang D, Lee D-J, et al (2016) A sparse representation based classification algorithm for Chinese food recognition. In: International Symposium on Visual Computing Springer, pp 3–10

  32. Zhang W, Zhao D, Gong W, et al (2016) Food image recognition with convolutional neural networks. Ubiquitous Intelligence and Computing and 2015 IEEE, Intl Conf on Autonomic and Trusted Computing and 2015 IEEE, Intl Conf on Scalable Computing and Communications and ITS Associated Workshops. IEEE, pp 690–693

  33. Zhao J, Han J, Shao L Unconstrained Face Recognition Using A Set-to-Set Distance Measure on Deep Learned Features. IEEE Transactions on Circuits and Systems for Video Technology. https://doi.org/10.1109/TCSVT.2017.2710120

    Article  Google Scholar 

  34. Zhu F, Marc B, Insoo W et al (2010) The use of mobile devices in aiding dietary assessment and evaluation. IEEE Journal of Selected Topics in Signal Processing 4(4):756–766

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronics and Information Technology, Sun Yat-sen University, Guangzhou, China

    Jianing Teng & Dong Zhang

  2. SYSU-CMU Shunde International Joint Research Institute, Shunde, Foshan, China

    Jianing Teng, Dong Zhang, Dah-Jye Lee & Yao Chou

  3. Department of Electrical and Computer Engineering, Brigham Young University, Provo, UT, USA

    Dah-Jye Lee & Yao Chou

Authors
  1. Jianing Teng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dah-Jye Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yao Chou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dong Zhang.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Teng, J., Zhang, D., Lee, DJ. et al. Recognition of Chinese food using convolutional neural network. Multimed Tools Appl 78, 11155–11172 (2019). https://doi.org/10.1007/s11042-018-6695-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6695-9

Keywords

Search

Navigation