Skip to main content
SpringerLink
Log in
Book cover

International Conference on Testbeds and Research Infrastructures

TridentCom 2019: Testbeds and Research Infrastructures for the Development of Networks and Communications pp 18–35Cite as

Food Recognition and Dietary Assessment for Healthcare System at Mobile Device End Using Mask R-CNN

  • Conference paper
  • First Online:

Abstract

Monitoring and estimation of food intake is of great significance to health-related research, such as obesity management. Traditional dietary records are performed in manual way. These methods are of low efficiency and a waste of labor, which are highly dependent on human interaction. In recent years, some researches have made progress in the estimation of food intake by using the computer vision technology. However, the recognition results of these researches are usually for the whole food object in the image, and the accuracy is not high. In terms of this problem, we provide a method to the food smart recognition and automatic dietary assessment on the mobile device. First, the food image is processed by MASK R-CNN which is more efficient than traditional methods. And more accurate recognition, classification and segmentation results of the multiple food items are output. Second, the OpenCV is used to display the food category and the corresponding food information of unit volume on the recognition page. Finally, in order to facilitate daily use, TensorFlow Lite is used to process the model to transplant to the mobile device, which can help to monitor people’s dietary intake.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. WHO Homepage. https://www.who.int/zh/news-room/fact-sheets/detail/obesity-and-overweight. Accessed 20 Oct 2019

  2. He, K., Gkioxari, G., Dollar, P., et al.: Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell. PP(99), 1 (2017)

    Google Scholar 

  3. Parrish, E., Goksel, A.K.: Pictorial pattern recognition applied to fruit harvesting. Trans. ASAE 20, 822–827 (1977)

    Article  Google Scholar 

  4. Yang, S., Chen, M., Pomerleau, D., et al.: Food recognition using statistics of pairwise local features. In: The Twenty-Third IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2010, San Francisco, CA, USA, 13–18 June 2010. IEEE (2010)

    Google Scholar 

  5. Matsuda, Y., Hoashi, H., Yanai, K.: Recognition of multiple-food images by detecting candidate regions. In: 2012 IEEE International Conference on Multimedia and Expo (ICME). IEEE (2012)

    Google Scholar 

  6. Martinel, N., Foresti, G.L., Micheloni, C.: Wide-slice residual networks for food recognition. In: Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018, vol. 2018-January, pp. 567–576, December 2016

    Google Scholar 

  7. He, H., Kong, F., Tan, J.: DietCam: multiview food recognition using a multikernel SVM. IEEE J. Biomed. Health Inf. 20(3), 848–855 (2017)

    Article  Google Scholar 

  8. Williamson, D.A., Allen, H.R.: Digital photography: a new method for estimating food intake in cafeteria settings. Eat. Weight Disord. – Stud. Anorexia Bulimia Obes. 9(1), 24–28 (2004)

    Article  Google Scholar 

  9. Noronha, J., Hysen, E., Zhang, H., Gajos, K.Z.: Platemate. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology - UIST 2011, p. 1 (2011)

    Google Scholar 

  10. Puri, M., Zhu, Z., Yu, Q., et al.: Recognition and volume estimation of food intake using a mobile device. In: IEEE Workshop on Applications of Computer Vision (WACV 2009), Snowbird, UT, USA, 7–8 December 2009. IEEE (2009)

    Google Scholar 

  11. Joutou, T., Yanai, K.: A food image recognition system with multiple kernel learning. In: IEEE International Conference on Image Processing. IEEE Press (2009)

    Google Scholar 

  12. Kong, F., Tan, J.: DietCam: automatic dietary assessment with mobile camera phones. Pervasive Mob. Comput. 8(1), 147–163 (2012)

    Article  Google Scholar 

  13. Kawano, Y., Yanai, K.: Real-time mobile food recognition system. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops. IEEE (2013)

    Google Scholar 

  14. Kawano, Y., Yanai, K.: FoodCam: a real-time food recognition system on a smartphone. Multimed. Tools Appl. 74(14), 5263–5287 (2015)

    Article  Google Scholar 

  15. Pouladzadeh, P., Shirmohammadi, S., Arici, T.: Intelligent SVM based food intake measurement system. In: IEEE International Conference on Computational Intelligence & Virtual Environments for Measurement Systems & Applications. IEEE (2013)

    Google Scholar 

  16. Boohee Homepage. http://www.boohee.com/food/. Accessed 20 Oct 2019

  17. COCO dataset Homepage. http://cocodataset.org/. Accessed 20 Oct 2019

Download references

Author information

Authors and Affiliations

  1. School of Computer Engineering and Science, Shanghai University, Shanghai, 200444, China

    Hui Ye

  2. Computing Center, Shanghai University, Shanghai, 200444, China

    Qiming Zou

Authors
  1. Hui Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qiming Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiming Zou .

Editor information

Editors and Affiliations

  1. Shanghai University, Shanghai, China

    Honghao Gao

  2. School of Software, Central South University, Changsha, China

    Kuang Li

  3. Shanghai Polytechnic University, Shanghai, China

    Xiaoxian Yang

  4. Hangzhou Dianzi University, Hangzhou Shi, Zhejiang, China

    Yuyu Yin

Rights and permissions

Reprints and permissions

Copyright information

© 2020 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ye, H., Zou, Q. (2020). Food Recognition and Dietary Assessment for Healthcare System at Mobile Device End Using Mask R-CNN. In: Gao, H., Li, K., Yang, X., Yin, Y. (eds) Testbeds and Research Infrastructures for the Development of Networks and Communications. TridentCom 2019. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 309. Springer, Cham. https://doi.org/10.1007/978-3-030-43215-7_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-43215-7_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-43214-0

  • Online ISBN: 978-3-030-43215-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation