Abstract
This paper presents a user-biased food recognition system. The presented approach has been developed in the context of the FoodRec project, which aims to define an automatic framework for the monitoring of people’s health and habits, during their smoke quitting program. The goal of food recognition is to extract and infer semantic information from the food images to classify diverse foods present in the image. We propose a novel Deep Convolutional Neural Network able to recognize food items of specific users and monitor their habits. It consists of a food branch to learn visual representation for the input food items and a user branch to take into account the specific user’s eating habits. Furthermore, we introduce a new FoodRec-50 dataset with 2000 images and 50 food categories collected by the iOS and Android smartphone applications, taken by 164 users during their smoking cessation therapy. The information inferred from the users’ eating habits is then exploited to track and monitor the dietary habits of people involved in a smoke quitting protocol. Experimental results show that the proposed food recognition method outperforms the baseline model results on the FoodRec-50 dataset. We also performed an ablation study which demonstrated that the proposed architecture is able to tune the prediction based on the users’ eating habits.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ortis, A., Farinella, G.M., Battiato, S.: Survey on visual sentiment analysis. IET Image Proc. 14(8), 1440–1456 (2020)
Ortis, A., Caponnetto, P., Polosa, R., Urso, S., Battiato, S.: A report on smoking detection and quitting technologies. Int. J. Environ. Res. Public Health 17(7), 2614 (2020)
Battiato, S., et al.: Food recognition for dietary monitoring during smoke quitting. In: IMPROVE, pp. 160–165 (2021)
Maguire, G., Chen, H., Schnall, R., Xu, W., Huang, M.C.: Smoking cessation system for preemptive smoking detection. IEEE Internet Things J. 9(5), 3204–3214 (2021)
Nishida, C., Uauy, R., Kumanyika, S., Shetty, P.: The joint WHO/FAO expert consultation on diet, nutrition and the prevention of chronic diseases: process, product and policy implications. Public Health Nutr. 7(1a), 245–250 (2004)
Kitamura, K., De Silva, C., Yamasaki, T., Aizawa, K.: Image processing based approach to food balance analysis for personal food logging. In: 2010 IEEE International Conference on Multimedia and Expo, pp. 625–630. IEEE, July 2010
Farinella, G.M., Allegra, D., Moltisanti, M., Stanco, F., Battiato, S.: Retrieval and classification of food images. Comput. Biol. Med. 77, 23–39 (2016)
Le, Q., Mikolov, T.: Distributed representations of sentences and documents. In: International Conference on Machine Learning, pp. 1188–1196. PMLR, June 2014
Min, W., Jiang, S., Liu, L., Rui, Y., Jain, R.: A survey on food computing. ACM Comput. Surv. (CSUR) 52(5), 1–36 (2019)
Allegra, D., Battiato, S., Ortis, A., Urso, S., Polosa, R.: A review on food recognition technology for health applications. Health Psychol. Res. 8(3), 9297 (2020)
Fakhrou, A., Kunhoth, J., Al Maadeed, S.: Smartphone-based food recognition system using multiple deep CNN models. Multimed. Tools Appl. 80, 33011–33032 (2021). https://doi.org/10.1007/s11042-021-11329-6
Lu, Y., Stathopoulou, T., Vasiloglou, M.F., Christodoulidis, S., Stanga, Z., Mougiakakou, S.: An artificial intelligence-based system to assess nutrient intake for hospitalised patients. IEEE Trans. Multimed. 23, 1136–1147 (2020)
Pfisterer, K.J., Amelard, R., Chung, A.G., Syrnyk, B., MacLean, A., Wong, A.: Fully-automatic semantic segmentation for food intake tracking in long-term care homes. arXiv e-prints, arXiv-1910 (2019)
Ciocca, G., Napoletano, P., Schettini, R.: Food recognition: a new dataset, experiments, and results. IEEE J. Biomed. Health Inform. 21(3), 588–598 (2016)
Mandal, B., Puhan, N.B., Verma, A.: Deep convolutional generative adversarial network-based food recognition using partially labeled data. IEEE Sens. Lett. 3(2), 1–4 (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1492–1500 (2017)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)
Acknowledgments
This investigator initiated study was sponsored by ECLAT srl, a spin-off of the University of Catania, with the help of a grant from the Foundation for a Smoke-Free World Inc., a US nonprofit 501(c)(3) private foundation with a mission to end smoking in this generation. The contents, selection, and presentation of facts, as well as any opinions expressed herein are the sole responsibility of the authors and under no circumstances shall be regarded as reflecting the positions of the Foundation for a Smoke-Free World, Inc. ECLAT srl. is a research based company from the University of Catania that delivers solutions to global health problems with special emphasis on harm minimization and technological innovation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Hussain, M., Ortis, A., Polosa, R., Battiato, S. (2022). User-Biased Food Recognition for Health Monitoring. In: Sclaroff, S., Distante, C., Leo, M., Farinella, G.M., Tombari, F. (eds) Image Analysis and Processing – ICIAP 2022. ICIAP 2022. Lecture Notes in Computer Science, vol 13233. Springer, Cham. https://doi.org/10.1007/978-3-031-06433-3_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-06433-3_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06432-6
Online ISBN: 978-3-031-06433-3
eBook Packages: Computer ScienceComputer Science (R0)