Abstract
Currently, many segmentation image datasets are open to the public. However, only a few open segmentation image dataset of food images exists. Among them, UEC-FoodPix is a large-scale food image segmentation dataset which consists of 10,000 food images with segmentation masks. However, it contains some incomplete mask images, because most of the segmentation masks were generated automatically based on the bounding boxes. To enable accurate food segmentation, complete segmentation masks are required for training. Therefore, in this work, we created “UEC-FoodPix Complete” by refining the 9,000 segmentation masks by hand which were automatically generated in the previous UEC-FoodPix. As a result, the segmentation performance was much improved compared to the segmentation model trained with the original UEC-FoodPix. In addition, as applications of the new food segmentation dataset, we performed food calorie estimation using the food segmentation models trained with “UEC-FoodPix Complete”, and food image synthesis from segmentation masks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Allegra, D., et al.: A multimedia database for automatic meal assessment systems. In: Proceedings of the ICIAP Workshop on Multimedia Assisted Dietary Management (2017)
Aslan, S., Ciocca, G., Mazzini, D., Schettini, R.: Benchmarking algorithms for food localization and semantic segmentation. Int. J. Mach. Learn. Cybern. 11(12), 2827–2847 (2020). https://doi.org/10.1007/s13042-020-01153-z
Bossard, L., Guillaumin, M., Van Gool, L.: Food-101 - mining discriminative components with random forests. In: Proc. of European Conference on Computer Vision (2014)
Chen, J., Ngo, C.W.: Deep-based ingredient recognition for cooking recipe retrival. In: Proceedings of ACM International Conference Multimedia (2016)
Chen, L., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of European Conference on Computer Vision (2018)
Chen, M.Y., et al.: Automatic Chinese food identification and quantity estimation. In: Proceedings of SIGGRAPH Asia (2012)
Ciocca, G., Napoletano, P., Schettini, R.: Food recognition: a new dataset, experiments and results. IEEE J. Biomed. Health Informat. 21(3), 588–598 (2017)
Ege, T., Yanai, K.: A new large-scale food image segmentation dataset and its application to food calorie estimation based on grains of rice. In: Proceedings of ACM MM Workshop on Multimedia Assisted Dietary Management (2019)
Everingham, M., Eslami, S., Van Gool, L., Williams, C., Winn, J., Zisserman, A.: The pascal visual object classes challenge: a retrospective. Int. J. Comput. Vision 88(2) (2010)
Gao, J., Tan, W., Ma, L., Wang, Y., Tang, W.: MUSEFood: multi-sensor-based food volume estimation on smartphones. arXiv:1903.07437 (2019)
He, K., Gkioxari, G., Dollar, P., Girshick, R.: Mask R-CNN. In: Proceedings of IEEE International Conference on Computer Vision (2017)
Kawano, Y., Yanai, K.: Automatic expansion of a food image dataset leveraging existing categories with domain adaptation. In: Proc. of ECCV Workshop on Transferring and Adapting Source Knowledge in Computer Vision (TASK-CV) (2014)
Lin, T., et al.: Microsoft coco: common objects in context. In: Proceedings of European Conference on Computer Vision (2014)
Lu, Y., Allegra, D., Anthimopoulos, M., Stanco, F., Farinella, G.M., Mougiakakou, S.: A multi-task learning approach for meal assessment. In: Proceedings of the IJCAI Joint Workshop on Multimedia for Cooking and Eating Activities and Multimedia Assisted Dietary Management, pp. 46–52 (2018)
Matsuda, Y., Hajime, H., Yanai, K.: Recognition of multiple-food images by detecting candidate regions. In: Proceedings of IEEE International Conference on Multimedia and Expo, pp. 25–30 (2012)
Myers, A., et al.: Im2Calories: towards an automated mobile vision food diary. In: Proceedings of IEEE International Conference on Computer Vision, pp. 1233–1241 (2015)
Okamoto, K., Yanai, K.: An automatic calorie estimation system of food images on a smartphone. In: Proceedings of ACM MM Workshop on Multimedia Assisted Dietary Management (2016)
Park, T., Liu, M., Zhu, J.: Semantic image synthesis with spatially-adaptive normalization. In: Proceedings of IEEE Computer Vision and Pattern Recognition (2019)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN:towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)
Rother, C., Kolmogorov, V., Blake, A.: “GrabCut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23(3), 309–314 (2004)
Shimoda, W., Yanai, K.: Predicting plate regions for weakly-supervised food image segmentation. In: Proceedings of IEEE International Conference on Multimedia and Expo (2020)
Tangseng, P., Wu, Z., Yamaguchi, K.: Looking at outfit to parse clothing. arXiv:1703.01386 (2017)
Acknowledgments
We would like to thank all those who worked on pixel-wise annotation for creating “UEC-FoodPix Complete.” This work was supported by JSPS KAKENHI Grant Number 17J10261, 15H05915, 17H01745, and 19H04929.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Okamoto, K., Yanai, K. (2021). UEC-FoodPix Complete: A Large-Scale Food Image Segmentation Dataset. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12665. Springer, Cham. https://doi.org/10.1007/978-3-030-68821-9_51
Download citation
DOI: https://doi.org/10.1007/978-3-030-68821-9_51
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68820-2
Online ISBN: 978-3-030-68821-9
eBook Packages: Computer ScienceComputer Science (R0)
