GrillCam: A Real-Time Eating Action Recognition System

Abstract

In this demo, we demonstrate a mobile real-time eating action recognition system, GrillCam. It continuously recognizes user’s eating action and estimates categories of eaten food items during mealtime. With this system, we can get to know total amount of eaten food items, and can calculate total calorie intake of eaten foods even for the meals where the amount of foods to be eaten is not decided before starting eating. The system implemented on a smartphone continuously monitors eating actions during mealtime. It detects the moment when a user eats foods, extract food regions near the user’s mouth and classify them. As a prototype system, we implemented a mobile system the target of which are Japanese-style meals, “Yakiniku” and “Oden”. It can recognize five different kinds of ingredients for each of “Yakiniku” and “Oden” in the real-time way with classification rates, 87.7 % and 80.8 %, respectively. It was evaluated as being superior to the baseline system which employed no eating action recognition by user study.