Clustering of Food Intake Images into Food and Non-food Categories

Abstract

For dietary assessment, food images could be useful to identify foods, portion sizes and estimate calorie in meals, either by human nutritionists or through image recognition. Images captured by a wearable camera during eating may include both food and non-food images. To avoid reviewing each image, only informative food images should be included in the analysis and non-food image should be discarded. This study proposed a methodology for clustering of food images into food and non-food groups based on histogram matching, without explicit recognition of the image content. Data was collected from 7 participants wearing an eyeglasses camera. A total of 10 meals were recorded at the sampling rate of 5 s, yielding a total of 1077 images. Each image was labeled by a human rater as a food or non-food image. Histogram matching with Bhattacharyya distance was applied to form a similarity matrix for extracted images from each meal. Both k-means and affinity propagation (AP) algorithms were investigated to cluster the images. Results show that the overall average food image clustering accuracy in respect to human annotation for 10 videos was 0.93 ± 0.04 for AP and 0.90 ± 0.03 for k-means. Similarly, overall average non-food image clustering accuracy was 0.81 ± 0.06 for AP and 0.70 ± 0.09 k-means method.