Recognizing Daily Activities from First-Person Videos with Multi-task Clustering

Abstract

The widespread adoption of low-cost wearable devices requires novel paradigms for analysing human behaviour. In particular, when focusing on first-person cameras continuously recording several hours of the users life, the task of activity recognition is especially challenging. As a huge amount of unlabeled data is automatically generated in this scenario, despite recent notable attempts, more scalable algorithms and more effective feature representations are required. In this paper, we address the problem of everyday activity recognition from visual data gathered from a wearable camera proposing a novel multi-task learning framework. We argue that, even if label information is not provided, we can take advantage of the fact that the tasks of recognizing activities of daily life of multiple individuals are related, i.e. typically people tend to perform the same actions in the same environment (e.g. people at home in the morning typically have breakfast and brush their teeth). To exploit this information we propose a novel multi-task clustering approach. With our method, rather than clustering data from different users separately, we look for data partitions which are similar among related tasks. Thorough experiments on two publicly available first-person vision datasets demonstrate that the proposed approach consistently and significantly outperforms several state-of-the-art methods.