Detecting Emotions During Cognitive Stimulation Training with the Pepper Robot

Abstract

Recently, social robots are being used in therapeutic interventions for elderly people affected by cognitive impairments. In this paper, we report the results of a study aiming at exploring the affective reactions of seniors during the cognitive stimulation therapy performed using a social robot. To this purpose an experimental study was performed with a group of 8 participants in a 3-weeks program in which the group was trained on specific memory tasks with the support of the Pepper robot. To assess and monitor the results, each session was video-recorded for human and automatic analyses. Given that aging causes many changes in facial shape and appearance, we detected emotions by means of a model specifically trained for recognizing facial expressions of elderly people. After testing the model accuracy and analyzing the differences with the human annotation, we used it to analyze automatically the collected videos. Results show that the model was able to detect a low number of neutral emotions and a high number of negative emotions. However, seniors showed also positive emotions during the various sessions and, while these were much higher than negative ones in the human annotation, this difference was smaller in the automatic detection. These results encourage the development of a module to adapt the interaction and the tasks to the user’s reactions in real time. In both cases, some correlations emerged showing that seniors with a lower level of cognitive impairment experienced fewer positive emotions than seniors with a more severe impairment measured with the Mini–Mental State Examination (MMSE). In our opinion, this could be due to the need for personalized cognitive stimulation therapy according to the senior’s MMSE thus providing more stimulating tasks. However, a deeper investigation should be conducted to confirm this hypothesis.