A Microcontroller-Based System for Human-Emotion Recognition with Edge-AI and Infrared Thermography

Abstract

Infrared thermography has shown great promise as a diagnostic method for health care, providing useful information on a person's physiological, and pathological state. Recently, the use of artificial intelligence combined with infrared technology has boosted the adoption of thermal imaging in various applications and has been proposed to recognize human emotion by measuring facial skin temperature. However, its application has been limited to laboratory settings due to demanding computational and hardware resources. In this scenario, this work presents the design and development of a portable system based on a low power microcontroller implementing an optimized Edge-AI solution for binary emotional state classification using minimal hardware resources. The recognition of happiness and sadness emotional states induced by audiovisual stimuli serves as a case-study for feasibility assessment. Thermal images, produced by an uncooled and low-cost thermal sensor, along with electrocardiogram, are acquired and processed with an Arm® Cortex®-M4 microcontroller. A simple, yet effective neural network has been developed, optimized, and deployed to run the emotion detection algorithm in real time. The complete system has been experimentally verified and results in terms of accuracy and hardware constraints are discussed. Specifically, by employing a dataset consisting of 60 infrared videos, an accuracy of 80% was achieved with a resource occupation of 3.4 kB of RAM and 76.4 kB of flash memory.