FASENet: A Two-Stream Fall Detection and Activity Monitoring Model Using Pose Keypoints and Squeeze-and-Excitation Networks

Abstract

Numerous frameworks have already been proposed for vision-based fall detection and activity monitoring. These works have leveraged state-of-the-art algorithms such as 2D and 3D convolutional neural networks in order to analyze and process video data. However, these models are computationally expensive which prevent their use at scale for low-resource devices. Moreover, previous works in literature have not considered modelling features for simple and complex actions given a video segment. This information is crucial when trying to identify actions for a given task. Hence, this work proposes an architecture called FASENet, a 1D convolutional neural network-based two-stream fall detection and activity monitoring model using squeeze-and-excitation networks. By using pose keypoints as inputs for the model instead of raw video frames, it is able to use 1D convolutions which is computationally cheaper than using 2D or 3D convolutions, thereby making the architecture more efficient. Furthermore, FASENet primarily has two streams to process pose segments, a compact and dilated stream which aims to extract features for simple and complex actions, respectively. Furthermore, squeeze-and-excitation networks are used in between these streams to recalibrate features after their combination based on their importance. The network was evaluated in three publicly available datasets, the Adhikari Dataset, the UP-Fall Dataset, and the UR-Fall Dataset. Through the experiments, FASENet was able to outperform prior state-of-the-art work on the Adhikari Dataset for accuracy, precision, and F1. The model was shown to have the best precision on the UP-Fall and UR-Fall Datasets. Finally, it was also observed that FASENet was able to reduce false positive rates compared to a previously related study.