Abstract
Accidental falls are the second leading cause of accidental death of the elderly. Early intervention measures can reduce the problem. However, so far, there are few related studies using Transformer coding module for fall detection feature extraction, and the real-time performance of existing algorithms is not so good. Therefore, we propose a fall detection method based on Transformer to extract spatiotemporal features. Specifically, we use an image reduction module based on a convolutional neural network to reduce the image size for computation. Then, we design a pyramid network based on an improved Transformer to extract spatial features. Finally, we design a feature fusion module that fuses spatial features of different scales. The fused features are input into the gate recurrent unit to extract time features and complete the recognition of falls and normal postures. Experimental results show that the proposed approach achieves an accuracy of 99.61% and 99.33% when tested with UR Fall Detection Dataset and Le2i Fall Detection Dataset. Compared with the state-of-the-art fall detection algorithms, our method has high accuracy while maintaining high detection speed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
Data generated during the current study will be made available at reasonable request.
References
Alarifi A, Alwadain A (2021) Killer heuristic optimized convolution neural network-based fall detection with wearable iot sensor devices. Measurement 167:108258–108267. https://doi.org/10.1016/j.measurement.2020.108258
Amsaprabhaa M (2022) Multimodal spatiotemporal skeletal kinematic gait feature fusion for vision-based fall detection. Expert Syst Appl 212:118681–118695. https://doi.org/10.1016/j.eswa.2022.118681
Beddiar DR, Oussalah M, Nini B (2022) Fall detection using body geometry and human pose estimation in video sequences. J Vis Commun Image Represent 82:103407–103419. https://doi.org/10.1016/j.jvcir.2021.103407
Belhadi A, Djenouri Y, Djenouri D et al (2020) A recurrent neural network for urban long-term traffic flow forecasting. Appl Intell 50(10):3252–3265. https://doi.org/10.1007/s10489-020-01716-1
Cao Z, Simon T, Wei SE et al (2017) Realtime multi-person 2d pose estimation using part affinity fields. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 1302–1310
Charfi I, Miteran J, Dubois J et al (2012) Definition and performance evaluation of a robust SVM based fall detection solution. In: 2012 eighth international conference on signal image technology and internet based Systems, pp 218–224
De A, Saha A, Kumar P et al (2022) Fall detection method based on spatio-temporal feature fusion using combined two-channel classification. Multimed Tools Appl 81(18):26081–26100. https://doi.org/10.1007/s11042-022-11914-3
Dentamaro V, Impedovo D, Pirlo G (2021) Fall detection by human pose estimation and kinematic theory. In: 2020 25th international conference on pattern recognition (ICPR), pp 2328–2335
Fei K, Wang C, Zhang J et al (2022) Flow-pose net: an effective two-stream network for fall detection. Vis Comput. https://doi.org/10.1007/s00371-022-02416-2
Galvão YM, Ferreira J, Albuquerque VA et al (2021) A multimodal approach using deep learning for fall detection. Expert Syst Appl 168:114226–114234. https://doi.org/10.1016/j.eswa.2020.114226
Hadjadji B, Saumard M, Aron M (2022) Multi-oriented run length based static and dynamic features fused with choquet fuzzy integral for human fall detection in videos. J Vis Commun Image Represent 82:103375–103388. https://doi.org/10.1016/j.jvcir.2021.103375
Hanifi K, Karsligil ME (2021) Elderly fall detection with vital signs monitoring using cw doppler radar. IEEE Sens J 21(15):16969–16978. https://doi.org/10.1109/JSEN.2021.3079835
Islam MM, Tayan O, Islam MR et al (2020) Deep learning based systems developed for fall detection: a review. IEEE Access 8:166117–166137. https://doi.org/10.1109/ACCESS.2020.3021943
Kerdjidj O, Boutellaa E, Amira A et al (2022) A hardware framework for fall detection using inertial sensors and compressed sensing. Microprocess Microsyst 91:104514–104521. https://doi.org/10.1016/j.micpro.2022.104514
Kwolek B, Kepski M (2014) Human fall detection on embedded platform using depth maps and wireless accelerometer. Comput Methods Progr Biomed 117(3):489–501. https://doi.org/10.1016/j.cmpb.2014.09.005
Liu Z, Ning J, Cao Y et al (2022) Video swin transformer. In: 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 3192–3201
Mamchur N, Shakhovska N, Gregusml M (2022) Person fall detection system based on video stream analysis. Procedia Comput Sci 198:676–681. https://doi.org/10.1016/j.procs.2021.12.305
Nasir M, Muhammad K, Ullah A et al (2022) Enabling automation and edge intelligence over resource constraint iot devices for smart home. Neurocomputing 491:494–506. https://doi.org/10.1016/j.neucom.2021.04.138
Ren L, Peng Y (2019) Research of fall detection and fall prevention technologies: a systematic review. IEEE Access 7:77702–77722. https://doi.org/10.1109/ACCESS.2019.2922708
Şengül G, Karakaya M, Misra S et al (2022) Deep learning based fall detection using smartwatches for healthcare applications. Biomed Signal Process Control 71:1–13. https://doi.org/10.1016/j.bspc.2021.103242
Soni PK, Choudhary A (2022) Grassmann manifold based framework for automated fall detection from a camera. Image Vis Comput 122:104431–104439. https://doi.org/10.1016/j.imavis.2022.104431
Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need. CoRR. https://arxiv.org/abs/1706.03762
Wahla SQ, Ghani MU (2023) Visual fall detection from activities of daily living for assistive living. IEEE Access 11:108876–108890. https://doi.org/10.1109/ACCESS.2023.3321192
Wang BH, Yu J, Wang K et al (2020) Fall detection based on dual-channel feature integration. IEEE Access 8:103443–103453. https://doi.org/10.1109/ACCESS.2020.2999503
Wang B, Zheng Z, Guo YX (2022a) Millimeter-wave frequency modulated continuous wave radar-based soft fall detection using pattern contour-confined doppler-time maps. IEEE Sens J 22(10):9824–9831. https://doi.org/10.1109/JSEN.2022.3165188
Wang P, Li Q, Yin P et al (2022b) A convolution neural network approach for fall detection based on adaptive channel selection of uwb radar signals. Neural Comput Appl. https://doi.org/10.1007/s00521-021-06795-w
Wei X, Li Y, Li Y et al (2022) Detection of false data injection attacks in smart grid: a secure federated deep learning approach. IEEE Trans Smart Grid 13(6):4862–4872. https://doi.org/10.1109/TSG.2022.3204796
Wu X, Zheng Y, Chu CH et al (2022) Applying deep learning technology for automatic fall detection using mobile sensors. Biomed Signal Process Control 72:103355–103363. https://doi.org/10.1016/j.bspc.2021.103355
Wu L, Huang C, Fei L et al (2024) Video-based fall detection using human pose and constrained generative adversarial network. IEEE Trans Circuits Syst Video Technol 34(4):2179–2194. https://doi.org/10.1109/TCSVT.2023.3303258
Xiao Z, Zhang H, Tong H et al (2022) An efficient temporal network with dual self-distillation for electroencephalography signal classification. In: 2022 IEEE international conference on bioinformatics and biomedicine (BIBM), pp 1759–1762
Xing H, Xiao Z, Qu R et al (2021) An efficient federated distillation learning system for multitask time series classification. IEEE Trans Instrum Meas 71:1–12. https://doi.org/10.1109/TIM.2022.3201203
Xing H, Xiao Z, Zhan D et al (2022) Selfmatch: robust semisupervised time-series classification with self-distillation. Int J Intell Syst 37(11):8583–8610. https://doi.org/10.1002/int.22957
Xiong X, Min W, Zheng WS et al (2020) S3d-cnn: skeleton-based 3d consecutive-low-pooling neural network for fall detection. Appl Intell 50(10):3521–3534. https://doi.org/10.1007/s10489-020-01751-y
Yadav SK, Luthra A, Tiwari K et al (2022) Arfdnet: an efficient activity recognition & fall detection system using latent feature pooling. Knowl Based Syst 239:107948–107958. https://doi.org/10.1016/j.knosys.2021.107948
Yang Y, Yang H, Liu Z et al (2022) Fall detection system based on infrared array sensor and multi-dimensional feature fusion. Measurement 192:110870–110879. https://doi.org/10.1016/j.measurement.2022.110870
Yao Y, Liu C, Zhang H et al (2022) Fall detection system using millimeter wave radar based on neural network and information fusion. IEEE Internet Things J. https://doi.org/10.1109/JIOT.2022.3175894
Zhang Y, Gan J, Zhao Z et al (2023) A real-time fall detection model based on blazepose and improved st-gcn. J Real Time Image Process 20(6):121. https://doi.org/10.1007/s11554-023-01377-6
Zou W, Zhang D, Lee DJ (2022) A new multi-feature fusion based convolutional neural network for facial expression recognition. Appl Intell 52(3):2918–2929. https://doi.org/10.1007/s10489-021-02575-0
Funding
Funding was provided by National Natural Science Foundation of China (61973185), Natural Science Foundation of Shandong Province (ZR2020MF097), Colleges and Universities Twenty Terms Foundation of Jinan City (2021GXRC100).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Li, J., Gao, M., Wang, P. et al. Fall detection algorithm based on pyramid network and feature fusion. Evolving Systems 15, 1957–1970 (2024). https://doi.org/10.1007/s12530-024-09601-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12530-024-09601-9