Abstract
Fall detection (FD) system tends to monitor the fall events with restricted movement patterns and provides alerts to detect actions and corresponds to human falls. Based on high-level features, the resultant information often requires well-detected results like activity monitoring, detection, and classification. The objective of the study focuses on the vision-based FD and activity monitoring (AM) methods using different types of cameras and determines the finest method for different backgrounds and complex surroundings in outdoor and indoor scenes. Several works of literature provide various detection algorithms which cannot differentiate the fall from other actions. So, there is a need for efficient detection techniques which can efficiently work on all sorts of fall event images. Also, the AM algorithm lies in different classification techniques but it is not robust to classify the actions being the same speed with the fall such as jumping, bending, etc. In this paper, we view the comparative study of vision-based FD and monitoring techniques such as Inactivity/Body shape change based, Posture based, 3D head motion-based, Spatial–temporal based, Gait based and skeleton tracking techniques based on the source of their techniques, types, description, advantages, and disadvantages. In addition, several performance metrics were used to evaluate the results and compare the resulting study with the previous comparative evaluations. This comparative analysis leads to a deeper understanding of different FD and AM techniques and suggests the possible direction for the researchers to identify a suitable method for their needs.
Similar content being viewed by others
References
Abobakr A, Hossny M, Nahavandi S (2017) A skeleton-free fall detection system from depth images using random decision forest. IEEE Syst J 12(3):2994–3005
Aguiar B, Rocha T, Silva J, Sousa I (2014) Accelerometer-based fall detection for smartphones. In: 2014 IEEE international symposium on medical measurements and applications (MeMeA), pp 1–6
Alaoui AY, El Hassouny A, Thami RO, Tairi H (2017) Video based human fall detection using von mises distribution of motion vectors. In: 2017 Intelligent systems and computer vision (ISCV). IEEE, pp 1–5
Alaoui AY, El Fkihi S, Thami ROH (2019) Fall detection for elderly people using the variation of key points of human skeleton. IEEE Access 7:154786–154795
Alhimale L, Zedan H, Al-Bayatti A (2014) The implementation of an intelligent and video-based fall detection system using a neural network. Appl Soft Comput 18:59–69
Anishchenko L (2018) Machine learning in video surveillance for fall detection. In: 2018 ural symposium on biomedical engineering, radioelectronics and information technology (USBEREIT). IEEE, pp 99–102
Ariz M, Bengoechea JJ, Villanueva A, Cabeza R (2016) A novel 2D/3D database with automatic face annotation for head tracking and pose estimation. Comput vis Image Understanding 148:201–210
Ariz M, Villanueva A, Cabeza R (2019) Robust and accurate 2D-tracking-based 3D positioning method: application to head pose estimation. Comput vis Image Understanding 180:13–22
Beddiar DR, Nini B (2017) Vision based abnormal human activities recognition: an overview. In: 2017 8th international conference on information technology (ICIT). IEEE, pp 548–553
Bian ZP, Hou J, Chau LP, Magnenat-Thalmann N (2014) Fall detection based on body part tracking using a depth camera. IEEE J Biomed Health Inform 19(2):430–439
Biswas A, Dey B (2019) A literature review of current vision based fall detection methods. In: International conference on communication, devices and networking. Springer, Singapore, pp 411–421
Braham M, Van Droogenbroeck M (2016) Deep background subtraction with scene-specific convolutional neural networks. In: 2016 international conference on systems signals and image processing (IWSSIP) IEEE, pp 1–4
Brunetti A, Buongiorno D, Trotta GF, Bevilacqua V (2018) Computer vision and deep learning techniques for pedestrian detection and tracking: a survey. Neurocomputing 300:17–33
Cameron R, Zuo Z, Sexton G, Yang L (2017) A fall detection/recognition system and an empirical study of gradient-based feature extraction approaches. In: UK workshop on computational intelligence, pp 276–289
Chaaraoui AA, Padilla-López JR, Climent-Pérez P, Flórez-Revuelta F (2014) Evolutionary joint selection to improve human action recognition with RGB-D devices. Expert Syst Appl 41(3):786–794
Chaccour K, Darazi R, El Hassani AH, Andres E (2016) From fall detection to fall prevention: a generic classification of fall-related systems. IEEE Sens J 17(3):812–822
Chen Z, Ellis T, Velastin SA (2016) Vision-based traffic surveys in urban environments. J Electron Imaging 25(5):051206
Chua JL, Chang YC, Lim WK (2015) A simple vision-based fall detection technique for indoor video surveillance. SIViP 9(3):623–633
Debard G, Baldewijns G, Goedemé T, Tuytelaars T, Vanrumste B (2015) Camera-based fall detection using a particle filter. In: 2015 37th annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 6947–6950
Espinosa R, Ponce H, Gutiérrez S, Martínez-Villaseñor L, Brieva J, Moya-Albor E (2020) Application of convolutional neural networks for fall detection using multiple cameras. In: Challenges and trends in multimodal fall detection for healthcare. Springer, Cham, pp 97–120
Fan Y, Levine MD, Wen G, Qiu S (2017) A deep neural network for real-time detection of falling humans in naturally occurring scenes. Neurocomputing 260:43–58
Fan K, Wang P, Zhuang S (2019) Human fall detection using slow feature analysis. Multimed Tools Appl 78(7):9101–9128
Feng W, Liu R, Zhu M (2014) Fall detection for elderly person care in a vision-based home surveillance environment using a monocular camera. Signal Image Video Process 8(6):1129–1138
Feng Q, Gao C, Wang L, Zhao Y, Song T, Li Q (2020) Spatio-temporal fall event detection in complex scenes using attention guided LSTM. Pattern Recogn Lett 130:242–249
Gomes V, Barcellos P, Scharcanski J (2017) Stochastic shadow detection using a hypergraph partitioning approach. Pattern Recogn 63:30–44
González I, López-Nava IH, Fontecha J, Muñoz-Meléndez A, Pérez-SanPablo AI, Quiñones-Urióstegui I (2016) Comparison between passive vision-based system and a wearable inertial-based system for estimating temporal gait parameters related to the GAITRite electronic walkway. J Biomed Inform 62:210–223
Gopalakrishnan K, Khaitan SK, Choudhary A, Agrawal A (2017) Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection. Constr Build Mater 157:322–330
Gracewell JJ, Pavalarajan S (2019) Fall detection based on posture classification for smart home environment. J Ambient Intell Human Comput 20:1–8
Gutiérrez J, Rodríguez V, Martin S (2021) Comprehensive review of vision-based fall detection systems. Sensors 21(3):947
Han S, Lee S (2013) A vision-based motion capture and recognition framework for behavior-based safety management. Autom Constr 35:131–141
Hanghan L (2017) Fall detection using wavelet transform and support vector machine. Diss. School of Telecommunication Engineering Institute of Engineering Suranaree University of Technology
Harrou F, Zerrouki N, Sun Y, Houacine A (2017) Vision-based fall detection system for improving safety of elderly people. IEEE Instrum Meas Mag 20(6):49–55
Harrou F, Zerrouki N, Sun Y, Houacine A (2019) An integrated vision-based approach for efficient human fall detection in a home environment. IEEE Access 7:114966–114974
Hassanalieragh M, Page A, Soyata T, Sharma G, Aktas M, Mateos G, Andreescu S (2015) Health monitoring and management using Internet-of-Things (IoT) sensing with cloud-based processing: opportunities and challenges. In: 2015 IEEE international conference on services computing. IEEE, pp 285–292
He Y, Li Y, Bao SD (2012) Fall detection by built-in tri-accelerometer of smartphone. In: Proceedings of 2012 IEEE-EMBS international conference on biomedical and health informatics, pp 184–187
Hsu YW, Perng JW, Liu HL (2015) Development of a vision based pedestrian fall detection system with back propagation neural network. In: 2015 IEEE/SICE international symposium on system integration (SII), pp 433–437
Huang Z, Liu Y, Fang Y, Horn BK (2018) Video-based fall detection for seniors with human pose estimation. In: 2018 4th international conference on universal village (UV). IEEE, pp 1–4
Igual R, Medrano C, Plaza I (2015) A comparison of public datasets for acceleration-based fall detection. Med Eng Phys 37(9):870–878
Jansi R, Amutha R (2020) Detection of fall for the elderly in an indoor environment using a tri-axial accelerometer and Kinect depth data. Multidimen Syst Signal Process 31(4):1207–1225
Jeni LA, Cohn JF, Kanade T (2017) Dense 3d face alignment from 2d video for real-time use. Image vis Comput 58:13–24
Ji X, Cheng J, Tao D, Wu X, Feng W (2017) The spatial Laplacian and temporal energy pyramid representation for human action recognition using depth sequences. Knowl Based Syst 122:64–74
Kataoka H, Aoki Y, Iwata K, Satoh Y (2015) Evaluation of vision-based human activity recognition in dense trajectory framework, In: International symposium on visual computing, pp 634–646
Kepski M, Kwolek B (2014) Fall detection using ceiling-mounted 3d depth camera. In: 2014 International conference on computer vision theory and applications (VISAPP), vol, 2. IEEE, pp 640–647
Kerdegari H, Samsudin K, Ramli AR, Mokaram S (2012) Evaluation of fall detection classification approaches. In: 2012 4th international conference on intelligent and advanced systems (ICIAS2012), vol, 1. IEEE, pp 131–136
Khan SS, Hoey J (2017) Review of fall detection techniques: a data availability perspective. Med Eng Phys 39:12–22
Kianoush S, Savazzi S, Vicentini F, Rampa V, Giussani M (2016) Device-free RF human body fall detection and localization in industrial workplaces. IEEE Internet Things J 4(2):351–362
Kong Y, Huang J, Huang S, Wei Z, Wang S (2019) Learning spatiotemporal representations for human fall detection in surveillance video. J vis Commun Image Represent 59:215–230
Kozina S, Gjoreski H, Gams M, Luštrek M (2013) Efficient activity recognition and fall detection using accelerometers. In: International competition on evaluating AAL systems through competitive benchmarking. Springer, Berlin, Heidelberg, pp 13–23
Kumar G, Bhatia PK (2014) A detailed review of feature extraction in image processing systems. In: 2014 Fourth international conference on advanced computing communication technologies. IEEE, pp 5–12
Kwolek B, Kepski M (2014) Human fall detection on embedded platform using depth maps and wireless accelerometer. Comput Methods Programs Biomed 117(3):489–501
Kwolek B, Kepski M (2016) Fuzzy inference-based fall detection using kinect and body-worn accelerometer. Appl Soft Comput 40:305–318
Lapierre N, Neubauer N, Miguel-Cruz A, Rincon AR, Liu L, Rousseau J (2018) The state of knowledge on technologies and their use for fall detection: a scoping review. Int J Med Inf 111:58–71
Lie WN, Le AT, Lin GH (2018) Human fall-down event detection based on 2D skeletons and deep learning approach. In: 2018 International workshop on advanced image technology (IWAIT), pp 1–4
Liu CL, Lee CH, Lin PM (2010) A fall detection system using k-nearest neighbor classifier. Expert Syst Appl 37(10):7174–7181
Liu J, Xia Y, Tang Z (2021) Privacy-preserving video fall detection using visual shielding information. Vis Comput 37(2):359–370
López-Nava IH, González I, Muñoz-Meléndez A, Bravo J (2015) Comparison of a vision-based system and a wearable inertial-based system for a quantitative analysis and calculation of spatio-temporal parameters. In: Ambient intelligence for health, pp, 116–122
Lotfi A, Albawendi S, Powell H, Appiah K, Langensiepen C (2018) Supporting independent living for older adults; employing a visual based fall detection through analysing the motion and shape of the human body. IEEE Access 6:70272–70282
Lu G, Zhou Y, Li X, Kudo M (2016) Efficient action recognition via local position offset of 3D skeletal body joints. Multimed Tools Appl 75(6):3479–3494
Luvizon DC, Tabia H, Picard D (2017) Learning features combination for human action recognition from skeleton sequences. Pattern Recogn Lett 99:13–20
Ma X, Wang H, Xue B, Zhou M, Ji B, Li Y (2014) Depth-based human fall detection via shape features and improved extreme learning machine. IEEE J Biomed Health Inform 18(6):1915–1922
Malasinghe LP, Ramzan N, Dahal K (2019) Remote patient monitoring: a comprehensive study. Computing 10(1):57–76
Mellone S, Tacconi C, Schwickert L, Klenk J, Becker C, Chiari L (2012) Smartphone-based solutions for fall detection and prevention: the FARSEEING approach, Zeitschrift für Gerontologie und Geriatrie 45(8):b 722–727
Merrouche F, Baha N (2016) Depth camera based fall detection using human shape and movement. In: 2016 IEEE international conference on signal and image processing (ICSIP). IEEE, pp 586–590
Min W, Cui H, Rao H, Li Z, Yao L (2018) Detection of human falls on furniture using scene analysis based on deep learning and activity characteristics. IEEE Access 6:9324–9335
Min W, Zou S, Li J (2019) Human fall detection using normalized shape aspect ratio. Multimed Tools Appl 78(11):14331–14353
Mubashir M, Shao L, Seed L (2013) A survey on fall detection: principles and approaches. Neurocomputing 100:144–152
Nieto-Hidalgo M, Ferrández-Pastor FJ, Valdivieso-Sarabia RJ, Mora-Pascual J, García-Chamizo JM (2016) A vision based proposal for classification of normal and abnormal gait using RGB camera. J Biomed Inform 63:82–89
Nieto-Hidalgo M, García-Chamizo JM (2017) Classification of pathologies using a vision based feature extraction. In: International conference on ubiquitous computing and ambient intelligence. Springer, Cham, pp 265–274
Nogas J, Khan SS, Mihailidis A (2018) DeepFall: non-invasive fall detection with deep spatio-temporal convolutional autoencoders. J Healthc Inf Res 2018:1–21
Nunez JC, Cabido R, Pantrigo JJ, Montemayor AS, Velez JF (2018) Convolutional neural networks and long short-term memory for skeleton-based human activity and hand gesture recognition. Pattern Recogn 76:80–94
Ortells J, Herrero-Ezquerro MT, Mollineda RA (2018) Vision-based gait impairment analysis for aided diagnosis. Med Biol Eng Comput 56(9):1553–1564
Panahi L, Ghods V (2018) Human fall detection using machine vision techniques on RGB–D images. Biomed Signal Process Control 44:146–153
Parra-Dominguez GS, Snoek J, Taati B, Mihailidis A (2015) Lower body motion analysis to detect falls and near falls on stairs. Biomed Eng Lett 5(2):98–108
Paulo J, Asvadi A, Peixoto P, Amorim P (2017) Human gait pattern changes detection system: a multimodal vision-based and novelty detection learning approach. Biocybern Biomed Eng 37(4):701–717
Poonsri A, Chiracharit W (2017) Fall detection using Gaussian mixture model and principle component analysis. In: 2017 9th International conference on information technology and electrical engineering (ICITEE). IEEE, pp 1–4
Rastogi S, Singh J (2021) A systematic review on machine learning for fall detection system. Comput Intell 37:951–974
Romero A, Gatta C, Camps-Valls G (2015) Unsupervised deep feature extraction for remote sensing image classification. IEEE Trans Geosci Remote Sens 54(3):1349–1362
Rougier C, Meunier J, St-Arnaud A, Rousseau J (2013) 3D head tracking for fall detection using a single calibrated camera. Image vis Comput 31(3):246–254
Sanches SR, Oliveira C, Sementille AC, Freire V (2019) Challenging situations for background subtraction algorithms. Appl Intell 49(5):1771–1784
Schwickert L, Becker C, Lindemann U, Maréchal C, Bourke A, Chiari L, Klenk J (2013) Fall detection with body-worn sensors. Z Gerontol Geriatr 46(8):706–719
Sehairi K, Chouireb F, Meunier J (2018) Elderly fall detection system based on multiple shape features and motion analysis. In: 2018 International conference on intelligent systems and computer vision (ISCV). IEEE, pp 1–8
Senouci B, Charfi I, Heyrman B, Dubois J, Miteran J (2016) Fast prototyping of a SoC-based smart-camera: a real-time fall detection case study. J Real-Time Image Proc 12(4):649–662
Shi Y, Shi Y, Wang X (2012) Fall detection on mobile phones using features from a five-phase model. In: 2012 9th international conference on ubiquitous intelligence and computing and 9th international conference on autonomic and trusted computing. IEEE, pp 951–956
Sobral A, Vacavant A (2014) A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos. Comput vis Image Understading 122:4–21
Soni PK, Choudhary A (2018) Automated fall detection using computer vision. In: International conference on intelligent human computer interaction, pp 220–229
Stone EE, Skubic M (2014) Fall detection in homes of older adults using the Microsoft Kinect. IEEE J Biomed Health Inf 19(1):290–301
Su X, Tong H, Ji P (2014) Activity recognition with smartphone sensors. Tsinghua Sci Technol 19(3):235–249
Su S, Wu SS, Chen SY, Duh DJ, Li S (2016) Multi-view fall detection based on spatio-temporal interest points. Multimed Tools Appl 75(14):8469–8492
Sun SW, Kuo CH, Chang PC (2016) People tracking in an environment with multiple depth cameras: a skeleton-based pairwise trajectory matching scheme. J vis Commun Image Represent 35:36–54
Theodoridis T, Solachidis V, Vretos N, Daras P (2017) Human fall detection from acceleration measurements using a recurrent neural network. In: International conference on biomedical and health informatics, pp 145–149
Vishwakarma DK, Kapoor R (2015) Hybrid classifier based human activity recognition using the silhouette and cells. Expert Syst Appl 42(20):6957–6965
Wang J, Lu Y, Gu L, Zhou C, Chai X (2014) Moving object recognition under simulated prosthetic vision using background-subtraction-based image processing strategies. Inf Sci 277:512–524
Wang S, Chen L, Zhou Z, Sun X, Dong J (2016a) Human fall detection in surveillance video based on PCANet. Multimed Tools Appl 75(19):11603–11613
Wang K, Cao G, Meng D, Chen W, Cao W (2016b) Automatic fall detection of human in video using combination of features. In: 2016 IEEE international conference on bioinformatics and biomedicine (BIBM). IEEE, pp 1228–1233
Wang Z, Ren J, Zhang D, Sun M, Jiang J (2018) A deep-learning based feature hybrid framework for spatiotemporal saliency detection inside videos. Neurocomputing 287:68–83
Wissel T, Stüber P, Wagner B, Bruder R, Erdmann C, Deutz CS, Ernst F (2016) Enhanced optical head tracking for cranial radiation therapy: supporting surface registration by cutaneous structures. Int J Radiat Oncol Biol Phys 95(2):810–817
Yang L, Ren Y, Zhang W (2016) 3D depth image analysis for indoor fall detection of elderly people. Digital Commun Netw 2(1):24–34
Yoo S, Oh D (2018) An artificial neural network–based fall detection. Int J Eng Busin Manag 20(10):1847979018787905
Yu M, Yu Y, Rhuma A, Naqvi SMR, Wang L, Chambers JA (2013) An online one class support vector machine-based person-specific fall detection system for monitoring an elderly individual in a room environment. IEEE J Biomed Health Inf 17(6):1002–1014
Yu S, Chen H, Brown RA (2017a) Hidden Markov model-based fall detection with motion sensor orientation calibration: a case for real-life home monitoring. IEEE J Biomed Health Inform 22(6):1847–1853
Yu S, Chen H, Wang Q, Shen L, Huang Y (2017b) Invariant feature extraction for gait recognition using only one uniform model. Neurocomputing 239:81–93
Yun Y, Gu IYH (2016) Human fall detection in videos by fusing statistical features of shape and motion dynamics on riemannian manifolds. Neurocomputing 207:726–734
Yun Y, Gu IYH (2017) Visual information-based activity recognition and fall detection for assisted living and eHealthCare. In: Ambient assisted living and enhanced living environments. Butterworth-Heinemann, pp 395–425
Zeng Z, Jia J, Zhu Z, Yu D (2016) Adaptive maintenance scheme for codebook-based dynamic background subtraction. Comput vis Image Understanding 152:58–66
Zerrouki N, Houacine A (2018) Combined curvelets and hidden Markov models for human fall detection. Multimed Tools Appl 77(5):6405–6424
Zerrouki N, Harrou F, Sun Y, Houacine A (2018) Vision-based human action classification using adaptive boosting algorithm. IEEE Sens J 18(12):5115–5121
Zhang Z, Conly C, Athitsos V (2015) A survey on vision-based fall detection. In: Proceedings of the 8th ACM international conference on PErvasive technologies related to assistive environments, pp 1–7
Acknowledgements
Not Applicable.
Funding
There is no funding for this study.
Author information
Authors and Affiliations
Contributions
All the authors have participated in writing the manuscript and have revised the final version. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
Authors declares that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants and/or animals performed by any of the authors.
Consent to participate
There is no informed consent for this study.
Consent for publication
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Rastogi, S., Singh, J. Human fall detection and activity monitoring: a comparative analysis of vision-based methods for classification and detection techniques. Soft Comput 26, 3679–3701 (2022). https://doi.org/10.1007/s00500-021-06717-x
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-021-06717-x