Skip to main content
SpringerLink
Log in

Vision based human fall detection with Siamese convolutional neural networks

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Fall detection is drawing serious attention all across the globe, as unattended fall of senior citizens creates long lasting injuries. This necessitates the deployment of automatic fall detection systems to facilitate smart care health environments for the elderly people living in various settings, viz., living independently in their homes, hospitalized or living in care homes. The proposed work employs Siamese network with one shot classification for human fall detection. Unlike the neural network that classifies the video sequences, this network learns to differentiate the video sequences by computing the similarity score. The network contains two identical CNNs, receiving pair of video sequences as the input. The features of these networks are merged at the final layer through the similarity function. Two different architectures viz., one with 2D convolutional filters and the other with depth wise convolutional filters, each operated on two set of features, RGB and optical flow features are developed. Experimental results demonstrate the effectiveness and feasibility of the proposed work compared to state-of-the methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Abobakr A, Hossny M, Nahavandi S (2018) A skeleton-free fall detection system from depth images using random decision forest. IEEE Syst J 12(3):2994–3005

    Article  Google Scholar 

  • Adhikari K, Bouchachia H, Nait-Charif H (2017) Activity recognition for indoor fall detection using convolutional neural network, In: 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA), Nagoya, pp 81–84

  • Berlin SJ, Mala J (2020) Light weight convolutional models with spiking neural network based human action recognition. J Intell Fuzzy Syst 39:961–973

    Article  Google Scholar 

  • Bhavya KR, Park J, Park H, Kim H, Paik J (2016) Fall detection using motion estimation and accumulated image map. In: 2016 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia), Seoul, pp 1–2

  • Boudouane I, Makhlouf A, Harkat MA et al (2020) Fall detection system with portable camera. J Ambient Intell Humaniz Comput 11:2647–2659

    Article  Google Scholar 

  • Charfi I, Miteran J, Dubois J, Atri M, Tourki R (2012) Definition and performance evaluation of a robust SVM based fall detection solution. In: 8th international conference on signal image technology and internet based systems, pp 218–224

  • Chen L, Kong X, Tomiyama H, Meng L (2019) Multiple states fall detection system for senior citizens. In: Proceedings of the international conference on advanced mechatronic systems (ICAMechS), Japan, pp 169–174

  • Chopra S, Hadsell R, LeCun Y (2005) Learning a similarity metric discriminatively, with application to face verification. In: IEEE computer society conference on computer vision and pattern recognition (CVPR’05), USA, pp 539–546

  • Deandrea S, Lucenteforte E, Bravi F, Foschi R, La Vecchia C, Negri E (2010) Risk factors for falls in community-dwelling older people: a systematic review and meta-analysis. Epidemiology 21(5):658–668

    Article  Google Scholar 

  • Fan K, Wang P, Zhuang S (2019) Human fall detection using slow feature analysis. Multimed Tools Appl 78(7):9101–9128

    Article  Google Scholar 

  • Gracewell JJ, Pavalarajan S (2019) Fall detection based on posture classification for smart home environment. J Ambient Intell Hum Comput

  • Harrou F, Zerrouki N, Sun Y, Houacine A (2017) Vision-based fall detection system for improving safety of elderly people. IEEE Instrum Meas Mag 20(6):49–55

    Article  Google Scholar 

  • Harrou F, Zerrouki N, Sun Y, Houacine A (2019) An integrated vision-based approach for efficient human fall detection in a home environment. IEEE Access 7:114966–114974

    Article  Google Scholar 

  • Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861

  • Iazzi A, Rziza M, Thami ROH (2020) Efficient fall activity recognition by combining shape and motion features. Comput Vis Media 6(3):247–263

    Article  Google Scholar 

  • Jansi R, Amutha R (2020) Detection of fall for the elderly in an indoor environment using a tri-axial accelerometer and Kinect depth data. Multidimens Syst Signal Process 31(4):1207–1225

    Article  Google Scholar 

  • Joshi NB, Nalbalwar SL (2017) A fall detection and alert system for an elderly using computer vision and Internet of Things. In: 2017 2nd IEEE international conference on recent trends in electronics, information and communication technology (RTEICT), Bangalore, pp 1276–1281

  • Juang C, Chang C (2007) Human body posture classification by a neural fuzzy network and home care system application. IEEE Trans Syst Man Cybern Part A Syst Hum 37(6):984–994

    Article  Google Scholar 

  • Kerdjidj O, Ramzan N, Ghanem K et al (2020) Fall detection and human activity classification using wearable sensors and compressed sensing. J Ambient Intell Human Comput 11:349–361. https://doi.org/10.1007/s12652-019-01214-4

    Article  Google Scholar 

  • Khan SS, Hoey J (2017) Review of fall detection techniques: a data availability perspective. Med Eng Phys 39:12–22

    Article  Google Scholar 

  • Khraief C, Benzarti F, Amiri H (2020) Elderly fall detection based on multi-stream deep convolutional networks. Multimed Tools Appl 1–24

  • Kwolek B, Kepski M (2014) Human fall detection on embedded platform using depth maps and wireless accelerometer. Comput Methods Programs Biomed 117(3):489–501

    Article  Google Scholar 

  • Leal-Taixe L, Canton-Ferrer C, Schindler K (2016) Learning by tracking: Siamese CNN for robust target association. In: IEEE conference on computer vision and pattern recognition workshops, pp 33–40

  • Lotfi A, Albawendi S, Powell H, Appiah K, Langensiepen C (2018) Supporting independent living for older adults; employing a visual based fall detection through analysing the motion and shape of the human body. IEEE Access 6:70272–70282

    Article  Google Scholar 

  • Liu C, Lee C, Lin P (2010) A fall detection system using k-nearest neighbor classifier. Expert Syst Appl 37:7174–7181

    Article  Google Scholar 

  • Ma X, Wang H, Xue B, Zhou M, Ji B, Li Y (2014) Depth-based human fall detection via shape features and improved extreme learning machine. IEEE J Biomed Health Inf 18(6):1915–1922

    Article  Google Scholar 

  • Mastorakis G, Makris D (2014) Fall detection system using Kinect’s infrared sensor. J Real-Time Image Proc 9:635–646

    Article  Google Scholar 

  • Min W, Cui H, Rao H, Li Z, Yao L (2018) Detection of human falls on furniture using scene analysis based on deep learning and activity characteristics. IEEE Access 6:9324–9335

    Article  Google Scholar 

  • Mirmahboub B, Samavi S, Karimi N, Shirani S (2013) Automatic monocular system for human fall detection based on variations in Silhouette Area. IEEE Trans Biomed Eng 60(2):427–436

    Article  Google Scholar 

  • Nunez-Marcos A, Azkune G, Arganda-Carreras I (2017) Vision-based fall detection with convolutional neural networks. Wirel Commun Mobile Comput

  • Ozcan K, Velipasalar S, Varshney PK (2017) Autonomous fall detection with wearable cameras by using relative entropy distance measure. IEEE Trans Hum Mach Syst 47(1):31–39

    Google Scholar 

  • Rougier C, Meunier J, St-Arnaud A, Rousseau J (2011) Robust video surveillance for fall detection based on human shape deformation. IEEE Trans Circuits Syst Video Technol 21(5):611–622

    Article  Google Scholar 

  • Rougier C, Meunier J, St-Arnaud A, Rousseau J (2013) 3D head tracking for fall detection using a single calibrated camera. Image Vis Comput 31246–31254

  • Rubenstein LZ (2006) Falls in older people: epidemiology, risk factors and strategies for prevention. Age Ageing (35)

  • Ruiz V, Linares I, Sanchez A, Velez JF (2020) Off-line handwritten signature verification using compositional synthetic generation of signatures and Siamese Neural Networks. Neurocomputing 374:30–41

  • Sehairi K, Chouireb F, Meunier J (2018) Elderly fall detection system based on multiple shape features and motion analysis. In: International conference on intelligent systems and computer vision (ISCV), pp 1–8

  • Shieh WY, Huang JC (2012) Falling-incident detection and throughput enhancement in a multi-camera video-surveillance system. Med Eng Phys 34(7):954–963

    Article  Google Scholar 

  • Soni PK, Choudhary A (2018) Automated fall detection using computer vision. In: Tiwary U (eds) Intelligent human computer interaction. IHCI 2018. Lecture Notes in Computer Science, vol 11278. Springer, Cham

  • Sun D, Yang X, Liu MY, Kautz J (2018) PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. In: IEEE/CVF international conference on computer vision and pattern recognition, USA, pp 8934–8943

  • Taigman Y, Yang M, Ranzato MA, Wolf L (2014) Deepface: Closing the gap to human-level performance in face verification. In: IEEE conference on computer vision and pattern recognition, pp 1701–1708

  • Vallabh P, Malekian R (2018) Fall detection monitoring systems: a comprehensive review. J Ambient Intell Human Comput 9:1809–1833

    Article  Google Scholar 

  • Wang K, Cao G, Meng D, Chen W, Cao W (2016a) Automatic fall detection of human in video using combination of features, In: 2016a IEEE international conference on bioinformatics and biomedicine (BIBM), Shenzhen, pp 1228–1233

  • Wang S, Chen L, Zhou Z et al (2016b) Human fall detection in surveillance video based on PCANet. Multimed Tools Appl 75:11603–11613

    Article  Google Scholar 

  • Wang F, Yang B, Li J, Hu X, Ji Z (2020) Attention-based siamese region proposals network for visual tracking. IEEE Access 8:86595–86607

    Article  Google Scholar 

  • Yacchirema D, de Puga JS, Palau C, Esteve M (2019) Fall detection system for elderly people using IoT and ensemble machine learning algorithm. Pers Ubiquit Comput 23(5):801–817

    Article  Google Scholar 

  • Yao C, Hu J, Min W, Deng Z, Zou S, Min W (2020) A novel real-time fall detection method based on head segmentation and convolutional neural network. J Real-Time Image Proc 17:1939–1949

    Article  Google Scholar 

  • Yu M, Rhuma A, Naqvi SM, Wang L, Chambers J (2012) A posture recognition-based fall detection system for monitoring an elderly person in a smart home environment. IEEE Trans Inf Technol Biomed 16(6):1274–1286

    Article  Google Scholar 

  • Zerrouki N, Houacine (2018) A Combined curvelets and hidden Markov models for human fall detection. Multimed Tools Appl 77:6405–6424

    Article  Google Scholar 

  • Zhang Z, Ma X, Wu H, Li Y (2019a) Fall detection in videos with trajectory-weighted deep-convolutional rank-pooling descriptor. IEEE Access 7:4135–4144

    Article  Google Scholar 

  • Zhang Z, Zhang Y, Cheng X, Li K (2019b) Siamese network for real-time tracking with action-selection. J Real-Time Image Process 1–11

  • Zhang C, Wang H, Wen J, Peng L (2020) Deeper siamese network with stronger feature representation for visual tracking. IEEE Access 8:119094–119104

    Article  Google Scholar 

Download references

Acknowledgements

The first author is a recipient of DST INSPIRE Fellowship and wishes to thank DST, India for the same.

Author information

Authors and Affiliations

  1. Department of Electronics Engineering, Madras Institute of Technology, Anna University, Chennai, India

    S. Jeba Berlin & Mala John

Authors
  1. S. Jeba Berlin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mala John
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. Jeba Berlin.

Ethics declarations

Conflict of interest

The authors declare that there are no conflicts of interest in the authorship or publication of this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Berlin, S.J., John, M. Vision based human fall detection with Siamese convolutional neural networks. J Ambient Intell Human Comput 13, 5751–5762 (2022). https://doi.org/10.1007/s12652-021-03250-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-021-03250-5

Keywords

Search

Navigation