Skip to main content

Advertisement

SpringerLink
Log in

Fall detection on embedded platform using infrared array sensor for healthcare applications

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Previous vision-based research has predominantly used common visible light cameras as sensors for detecting falls in home environments. While some studies have explored the use of infrared cameras for this purpose, personal privacy protection and computational capability on an embedded platform remain crucial concerns. To address these challenges and achieve accurate human fall detection on an embedded platform, we propose a new lightweight human fall detection method based on a deep learning network. In the first stage, we designed an image acquisition device based on an infrared array sensor to collect an infrared human fall dataset (https://github.com/Flier-01/Deeplearning-based-Fall-Detection-Using-Infrared-Array-Dataset). This dataset consists of 10240 images, including 5216 pictures of falls, 4024 pictures of non-fall walking, and 1000 pictures of other poses. Furthermore, we have included an additional set of 10 videos specifically for testing purposes. These images were captured within living environments with varying ambient temperatures. To address challenges associated with infrared images, such as excessive noise and low definition, we adopted the RetinexNet algorithm to preprocess the collected images. This pre-processing step significantly improves the quality of the infrared images, enabling more accurate analysis and detection. Subsequently, we developed a modified YOLOv5 network that incorporates a comprehensive enhancement strategy by integrating the CBAM and TPH modules. These modules enhance the network’s ability to capture and extract features relevant to fall detection. Furthermore, to optimize the network’s performance, we employed the GhostNet architecture and deployed the resulting model on the Huawei Altas embedded platform. Through video testing, our fall detection system achieved a real-time detection frame rate of 38.61 FPS, surpassing the performance of the original YOLOv5-based fall detector, which attained a frame rate of 34.78 FPS. Notably, our proposed method demonstrated remarkable performance in terms of fall detection accuracy. The average accuracy of our fall detector reached an impressive 96.52%, outperforming the original YOLOv5 fall detector, which achieved an average accuracy of 88.46%. These experimental results affirm the superiority of our approach, exhibiting improved fall detection accuracy and real-time performance compared to the original YOLOv5 algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Algorithm 1
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Data Availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Kamel MH, Abdulmajeed AA, Sally EI (2013) Risk factors of falls among elderly living in Urban Suez-Egypt. Pan Afr Med J 14(1)

  2. Xing X-Y et al. (2020) Mortality and disease burden of injuries from 2008 to 2017 in Anhui Province, China. BioMed Res Int 2020

  3. Ojetola O, Gaura EI, Brusey J (2011) Fall detection with wearable sensors-safe (Smart fall detection). In: Seventh international conference on intelligent environments, pp 318–321. https://doi.org/10.1109/IE.2011.38

  4. Nyan MN, Tay FE, Murugasu E (2008) A wearable system for pre-impact fall detection. J Biomech 41(16):3475–3481

    Article  CAS  PubMed  Google Scholar 

  5. Tai KY et al (2020) Smart fall prediction for elderly care using iPhone and apple watch. Wirel Pers Commun 114(1):347–365

    Article  Google Scholar 

  6. Torti E et al. (2018) Embedded real-time fall detection with deep learning on wearable devices. In: 2018 21st Euromicro conference on digital system design (DSD), pp 405–412. https://doi.org/10.1109/DSD.2018.00075

  7. Qian Z et al (2022) Development of a real time wearable fall detection system in the context of internet of things. IEEE Internet Things J. https://doi.org/10.1109/JIOT.2022.3181701

    Article  Google Scholar 

  8. Kim JK, Kangbok L, Sang GH (2023) Detection of important features and comparison of datasets for fall detection based on wrist-wearable devices. Expert Syst Appl 234:121034

    Article  Google Scholar 

  9. de Quadros T, Lazzaretti AE, Schneider FK (2018) A movement decomposition and machine learning-based fall detection system using wrist wearable device. IEEE Sens J 18(12):5082–5089. https://doi.org/10.1109/JSEN.2018.2829815

    Article  ADS  Google Scholar 

  10. Mokhtari G, Zhang Q, Fazlollahi A (2017) Non-wearable UWB sensor to detect falls in smart home environment. In: IEEE international conference on pervasive computing and communications workshops (PerCom Workshops), pp 274–278. https://doi.org/10.1109/PERCOMW.2017.7917571

  11. Palipana S et al (2018) FallDeFi: ubiquitous fall detection using commodity Wi-Fi devices. Proc ACM Interact, Mob, Wearable Ubiquitous Technol 1(4):1–25

    Article  Google Scholar 

  12. Youngkong P, Panpanyatep W (2021) A novel double pressure sensors-based monitoring and alarming system for fall detection. In: 2021 second international symposium on instrumentation, control, artificial intelligence, and robotics (ICA-SYMP), pp.1–5. https://doi.org/10.1109/ICA-SYMP50206.2021.9358439

  13. Amsaprabhaa M (2023) Multimodal spatiotemporal skeletal kinematic gait feature fusion for vision-based fall detection. Expert Syst Appl 212:118681

    Article  Google Scholar 

  14. Chen Z, Yiye W, Wankou Y (2022) Video based fall detection using human poses. In: CCF conference on big data. Singapore, Springer Singapore

  15. Chua J-L, Chang YC, Lim WK (2013) Visual based fall detection through human shape variation and head detection. IMPACT-2013, pp 61–65. https://doi.org/10.1109/MSPCT.2013.6782088

  16. Saurav S, Saini R, Singh S (2022) A dual-stream fused neural network for fall detection in multi-camera and \(360^{\circ }\) videos. Neural Comput Appl 34(2):1455–1482

    Article  Google Scholar 

  17. Georgakopoulos SV et al (2020) Change detection and convolution neural networks for fall recognition. Neural Comput Appl 32(23):17245–17258

    Article  Google Scholar 

  18. Stone EE, Skubic M (2014) Fall detection in homes of older adults using the Microsoft Kinect. IEEE J Biomed Health Inform 19(1):290–301

    Article  PubMed  Google Scholar 

  19. Mashiyama S, Hong J, Ohtsuki T (2014) A fall detection system using low resolution infrared array sensor. In: 2014 IEEE 25th annual international symposium on personal, indoor, and mobile radio communication (PIMRC), pp 2109–2113. https://doi.org/10.1109/PIMRC.2014.7136520

  20. Hayashida A, Vasily M, Koji H (2017) New approach for indoor fall detection by infrared thermal array sensor. In: 2017 IEEE 60th international midwest symposium on circuits and systems (MWSCAS). IEEE

  21. Tzeng H-W, Mei-Yung C, Jai-Yu C (2010) Design of fall detection system with floor pressure and infrared image. In: 2010 international conference on system science and engineering. IEEE

  22. Liu Z et al (2020) Fall detection and personnel tracking system using infrared array sensors. IEEE Sens J 20(16):9558–9566

    Google Scholar 

  23. Spasov G, Tsvetkov V, Petrova G (2019) Using IR array MLX90640 to build an IoT solution for ALL and security smart systems. In: IEEE XXVIII international scientific conference electronics (ET), pp 1–4. https://doi.org/10.1109/ET.2019.8878637

  24. Tang H et al (2022) An improved algorithm for low-light image enhancement based on retinexnet. Appl Sci 12(14):7268

    Article  CAS  Google Scholar 

  25. Wei C , Wang W , Yang W, et al. (2018) Deep retinex decomposition for low-light enhancement. https://doi.org/10.48550/arXiv.1808.04560[P]

  26. Land EH (1977) The retinex theory of color vision. Sci Am 237(6):108–129

    Article  CAS  PubMed  Google Scholar 

  27. Han K et al. (2020) Ghostnet: more features from cheap operations. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

  28. Woo S, Jongchan P, Joon-Young L, Kweon IS (2018) Cbam: convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19

  29. Liu Y, Guolei S, Yu Q, Le Z, Ajad C, Luc VG (2021) Transformer in convolutional neural networks. arXiv preprint arXiv:2106.03180

  30. Thuan D (2021) Evolution of Yolo algorithm and Yolov5: the state-of-the-art object detention algorithm

  31. Zhu X, Shuchang L, Xu W, Qi Z (2021) TPH-YOLOv5: improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 2778–2788

  32. Zheng Z, Wang P, Liu W, Li J, Ye R, Ren D (2020) Distance-IoU loss: faster and better learning for bounding box regression. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, No(07), pp 12993–13000

  33. Ma B, Yongkang F, Wang C, Li J, Wang Yuli (2022) A high-performance insulators location scheme based on YOLOv4 deep learning network with GDIoU loss function. IET Image Proc 16(4):1124–1134

    Article  Google Scholar 

  34. Henderson P, Ferrari V(2017) End-to-end training of object class detectors for mean average precision. Asian conference on computer vision. Springer, Cham

Download references

Acknowledgements

This work was supported by the Science and Technology Projects of Sichuan under Grants Nos. 2021YFSY0059, 2021YFQ0055; and by the Science and Technology Projects of Chengdu under Grant No. 2022-YF05-00379-SN.

Author information

Author notes

  1. Tianyi Gong, Lingfeng He, Shicheng Yan and Xiaoping Wu have contributed equally to this work.

Authors and Affiliations

  1. Southwest Jiaotong University, Chengdu City, Sichuan Province, China

    Yan Jiang, Tianyi Gong, Lingfeng He, Shicheng Yan, Xiaoping Wu & Jianyang Liu

  2. School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu, 611756, China

    Yan Jiang & Lingfeng He

  3. School of Information Science and Technology, Southwest Jiaotong University, Chengdu, 611756, China

    Tianyi Gong

  4. School of Mechanical Engineering, Southwest Jiaotong University, Chengdu, 611756, China

    Shicheng Yan, Xiaoping Wu & Jianyang Liu

Authors
  1. Yan Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tianyi Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lingfeng He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shicheng Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaoping Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jianyang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by YJ, TG, LH, SY, and XW. The first draft of the manuscript was written by YJ, the draft of the manuscript was modified by JL, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Jianyang Liu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, Y., Gong, T., He, L. et al. Fall detection on embedded platform using infrared array sensor for healthcare applications. Neural Comput & Applic 36, 5093–5108 (2024). https://doi.org/10.1007/s00521-023-09334-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-09334-x

Keywords

Search

Navigation