Skip to main content
SpringerLink
Log in

A novel real-time fall detection method based on head segmentation and convolutional neural network

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

As the computer vision develops, real-time fall detection based on computer vision has become increasingly popular in recent years. In this paper, a novel real-time indoor fall detection method based on computer vision by using geometric features and convolutional neural network (CNN) is proposed. Gaussian mixture model (GMM) is applied to detect the human target and find out the minimum external elliptical contour. Differently from the traditional fall detection method based on geometric features, we consider the importance of the head in fall detection and propose to use two different ellipses to represent the head and the torso, respectively. Three features including the long and short axis ratio, the orientation angle and the vertical velocity are extracted from the two different ellipses in each frame, respectively, and fused into a motion feature based on time series. In addition, a shallow CNN is applied to find out the correlation between the two elliptic contour features for detecting indoor falls and distinguishing some similar activities. Our novel method can effectively distinguish some similar activities in real time, which cannot be distinguished by some traditional methods based on geometric features, and has a better detection rate.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Chen, M.C.: A video surveillance system designed to detect multiple falls. Adv. Mech. Eng. 8(4), 1–11 (2016)

    Google Scholar 

  2. Chen, Y.T., Lin, Y.C., Fang, W.H.: A hybrid human fall detection scheme. In: 2010 IEEE International Conference on Image Processing (ICIP), pp. 3485–3488. IEEE (2010)

  3. Chua, J.L., Chang, Y., Lim, W.K.: A simple vision-based fall detection technique for indoor video surveillance. Signal Image Video Process. 9(3), 623–633 (2015)

    Article  Google Scholar 

  4. Comaniciu, D., Ramesh, V., Meer, P.: Real-time tracking of non-rigid objects using mean shift. In: 2000 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2142. IEEE (2002)

  5. Fan, K., Wang, P., Hu, Y., Dou, B.: Fall detection via human posture representation and support vector machine. Int. J. Distrib. Sens. Netw. 13(5), 1–21 (2017)

    Article  Google Scholar 

  6. Fan, Y., Levine, M.D., Wen, G., Qiu, S.: A deep neural network for real-time detection of falling humans in naturally occurring scenes. Neurocomputing 260(18), 43–58 (2017)

    Article  Google Scholar 

  7. Feng, W., Liu, R., Zhu, M.: Fall detection for elderly person care in a vision-based home surveillance environment using a monocular camera. Signal Image Video Process. 8(6), 1129–1138 (2014)

    Article  Google Scholar 

  8. Fischer, T., Krauss, C.: Deep learning with long short-term memory networks for financial market predictions. Eur. J. Oper. Res. 270(2), 654–669 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  9. Gong, W., Qi, L., Xu, Y.: Privacy-aware multidimensional mobile service quality prediction and recommendation in distributed fog environment. Wirel. Commun. Mob. Comput. 2018, 1–8 (2018)

    Google Scholar 

  10. Kendall, A., Gal, Y.: What uncertainties do we need in Bayesian deep learning for computer vision? In: Advances in Neural Information Processing Systems (NIPS), pp. 5574–5584 (2017)

  11. Liu, H., Zuo, C.: An improved algorithm of automatic fall detection. AASRI Procedia 1, 353–358 (2012)

    Article  Google Scholar 

  12. Min, W., Wei, L., Han, Q., Ke, Y.: Human fall detection based on motion tracking and shape aspect ratio. Int. J. Multimed. Ubiquitous Eng. 10(11), 1–14 (2016)

    Google Scholar 

  13. Min, W., Cui, H., Rao, H., Li, Z., Yao, L.: Detection of human falls on furniture using scene analysis based on deep learning and activity characteristics. IEEE Access 6, 9324–9335 (2018)

    Article  Google Scholar 

  14. Nez-Marcos, A., Azkune, G., Arganda-Carreras, I.: Vision-based fall detection with convolutional neural networks. Wirel. Commun. Mob. Comput. 2017, 1–16 (2018)

    Google Scholar 

  15. Nguyen, T.T., Cho, M.C., Lee, T.S.: Automatic fall detection using wearable biomedical signal measurement terminal. In: 2009 IEEE International Conference of Engineering in Medicine and Biology Society, pp. 5203–5206. IEEE (2009)

  16. Ozcan, A., Donat, H., Gelecek, N., Ozdirenc, M., Karadibak, D.: The relationship between risk factors for falling and the quality of life in older adults. BMC Public Health 5, 1–6 (2005)

    Article  Google Scholar 

  17. Pratt Jr., W.K., Adams, J.E.: Digital image processing. J. Electron. Imaging 16(2), 633–640 (2007)

    Article  Google Scholar 

  18. Qi, L., Chen, Y., Yuan, Y., Fu, S., Zhang, X., Xu, X.: A qos-aware virtual machine scheduling method for energy conservation in cloud-based cyber-physical systems. World Wide Web 23, 1275–1297 (2020)

    Article  Google Scholar 

  19. Qi, L., Dai, P., Yu, J., Zhou, Z., Xu, Y.: Time–location–frequency-aware Internet of things service selection based on historical records. Int. J. Distrib. Sens. Netw. 13(1), 1550147716688696 (2017)

    Article  Google Scholar 

  20. Qi, L., Dou, W., Chen, J.: Weighted principal component analysis-based service selection method for multimedia services in cloud. Computing 98(1–2), 195–214 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  21. Qi, L., Wang, R., Hu, C., Li, S., He, Q., Xu, X.: Time-aware distributed service recommendation with privacy-preservation. Inf. Sci. 480, 354–364 (2019)

    Article  Google Scholar 

  22. Qi, L., Xu, X., Dou, W., Yu, J., Zhou, Z.: Time-aware IoE service recommendation on sparse data. Mob. Inf. Syst. 2016, 1–12 (2016)

    Google Scholar 

  23. Qi, L., Yu, J., Zhou, Z.: An invocation cost optimization method for web services in cloud environment. Sci. Program. 2017, 1–9 (2017)

    Google Scholar 

  24. Qi, L., Zhang, X., Dou, W., Hu, C., Yang, C., Chen, J.: A two-stage locality-sensitive hashing based approach for privacy-preserving mobile service recommendation in cross-platform edge environment. Futur. Gener. Comp. Syst. 88, 636–643 (2018)

    Article  Google Scholar 

  25. Qi, L., Zhang, X., Dou, W., Ni, Q.: A distributed locality-sensitive hashing-based approach for cloud service recommendation from multi-source data. IEEE J. Sel. Areas Commun. 35(11), 2616–2624 (2017)

    Article  Google Scholar 

  26. Qi, L., Zhou, Z., Yu, J., Liu, Q.: Data-sparsity tolerant web service recommendation approach based on improved collaborative filtering. IEICE Trans. Inf. Syst. 100(9), 2092–2099 (2017)

    Article  Google Scholar 

  27. Rougier, C., Meunier, J., St-Arnaud, A., Rousseau, J.: Fall detection from human shape and motion history using video surveillance. In: 2007 IEEE International Conference on Advanced Information NETWORKING and Applications Work-shops, vol 2, pp. 875–880. IEEE (2007)

  28. Rougier, C., Meunier, J., St-Arnaud, A., Rousseau, J.: Robust video surveillance for fall detection based on human shape deformation. IEEE Trans. Circuits Syst. Video Technol. 21(5), 611–622 (2011)

    Article  Google Scholar 

  29. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J.: Rethinking the inception architecture for computer vision. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2016, pp. 2818–2826. IEEE (2016)

  30. Wang, Q., Chen, M., Nie, F., Li, X.: Detecting coherent groups in crowd scenes by multiview clustering. IEEE Trans. Pattern Anal. Mach. Intell. 42(1), 46–58 (2020)

    Article  Google Scholar 

  31. Wang, Q., Gao, J., Li, X.: Weakly supervised adversarial domain adaptation for semantic segmentation in urban scenes. IEEE Trans. Image Process. 28(9), 4376–4386 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  32. Williams, A., Ganesan, D., Hanson, A.: Aging in place: fall detection and localization in a distributed smart camera network. In: International Conference on Multimedia, pp. 892–901. ACM (2007)

  33. Xiao, H., Wang, X., Li, Q., Wang, Z.: Gaussian mixture model for background based automatic fall detection. In: International Conference onCyberspace Technology, pp. 234-237. IET (2013)

  34. Xu, T., Zhou, Y.: Elders’ fall detection based on biomechanical features using depth camera. Int. J. Wavelets Multiresolut. Inf. Process. 16(2), 1840005 (2018)

    Article  MATH  Google Scholar 

  35. Yang, Z., Leng, L., Kim, B.-G.: StoolNet for color classification of stool medical images. Electronics 8(12), 1464 (2019)

    Article  Google Scholar 

  36. Yang, Z., Li, J., Min, W., Wang, Q.: Real-time pre-identification and cascaded detection for tiny faces. Appl. Sci. 9(20), 4344 (2019)

    Article  Google Scholar 

  37. Yann, L.C., Yoshua, B., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2016)

    Google Scholar 

  38. Yao, L., Min, W., Lu, K.: A new approach to fall detection based on the human torso motion model. Appl. Sci. 7(10), 993 (2017)

    Article  Google Scholar 

  39. Yu, M., Yu, Y., Rhuma, A., Naqvi, S.M.R., Wang, L., Chambers, J.A.: An online one class support vector machine-based person-specific fall detection system for monitoring an elderly individual in a room environment. IEEE J. Biomed. Health Inform. 17(6), 1002–1014 (2013)

    Article  Google Scholar 

  40. Zerrouki, N., Houacine, A.: Combined curvelets and hidden Markov models for human fall detection. Multimed. Tools Appl. 77(5), 6405–6424 (2018)

    Article  Google Scholar 

  41. Zheng, Y., Xu, X., Qi, L.: Deep CNN-Assisted personalized recommendation over big data for mobile wireless networks. Wirel. Commun. Mob. Comput. 2019, 1–6 (2019)

    Google Scholar 

  42. Zhou, Q., Wang, Y., Liu, J., Jin, X., Latecki, L.J.: An open-source project for real-time image semantic segmentation. Sci. China Inf. Sci. 62(2), 227101–227102 (2019)

    Article  Google Scholar 

  43. Zhou, Q., Yang, W., Gao, G., Ou, W., Lu, H., Chen, J., Latecki, L.J.: Multi-scale deep context convolutional neural networks for semantic segmentation. World Wide Web 22(4), 1–16 (2019)

    Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China under Grant No. 61762061, the Natural Science Foundation of Jiangxi Province, China, under Grant No. 20161ACB20004 and Jiangxi Key Laboratory of Smart City under Grant No. 20192BCD40002. The authors Chenguang Yao and Jun Hu contributed equally to this paper and shall be considered as co-first authors.

Author information

Authors and Affiliations

  1. School of Software, Nanchang University, Nanchang, 330047, China

    Chenguang Yao, Jun Hu & Weidong Min

  2. Jiangxi Key Laboratory of Smart City, Nanchang, 330047, China

    Weidong Min

  3. School of Information Engineering, Nanchang University, Nanchang, 330031, China

    Zhifeng Deng & Song Zou

  4. School of Tourism, Jiangxi Science and Technology Normal University, Nanchang, 330038, China

    Weiqiong Min

Authors
  1. Chenguang Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weidong Min
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhifeng Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Song Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Weiqiong Min
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weidong Min.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Informed consent

All participants were informed and agreed that the study was incompliance with relevant ethical requirements.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yao, C., Hu, J., Min, W. et al. A novel real-time fall detection method based on head segmentation and convolutional neural network. J Real-Time Image Proc 17, 1939–1949 (2020). https://doi.org/10.1007/s11554-020-00982-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-020-00982-z

Keywords

Search

Navigation