Abstract
A framework has been proposed for detecting fall actions from videos to solve the problem of imbalance between fall action data and Activity of Daily Life (ADL) data. In the framework, a 3D-convolutional variational auto-encoder (VAE) was used to reconstruct ADL videos, and reconstruction errors were used to recognize fall actions. In this paper, we propose an improved method using unsupervised clustering learning to cluster fall actions. The 3D-convolutional VAE extracts representations from videos, and additionally proposed fully-connected VAE to gather those representations into two clusters, where representations of fall actions are distinguished from distribution of ADL data. The experimental results showed that our method achieved a promising level of accuracy and better generalization ability compared to methods using supervised learning with well-labeled data. We further show visualization results of latent variables during unsupervised clustering, which showed the representations were clustered into two distinct clusters.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Albawendi, S., Appiah, K., Powell, H., Lotfi, A.: Video based fall detection with enhanced motion history images. In: Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments, pp. 1–7 (2016)
Baldewijns, G., Debard, G., Mertes, G., Vanrumste, B., Croonenborghs, T.: Bridging the gap between real-life data and simulated data by providing a highly realistic fall dataset for evaluating camera-based fall detection algorithms. Healthc. Technol. Lett. 3(1), 6–11 (2016)
Bellman, R.E.: Adaptive Control Processes: A Guided Tour. Princeton University Press, Princeton (2015)
Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., Raffel, C.A.: MixMatch: a holistic approach to semi-supervised learning. In: Advances in Neural Information Processing Systems, vol. 32. Curran Associates, Inc. (2019)
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2006)
Charfi, I., Miteran, J., Dubois, J., Atri, M., Tourki, R.: Optimized spatio-temporal descriptors for real-time fall detection: comparison of support vector machine and adaboost-based classification. J. Electron. Imaging 22(4), 041106 (2013)
Chong, Y.S., Tay, Y.H.: Abnormal event detection in videos using spatiotemporal autoencoder. In: Cong, F., Leung, A., Wei, Q. (eds.) ISNN 2017. LNCS, vol. 10262, pp. 189–196. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59081-3_23
Dilokthanakul, N., et al.: Deep unsupervised clustering with gaussian mixture variational autoencoders. CoRR (2016)
Ester, M., Kriegel, H.P., Sander, J., Xu, X., et al.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: KDD, vol. 96, pp. 226–231 (1996)
Fan, Y., Levine, M.D., Wen, G., Qiu, S.: A deep neural network for real-time detection of falling humans in naturally occurring scenes. Neurocomputing 260, 43–58 (2017)
Grandvalet, Y., Bengio, Y.: Semi-supervised learning by entropy minimization. In: Saul, L., Weiss, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems, vol. 17. MIT Press (2005)
Hara, K., Kataoka, H., Satoh, Y.: Learning spatio-temporal features with 3D residual networks for action recognition. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 3154–3160 (2017)
Jiang, Z., Zheng, Y., Tan, H., Tang, B., Zhou, H.: Variational deep embedding: An unsupervised and generative approach to clustering. arXiv preprint arXiv:1611.05148 (2016)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
Kiran, B.R., Thomas, D.M., Parakkal, R.: An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos. J. Imaging 4(2), 36 (2018)
Lin, C.Y., Wang, S.M., Hong, J.W., Kang, L.W., Huang, C.L.: Vision-based fall detection through shape features. In: Proceedings of the IEEE Second International Conference on Multimedia Big Data (BigMM), pp. 237–240. IEEE (2016)
Luxburg, U.V.: A tutorial on spectral clustering (2007)
Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(11) (2008)
MacQueen, J., et al.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Oakland, CA, USA, vol. 1, pp. 281–297 (1967)
Min, E., Guo, X., Liu, Q., Zhang, G., Cui, J., Long, J.: A survey of clustering with deep learning: From the perspective of network architecture. IEEE Access 6, 39501–39514 (2018)
Nguyen, V.A., Le, T.H., Nguyen, T.T.: Single camera based fall detection using motion and human shape features. In: Proceedings of the Seventh Symposium on Information and Communication Technology, pp. 339–344 (2016)
Nogas, J., Khan, S.S., Mihailidis, A.: Deepfall: non-invasive fall detection with deep spatio-temporal convolutional autoencoders. J. Healthc. Inform. Res. 4(1), 50–70 (2020)
Nunez-Marcos, A., Azkune, G., Arganda-Carreras, I.: Vision-based fall detection with convolutional neural networks. Wirel. Commun. Mob. Comput. 2017 (2017)
Xie, J., Girshick, R., Farhadi, A.: Unsupervised deep embedding for clustering analysis. In: International Conference on Machine Learning, pp. 478–487. PMLR (2016)
Xu, T., Zhou, Y.: Elders’ fall detection based on biomechanical features using depth camera. Int. J. Wavelets Multiresolut. Inf. Process. 16(02), 1840005 (2018)
Zhao, Y., Deng, B., Shen, C., Liu, Y., Lu, H., Hua, X.S.: Spatio-temporal autoencoder for video anomaly detection. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 1933–1941 (2017)
Zhou, J., Komuro, T.: Recognizing fall actions from videos using reconstruction error of variational autoencoder. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 3372–3376 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhou, J., Komuro, T. (2022). Detecting Fall Actions of Videos by Using Weakly-Supervised Learning and Unsupervised Clustering Learning. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2022. Lecture Notes in Computer Science, vol 13598. Springer, Cham. https://doi.org/10.1007/978-3-031-20713-6_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-20713-6_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20712-9
Online ISBN: 978-3-031-20713-6
eBook Packages: Computer ScienceComputer Science (R0)