Abstract
Considering humans as a non-deterministic factor makes anomaly detection in Human-Robot Interaction scenarios rather a challenging problem. Anomalous events like unexpected user interaction or unforeseen environment changes are unknown before they happen. On the other hand, the work process or user intentions could evolve in time. To address this issue, a modular deep learning approach is presented that is able to learn normal behavior patterns in an unsupervised manner. We combined the unsupervised feature extraction learning ability of an autoencoder with a sequence modeling neural network. Both models were firstly evaluated on benchmark video datasets, revealing adequate performance comparable to the state-of-the-art methods. For HRI application, a continuous training approach for real-time anomaly detection was developed and evaluated in an HRI-experiment with a collaborative robot, ToF camera, and proximity sensors. In the user study with 10 subjects irregular interactions and misplaced objects were the most common anomalies, which system was able to detect reliably.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Oldenbourg, R.: Gesundheits-Ingenieur. 13, 39, 40, 163 (1890)
Hasan, M., et al.: Learning temporal regularity in video sequences. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)
Del Giorno, A., Bagnell, J.A., Hebert, M.: A discriminative framework for anomaly detection in large videos. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 334–349. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46454-1_21
Medel, J.R.: Anomaly detection using predictive convolutional long short-term memory units (2016)
Chong, Y.S., Tay, Y.H.: Abnormal event detection in videos using spatiotemporal autoencoder. In: Cong, F., Leung, A., Wei, Q. (eds.) ISNN 2017. LNCS, vol. 10262, pp. 189–196. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59081-3_23
Sabokrou, M., et al.: Real-time anomaly detection and localization in crowded scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2015)
Wang, J.H., Miao, Y.Q., Khamis, A., Karray, F., Liang, J.: Adaptation approaches in unsupervised learning: a survey of the state-of-the-art and future directions. In: Campilho, A., Karray, F. (eds.) ICIAR 2016. LNCS, vol. 9730, pp. 3–11. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41501-7_1
Park, C., Huang, J.Z., Ding, Y.: A computable plug-in estimator of minimum volume sets for novelty detection. Oper. Res. 58(5), 1469–1480 (2009)
Breaban, M., Luchian, H.: Outlier detection with nonlinear projection pursuit. Int. J. Comput. Commun. Control 8(1), 30–36 (2013)
Zhang, K., Hutter, M., Jin, H.: A new local distance-based outlier detection approach for scattered real-word data. Adv. Knowl. Discov. Data Min. 5476, 813–822 (2009)
Cabral, G.G., Oliveira, A.L.I., Cahu, C.B.G.: Combining nearest neighbor data description and structural risk minimization for one-class classification. Neural Comput. Appl. 18(2), 175–183 (2009)
Miao, Y.Q., Farahat, A.K., Kamel, M.S.: Locally adaptive density ratio for detecting novelty in twitter streams. In: The 6th International Workshop on Modeling Social Media (MSM) - Behavioral Analytics in Social Media, Big Data and the Web, pp. 799–804 (2015)
Song, L., Teo, C.H., Smola, A.J.: Relative novelty detection. In: International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 536–543 (2009)
Sodemann, A.A., Ross, M.P., Borghetti, B.J.: A review of anomaly detection in automated surveillance. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 42(6), 1257–1272 (2012)
Chen, Z., Li, W.: Multisensor feature fusion for bearing fault diagnosis using sparse autoencoder and deep belief network. IEEE Trans. Instrum. Meas. 66(7), 1693–1702 (2017)
Khamis, A., et al.: The effects of outliers data on neural network performance. JApSc 5(8), 1394–1398 (2005)
Klein, B.D., Rossin, D.F.: Data quality in neural network models: effect of error rate and magnitude of error on predictive accuracy. Omega 27(5), 569–582 (1999)
Bai, S., Kolter, J.Z., Koltun, V.: An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018)
Lu, C., Shi, J., Jia, J.: Abnormal event detection at 150 fps in matlab. In: Proceedings of the IEEE International Conference on Computer Vision (2013)
Mahadevan, V., et al.: Anomaly detection in crowded scenes. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE (2010)
Alagi, H., Navarro, S.E., Mende, M., Hein, B.: A versatile and modular capacitive tactile proximity sensor. In:2016 IEEE Haptics Symposium (HAPTICS), Philadelphia, PA, pp. 290–296 (2016). https://doi.org/10.1109/HAPTICS.2016.7463192
Wang, T., Snoussi, H.: Histograms of optical flow orientation for abnormal events detection. In:2013 IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS). IEEE (2013)
Finch, T.: Incremental calculation of weighted mean and variance. Univ. Cambridge 4(11–5), 41–42 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Sóti, G., Mamaev, I., Hein, B. (2020). A Modular Deep Learning Architecture for Anomaly Detection in HRI. In: Ronzhin, A., Rigoll, G., Meshcheryakov, R. (eds) Interactive Collaborative Robotics. ICR 2020. Lecture Notes in Computer Science(), vol 12336. Springer, Cham. https://doi.org/10.1007/978-3-030-60337-3_29
Download citation
DOI: https://doi.org/10.1007/978-3-030-60337-3_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60336-6
Online ISBN: 978-3-030-60337-3
eBook Packages: Computer ScienceComputer Science (R0)