Abstract
A non-neglectable number of car accidents are caused by driver’s loss of ability to drive the car, which may be caused by serious health problems, e.g. heart attack, stroke, drug or alcohol influence, as well as by drowsiness and other problems. In this paper, a method is presented for detecting the anomaly situations during driving. The method is based on detecting the particular parts of driver’s body in the sequence of images obtained from an in-car camera. A feature vector containing the distances between the body parts and describing the situation in a chosen number of frames is computed and used for detection. For the detection itself, the neural network of the autoencoder type containing the LSTM units is used. The method is compared with some other methods; the results show that the method is useful. Moreover, the video sequences used for training and testing are presented, which may be regarded as an additional contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous systems (2015).http://tensorflow.org/, software available from tensorflow.org
Breunig, M.M., Kriegel, H.P., Ng, R.T., Sander, J.: Lof: identifying density-based local outliers. SIGMOD Rec. 29(2), 93–104 (2000). 10.1145/335191.335388
Cao, Z., Hidalgo, G., Simon, T., Wei, S.E., Sheikh, Y.: Openpose: Realtime multi-person 2d pose estimation using part affinity fields (2018). 10.48550/ARXIV.1812.08008, https://arxiv.org/abs/1812.08008
Chollet, F., et al.: Keras (2015).https://keras.io
Fusek, R., et al.: MRL driver dataset (September 2022). http://mrl.cs.vsb.cz/driverdataset
Goodfellow, I.J., et al.: Generative adversarial networks (2014). https://doi.org/10.48550/ARXIV.1406.2661, https://arxiv.org/abs/1406.2661
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015). https://doi.org/10.48550/ARXIV.1512.03385
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (12 1997). https://doi.org/10.1162/neco.1997.9.8.1735
Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., Keutzer, K.: SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and 0.5mb model size (2016). https://doi.org/10.48550/ARXIV.1602.07360, https://arxiv.org/abs/1602.07360
Choi, I.-H., Hong, S.K., Kim, Y.-G.: Real-time categorization of driver’s gaze zone using the deep learning techniques. In: 2016 International Conference on Big Data and Smart Computing (BigComp), pp. 143–148 (2016)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes (2013). https://doi.org/10.48550/ARXIV.1312.6114, https://arxiv.org/abs/1312.6114
Kiran, B.R., Thomas, D.M., Parakkal, R.: An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos (2018). https://doi.org/10.48550/ARXIV.1801.03149, https://arxiv.org/abs/1801.03149
Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. In: 25th Proceedings of the Conference on Neural Information Processing Systems, January 2012. https://doi.org/10.1145/3065386
Liu, F.T., Ting, K.M., Zhou, Z.H.: Isolation forest. In: 2008 Eighth IEEE International Conference on Data Mining.,pp. 413–422 (2008). https://doi.org/10.1109/ICDM.2008.17
Liu, L., et al.: A novel fatigue driving state recognition and warning method based on EEG and EOG signals. J. Healthc. Eng. 2021, 1–10 (2021). https://doi.org/10.1155/2021/7799793
Lugaresi, C., et al.: Mediapipe: a framework for building perception pipelines (2019). https://doi.org/10.48550/ARXIV.1906.08172, https://arxiv.org/abs/1906.08172
Martin, M., et al.: Drive & act: a multi-modal dataset for fine-grained driver behavior recognition in autonomous vehicles. In: The IEEE International Conference on Computer Vision (ICCV), October 2019
Naqvi, R.A., Arsalan, M., Batchuluun, G., Yoon, H.S., Park, K.R.: Deep learning-based gaze detection system for automobile drivers using a Nir camera sensor. Sensors 18(2) (2018). https://doi.org/10.3390/s18020456https://www.mdpi.com/1424-8220/18/2/456
Nguyen, H., Tran, K., Thomassey, S., Hamad, M.: Forecasting and anomaly detection approaches using lSTM and LSTM autoencoder techniques with the applications in supply chain management. Int. J. Inf. Manag. 57, 102282 (2021). https://doi.org/10.1016/j.ijinfomgt.2020.102282, https://www.sciencedirect.com/science/article/pii/S026840122031481X
Osokin, D.: Real-time 2D multi-person pose estimation on CPU: Lightweight openpose (2018). https://doi.org/10.48550/ARXIV.1811.12004, https://arxiv.org/abs/1811.12004
Rousseeuw, P.J., Driessen, K.V.: A fast algorithm for the minimum covariance determinant estimator. Technometrics 41(3), 212–223 (1999). https://doi.org/10.1080/00401706.1999.10485670
Ruff, L., et al.: Deep one-class classification. In: Dy, J., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 80, pp. 4393–4402. PMLR, Stockholmsmässan, Stockholm Sweden (10–15 Jul 2018), http://proceedings.mlr.press/v80/ruff18a.html
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015). http://arxiv.org/abs/1409.1556
Vora, S., Rangesh, A., Trivedi, M.M.: On generalizing driver gaze zone estimation using convolutional neural networks. In: 2017 IEEE Intelligent Vehicles Symposium (IV), pp. 849–854 (2017)
Williams, C., Seeger, M.: Using the nyström method to speed up kernel machines. In: Leen, T., Dietterich, T., Tresp, V. (eds.) Advances in Neural Information Processing Systems. vol. 13. MIT Press (2000)
Yan, X., He, J., Wu, G., Zhang, C., Wang, C.: A proactive recognition system for detecting commercial vehicle driver’s distracted behavior. Sensors 22(6) (2022). https://doi.org/10.3390/s22062373, https://www.mdpi.com/1424-8220/22/6/2373
Yang, T., Li, Y.f., Mahdavi, M., Jin, R., Zhou, Z.H.: Nyström method vs random fourier features: a theoretical and empirical comparison. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in Neural Information Processing Systems. vol. 25. Curran Associates, Inc. (2012)
Yoon, H.S., Baek, N.R., Truong, N.Q., Park, K.R.: Driver gaze detection based on deep residual networks using the combined single image of dual near-infrared cameras. IEEE Access 7, 93448–93461 (2019)
Zhao, R., Wang, K., Su, H., Ji, Q.: Bayesian graph convolution LSTM for skeleton based action recognition. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6881–6891, October 2019. https://doi.org/10.1109/ICCV.2019.00698
Zhu, W., et al.: Co-occurrence feature learning for skeleton based action recognition using regularized deep LSTM networks. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp. 3697–3703. AAAI’16, AAAI Press (2016)
Acknowledgements
This work is partially supported by Grants of SGS No. SP2022/81, VSB - Technical University of Ostrava, Czech Republic.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Fusek, R., Sojka, E., Gaura, J., Halman, J. (2022). Driver State Detection from In-Car Camera Images. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2022. Lecture Notes in Computer Science, vol 13599. Springer, Cham. https://doi.org/10.1007/978-3-031-20716-7_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-20716-7_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20715-0
Online ISBN: 978-3-031-20716-7
eBook Packages: Computer ScienceComputer Science (R0)