Skip to main content
SpringerLink
Log in

Driver Anomaly Detection Using Skeleton Images

  • Conference paper
  • First Online:
Advances in Visual Computing (ISVC 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14361))

Included in the following conference series:

  • 440 Accesses

Abstract

Many unexpected situations can occur while driving that may lead to dangerous accidents. Some of them may be caused by sudden health problems (e.g. heart attack, stroke, total collapse) or by driver inattention (e.g. microsleep, visual distraction). This has motivated the need for developing the methods that are able to monitor the driver’s state in the first step and to prevent the accidents in the second step (e.g. by activating an acoustic signal, or even by taking over driving). In this paper, we propose a method that can be used for detecting the abnormal driving situations. Our approach is based on two main steps. In the first step, the MNIST-like skeleton images are created with the use of human pose detector. In the second step, an appropriate neural network is used for the final classification. Since we also include the anomalies consisting in an unusual trajectory of a certain body part (not only an unusual shape of body, which can be detected from the isolated images), short sequences of images are examined. The LSTM (long short-term memory) autoencoder is used as a main network architecture. The experiments that are presented show that the proposed method achieves better results than other compared methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adanu, E., Jones, S.: Effects of human-centered factors on crash injury severities. J. Adv. Transp. 1–11 2017 (2017). https://doi.org/10.1155/2017/1208170

  2. Arefnezhad, S., et al.: Driver drowsiness estimation using EEG signals with a dynamical encoder-decoder modeling framework. Sci. Rep. 12, 2650 (2022). https://doi.org/10.1038/s41598-022-05810-x

    Article  Google Scholar 

  3. Breunig, M.M., Kriegel, H.P., Ng, R.T., Sander, J.: LOF: identifying density-based local outliers. SIGMOD Rec. 29(2), 93–104 (2000). https://doi.org/10.1145/335191.335388

    Article  Google Scholar 

  4. Cao, Z., Hidalgo, G., Simon, T., Wei, S.E., Sheikh, Y.: Openpose: realtime multi-person 2D pose estimation using part affinity fields (2019)

    Google Scholar 

  5. Comi, A., Polimeni, A., Balsamo, C.: Road accident analysis with data mining approach: evidence from Rome. Transp. Res. Procedia 62, 798–805 (2022) https://doi.org/10.1016/j.trpro.2022.02.099, https://www.sciencedirect.com/science/article/pii/S2352146522002265. 24th Euro Working Group on Transportation Meeting

  6. Craye, C., Karray, F.: Driver distraction detection and recognition using RGB-D sensor. CoRR abs/1502.00250 (2015), http://arxiv.org/abs/1502.00250

  7. Fujiwara, K., et al.: Heart rate variability-based driver drowsiness detection and its validation with EEG. IEEE Trans. Biomed. Eng. 66(6), 1769–1778 (2019). https://doi.org/10.1109/TBME.2018.2879346

    Article  Google Scholar 

  8. Fusek, R., Sojka, E., Gaura, J., Halman, J.: Driver state detection from in-car camera images. In: Bebis, G., et al. (eds.) ISVC 2022. LNCS, pp. 307–319. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20716-7_24

    Chapter  Google Scholar 

  9. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006). https://doi.org/10.1126/science.1127647, https://www.science.org/doi/abs/10.1126/science.1127647

  10. Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications. CoRR abs/1704.04861 (2017). http://arxiv.org/abs/1704.04861

  11. Kingma, D.P., Welling, M.: Auto-encoding variational bayes (2013)

    Google Scholar 

  12. LeCun, Y., Cortes, C., Burges, C.: Mnist handwritten digit database. ATT Labs (2010). http://yann.lecun.com/exdb/mnist 2

  13. Liu, F.T., Ting, K.M., Zhou, Z.H.: Isolation forest. In: 2008 Eighth IEEE International Conference on Data Mining, pp. 413–422 (2008). https://doi.org/10.1109/ICDM.2008.17

  14. Malhotra, P., Ramakrishnan, A., Anand, G., Vig, L., Agarwal, P., Shroff, G.: LSTM-based encoder-decoder for multi-sensor anomaly detection. CoRR abs/1607.00148 (2016). http://arxiv.org/abs/1607.00148

  15. Martin, M., et al.: Drive & act: a multi-modal dataset for fine-grained driver behavior recognition in autonomous vehicles. In: The IEEE International Conference on Computer Vision (ICCV) (2019)

    Google Scholar 

  16. Martìn-delosReyes, L.M., Lardelli-Claret, P., García-Cuerva, L., Rivera-Izquierdo, M., Jiménez-Mejías, E., Martínez-Ruiz, V.: Effect of periodic vehicle inspection on road crashes and injuries: a systematic review. Int. J. Environ. Res. Public Health 18(12) (2021). https://doi.org/10.3390/ijerph18126476, https://www.mdpi.com/1660-4601/18/12/6476

  17. Moslemi, N., Azmi, R., Soryani, M.: Driver distraction recognition using 3D convolutional neural networks. In: 2019 4th International Conference on Pattern Recognition and Image Analysis (IPRIA), pp. 145–151 (2019). https://doi.org/10.1109/PRIA.2019.8786012

  18. Murugan, S., Selvaraj, J., Sahayadhas, A.: Detection and analysis: driver state with electrocardiogram (ECG). Phys. Eng. Sci. Med. 43, 525–537 (2020). https://doi.org/10.1007/s13246-020-00853-8

    Article  Google Scholar 

  19. Ortega, J.D., et al.: DMD: a large-scale multi-modal driver monitoring dataset for attention and alertness analysis. In: Bartoli, A., Fusiello, A. (eds.) ECCV 2020. LNCS, vol. 12538, pp. 387–405. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-66823-5_23

    Chapter  Google Scholar 

  20. Osokin, D.: Real-time 2d multi-person pose estimation on CPU: lightweight openpose (2018). https://doi.org/10.48550/ARXIV.1811.12004

  21. Rousseeuw, P.J., Driessen, K.V.: A fast algorithm for the minimum covariance determinant estimator. Technometrics 41(3), 212–223 (1999). https://doi.org/10.1080/00401706.1999.10485670

    Article  Google Scholar 

  22. Sandler, M., Howard, A.G., Zhu, M., Zhmoginov, A., Chen, L.: Inverted residuals and linear bottlenecks: mobile networks for classification, detection and segmentation. CoRR abs/1801.04381 (2018)

    Google Scholar 

  23. Wang, J., et al.: 100-driver: a large-scale, diverse dataset for distracted driver classification. In: Under Review (2022)

    Google Scholar 

  24. Williams, C., Seeger, M.: Using the nyström method to speed up kernel machines. In: Leen, T., Dietterich, T., Tresp, V. (eds.) Advances in Neural Information Processing Systems. vol. 13. MIT Press (2000)

    Google Scholar 

  25. Xing, Y., Lv, C., Wang, H., Cao, D., Velenis, E., Wang, F.Y.: Driver activity recognition for intelligent vehicles: a deep learning approach. IEEE Trans. Veh. Technol. 68(6), 5379–5390 (2019). https://doi.org/10.1109/TVT.2019.2908425

    Article  Google Scholar 

  26. Xing, Y., et al.: Identification and analysis of driver postures for in-vehicle driving activities and secondary tasks recognition. IEEE Trans. Comput. Soc. Syst. 5(1), 95–108 (2018). https://doi.org/10.1109/TCSS.2017.2766884

    Article  Google Scholar 

  27. Yang, T., Li, Y.F., Mahdavi, M., Jin, R., Zhou, Z.H.: Nyström method vs random Fourier features: a theoretical and empirical comparison. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in Neural Information Processing Systems, vol. 25. Curran Associates, Inc. (2012)

    Google Scholar 

  28. Yeole, M., Jain, R., Menon, R.: Road traffic accident prediction for mixed traffic flow using artificial neural network. Materials Today: Proceedings 77, 832–837 (2023). https://doi.org/10.1016/j.matpr.2022.11.490, https://www.sciencedirect.com/science/article/pii/S2214785322073473. international Conference on ‘Innovations in Mechanical and Civil Engineering’

  29. Zeng, H., Yang, C., Dai, G., Qin, F., Zhang, J., Kong, W.: EEG classification of driver mental states by deep learning. Cogn. Neurodyn. 12, 597–606 (2018). https://doi.org/10.1007/s11571-018-9496-y

    Article  Google Scholar 

  30. Zepf, S., Hernandez, J., Schmitt, A., Minker, W., Picard, R.W.: Driver emotion recognition for intelligent vehicles: a survey. ACM Comput. Surv. 53(3), 1–30 (2020). https://doi.org/10.1145/3388790

    Article  Google Scholar 

  31. Zhang, T., Wang, H., Chen, J., He, E.: Detecting unfavorable driving states in electroencephalography based on a PCA sample entropy feature and multiple classification algorithms. Entropy 22(11), 1248 (2020). https://doi.org/10.3390/e22111248, https://www.mdpi.com/1099-4300/22/11/1248

  32. Zhao, D., Zhong, Y., Fu, Z., Hou, J.J., Zhao, M.: A review for the driving behavior recognition methods based on vehicle multisensor information. J. Adv. Transp. 2022, 1–16 (2022). https://doi.org/10.1155/2022/7287511

    Article  Google Scholar 

Download references

Acknowledgments

This work is partially supported by Grants of SGS No. SP2023/072, VSB - Technical University of Ostrava, Czech Republic.

Author information

Authors and Affiliations

  1. Department of Computer Science, Faculty of Electrical Engineering and Computer Science, VSB - Technical University of Ostrava, 17. listopadu 2172/15, 70800, Ostrava, Czech Republic

    Radovan Fusek, Eduard Sojka, Jan Gaura & Jakub Halman

Authors
  1. Radovan Fusek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eduard Sojka
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jan Gaura
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jakub Halman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Radovan Fusek .

Editor information

Editors and Affiliations

  1. University of Nevada Reno, Reno, NV, USA

    George Bebis

  2. Google Research, Mountain View, CA, USA

    Golnaz Ghiasi

  3. New York University, New York, USA

    Yi Fang

  4. Ben-Gurion University, Be'er Sheva, Israel

    Andrei Sharf

  5. Microsoft Research, Beijing, China

    Yue Dong

  6. The University of Oklahoma, Norman, OK, USA

    Chris Weaver

  7. University of Maryland, Collage Park, MD, USA

    Zhicheng Leo

  8. University of Central Florida, Orlando, FL, USA

    Joseph J. LaViola Jr.

  9. InnerOptic Technology, Hillsborough, NC, USA

    Luv Kohli

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Fusek, R., Sojka, E., Gaura, J., Halman, J. (2023). Driver Anomaly Detection Using Skeleton Images. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2023. Lecture Notes in Computer Science, vol 14361. Springer, Cham. https://doi.org/10.1007/978-3-031-47969-4_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-47969-4_36

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-47968-7

  • Online ISBN: 978-3-031-47969-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation