Skip to main content
SpringerLink
Log in

Supporting Driver Physical State Estimation by Means of Thermal Image Processing

  • Conference paper
  • First Online:
Computational Science – ICCS 2021 (ICCS 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12746))

Included in the following conference series:

Abstract

In the paper we address a problem of estimating a physical state of an observed person by means of analysing facial portrait captured in thermal spectrum. The algorithm consists of facial regions detection combined with tracking and individual features classification. We focus on eyes and mouth state estimation. The detectors are based on Haar-like features and AdaBoost, previously applied to visible-band images. Returned face region is subject to eyes and mouth detection. Further, extracted regions are filtered using Gabor filter bank and the resultant features are classified. Finally, classifiers’ responses are integrated and the decision about driver’s physical state is taken. By using thermal image we are able to capture eyes and mouth states in very adverse lighting conditions, in contrast to the visible-light approaches. Experiments performed on manually annotated video sequences have shown that the proposed approach is accurate and can be a part of current Advanced Driver Assistant Systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Alioua, N., Amine, A., Rziza, M.: Driver’s fatigue detection based on yawning extraction. Int. J. Veh. Technol. 2014, 7 (2014). Article ID: 678786

    Google Scholar 

  2. Andrysiak, T., Choras, M.: Image retrieval based on hierarchical Gabor filters. Int. J. Appl. Math. Comput. Sci. 15(4), 471–480 (2005)

    MathSciNet  MATH  Google Scholar 

  3. Azim, T., Jaffar, M.A., Mirza, A.M.: Fully automated real time fatigue detection of drivers through fuzzy expert systems. Appl. Soft Comput. 18, 25–38 (2014)

    Article  Google Scholar 

  4. Berglund, B., Rossi, G.B., Townsend, J.T., Pendrill, L.R.: Measurement with Persons: Theory, Methods, and Implementation Areas, p. 422. Psychology Press, New York (2013)

    Book  Google Scholar 

  5. Bianconi, F., Fernandez, A.: Evaluation of the effects of Gabor filter parameters on texture classification. Pattern Recogn. 40(12), 3325–3335 (2007)

    Article  Google Scholar 

  6. Burduk, R.: The AdaBoost algorithm with the imprecision determine the weights of the observations. In: Nguyen, N.T., Attachoo, B., Trawiński, B., Somboonviwat, K. (eds.) ACIIDS 2014. LNCS (LNAI), vol. 8398, pp. 110–116. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-05458-2_12

    Chapter  Google Scholar 

  7. Craye, C., Rashwan, A., Kamel, M.S., Karray, F.: A multi-modal driver fatigue and distraction assessment system. Int. J. Intell. Transp. Syst. Res. 14(3), 173–194 (2016). https://doi.org/10.1007/s13177-015-0112-9

    Article  Google Scholar 

  8. Cyganek, B., Gruszczynski, S.: Hybrid computer vision system for drivers’ eye recognition and fatigue monitoring. Neurocomputing 126, 78–94 (2014)

    Article  Google Scholar 

  9. Daza, I.G., Bergasa, L.M., Bronte, S., Yebes, J.J., Almazán, J., Arroyo, R.: Fusion of optimized indicators from Advanced Driver Assistance Systems (ADAS) for driver drowsiness detection. Sensors 14, 1106–1131 (2014)

    Article  Google Scholar 

  10. Forczmański, P.: Performance evaluation of selected thermal imaging-based human face detectors. In: Kurzynski, M., Wozniak, M., Burduk, R. (eds.) CORES 2017. AISC, vol. 578, pp. 170–181. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-59162-9_18

    Chapter  Google Scholar 

  11. Forczmański, P., Kutelski, K.: Driver drowsiness estimation by means of face depth map analysis. In: Pejaś, J., El Fray, I., Hyla, T., Kacprzyk, J. (eds.) ACS 2018. AISC, vol. 889, pp. 396–407. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-03314-9_34

    Chapter  Google Scholar 

  12. Fu, R., Wang, H., Zhao, W.: Dynamic driver fatigue detection using hidden Markov model in real driving condition. Exp. Syst. Appl. 63, 397–411 (2016)

    Article  Google Scholar 

  13. Hussien, M.N., Lye, M., Fauzi, M.F.A., Seong, T.C., Mansor, S.: Comparative analysis of eyes detection on face thermal images. In: 2017 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), Kuching, pp. 385–389 (2017)

    Google Scholar 

  14. Jo, J., Lee, S.J., Park, K.R., Kim, I.J., Kim, J.: Detecting driver drowsiness using feature-level fusion and user-specific classification. Exp. Syst. Appl. 41(4), 1139–1152 (2014)

    Article  Google Scholar 

  15. Josephin, J., Lakshmi, C., James, S.: A review on the measures and techniques adapted for the detection of driver drowsiness. In: IOP Conference Series: Materials Science and Engineering, vol. 993, p. 012101 (2020). https://doi.org/10.1088/1757-899X/993/1/012101

  16. Knipling, R.R., Wierwille, W.W.: Vehicle-based drowsy driver detection: current status and future prospects. In: Moving Toward Deployment. Proceedings of the IVHS America Annual Meeting. 2 Volumes, vol. 1, pp. 245–256 (1994)

    Google Scholar 

  17. Knapik, M., Cyganek, B.: Driver’s fatigue recognition based on yawn detection in thermal images. Neurocomputing 338, 274–292 (2019)

    Article  Google Scholar 

  18. Kong, W., Zhou, L., Wang, Y., Zhang, J., Liu, J., Gao, S.: A system of driving fatigue detection based on machine vision and its application on smart device. J. Sens. 2015, 11 (2015). https://doi.org/10.1155/2015/548602. Article ID: 548602

  19. Krishnasree, V., Balaji, N., Rao, P.S.: A real time improved driver fatigue monitoring system. WSEAS Trans. Sig. Process. 10, 146–155 (2014)

    Google Scholar 

  20. Małecki, K., Forczmański, P., Nowosielski, A., Smoliński, A., Ozga, D.: A new benchmark collection for driver fatigue research based on thermal, depth map and visible light imagery. In: Burduk, R., Kurzynski, M., Wozniak, M. (eds.) CORES 2019. AISC, vol. 977, pp. 295–304. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-19738-4_30

    Chapter  Google Scholar 

  21. Małecki, K., Nowosielski, A., Forczmański, P.: Multispectral data acquisition in the assessment of driver’s fatigue. In: Mikulski, J. (ed.) TST 2017. CCIS, vol. 715, pp. 320–332. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66251-0_26

    Chapter  Google Scholar 

  22. Mitas, A., Czapla, Z., Bugdol, M., Rygula, A.: Registration and evaluation of biometric parameters of the driver to improve road safety, pp. 71–79. Scientific Papers of Transport, Silesian University of Technology (2010)

    Google Scholar 

  23. Nissimagoudar, P.C., Nandi, A.V., Gireesha, H.M.: A feature extraction and selection method for EEG based driver alert/drowsy state detection. In: Abraham, A., Jabbar, M.A., Tiwari, S., Jesus, I.M.S. (eds.) SoCPaR 2019. AISC, vol. 1182, pp. 297–306. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-49345-5_31

    Chapter  Google Scholar 

  24. Nowosielski, A.: Vision-based solutions for driver assistance. J. Theor. Appl. Comput. Sci. 8(4), 35–44 (2014)

    Google Scholar 

  25. Nowosielski, A., Forczmański, P.: Touchless typing with head movements captured in thermal spectrum. Pattern Anal. Appl. 22(3), 841–855 (2018). https://doi.org/10.1007/s10044-018-0741-0

    Article  MathSciNet  Google Scholar 

  26. Poster, D., Hu, S., Nasrabadi, N.M., Riggan, B.S.: An examination of deep-learning based landmark detection methods on thermal face imagery. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 980–987 (2019)

    Google Scholar 

  27. Sacco, M., Farrugia, R.A.: Driver fatigue monitoring system using support vector machines. In: 2012 5th International Symposium on Communications, Control and Signal Processing, Rome, pp. 1–5 (2012)

    Google Scholar 

  28. Satish, K., Lalitesh, A., Bhargavi, K., Prem, M.S., Anjali, T.: Driver drowsiness detection. In: 2020 International Conference on Communication and Signal Processing (ICCSP), pp. 0380–0384 (2020)

    Google Scholar 

  29. Smolensky, M.H., Di Milia, L., Ohayon, M.M., Philip, P.: Sleep disorders, medical conditions, and road accident risk. Accid. Anal. Prev. 43(2), 533–548 (2011)

    Article  Google Scholar 

  30. Trutschel, U., Sirois, B., Sommer, D., Golz, M., Edwards, D.: PERCLOS: an alertness measure of the past. In: Proceedings of the Sixth International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design, pp. 172–179 (2017)

    Google Scholar 

  31. Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004). https://doi.org/10.1023/B:VISI.0000013087.49260.fb

    Article  Google Scholar 

  32. Virginia Tech Transportation Institute: Day or Night, Driving while Tired a Leading Cause of Accidents (2013). http://www.vtnews.vt.edu/articles/2013/04/041513-vtti-fatigue.html

  33. Weller, G., Schlag, B.: Road user behavior model. Deliverable D8 project RIPCORD-ISERET, 6 Framework Programme of the European Union (2007). http://ripcord.bast.de

  34. Zhang, Y., Hua, C.: Driver fatigue recognition based on facial expression analysis using local binary patterns. Optik Int. J. Light Electron Opt. 126(23), 4501–4505 (2015)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Science and Information Technology, West Pomeranian University of Technology, Szczecin, Żołnierska Street 49, 71-210, Szczecin, Poland

    Paweł Forczmański & Anton Smoliński

Authors
  1. Paweł Forczmański
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anton Smoliński
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paweł Forczmański .

Editor information

Editors and Affiliations

  1. AGH University of Science and Technology, Krakow, Poland

    Maciej Paszynski

  2. Ludwig-Maximilians-Universität München, Munich, Germany

    Dieter Kranzlmüller

  3. University of Amsterdam, Amsterdam, The Netherlands

    Valeria V. Krzhizhanovskaya

  4. University of Tennessee at Knoxville, Knoxville, TN, USA

    Jack J. Dongarra

  5. University of Amsterdam, Amsterdam, The Netherlands

    Peter M.A. Sloot

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Forczmański, P., Smoliński, A. (2021). Supporting Driver Physical State Estimation by Means of Thermal Image Processing. In: Paszynski, M., Kranzlmüller, D., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M. (eds) Computational Science – ICCS 2021. ICCS 2021. Lecture Notes in Computer Science(), vol 12746. Springer, Cham. https://doi.org/10.1007/978-3-030-77977-1_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-77977-1_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-77976-4

  • Online ISBN: 978-3-030-77977-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation