skip to main content
research-article

Towards a Reliable Ground Truth for Drowsiness: A Complexity Analysis on the Example of Driver Fatigue

Published: 18 June 2020 Publication History

Abstract

The increasing number and complexity of advanced driver assistance systems (ADAS) pave the way for fully automated driving. Automated vehicles are said to increase road safety and prevent human-made (fatal) accidents, amongst others. In the lower levels of automation, however, the driver is still responsible as a fallback authority. As a consequence, systems that reliably monitor the driver's state, especially the risk factor drowsiness, become increasingly essential to ensure the driver's ability to take over control from the vehicle on time. In research, the use of supervised machine learning for drowsiness detection is the prevalent method. As the ground truth for drowsiness is both application- and user-dependent, and no golden standard exists for its definition, measures are usually applied in the form of observer ratings. Also, in this work, observer ratings were investigated with regard to the required level of detail/complexity. To this end, video data, recorded within a simulator study (N = 30) comprised of each 45-minute manual and automated driving sessions, were evaluated by trained raters. Correlation analysis results show that - depending on the number of drowsiness levels - a comparable ground truth can be generated by reducing the rating frequency and thus the rating complexity by a factor of five. The knowledge gained can be used in future studies in this research area, the collection of a reliable and valid ground truth of drowsiness, as well as for improving the process in developing interactive drowsiness detection systems.

References

[1]
Christer Ahlstrom, Carina Fors, Anna Anund, and David Hallvig. 2015. Video-based observer rated sleepiness versus self-reported subjective sleepiness in real road driving. European Transport Research Review, Vol. 7, 4 (23 November 2015), 38. https://doi.org/10.1007/s12544-015-0188-y
[2]
T. Åkerstedt and M. Gillberg. 1990. Subjective and objective sleepiness in the active individual. International Journal of Neuroscience, Vol. 52 (1990), 29--37. https://doi.org/10.3109/00207459008994241
[3]
Anna Anund, Carina Fors, David Hallvig, Torbjörn Åkerstedt, and Göran Kecklund. 2013. Observer Rated Sleepiness and Real Road Driving: An Explorative Study. PLOS ONE, Vol. 8, 5 (May 2013), 1--8. https://doi.org/10.1371/journal.pone.0064782
[4]
G. Borghini, G. Vecchiato, J. Toppi, L. Astolfi, A. Maglione, R. Isabella, C. Caltagirone, W. Kong, D. Wei, Z. Zhou, L. Polidori, S. Vitiello, and F. Babiloni. 2012. Assessment of mental fatigue during car driving by using high resolution EEG activity and neurophysiologic indices. In 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 6442--6445. https://doi.org/10.1109/EMBC.2012.6347469
[5]
Karel Brookhuis, Dick de Waard, and Will Janssen. 2019. Behavioural impacts of Advanced Driver Assistance Systems-an overview. European Journal of Transport and Infrastructure Research, Vol. 1, 3 (2019). https://journals.open.tudelft.nl/index.php/ejtir/article/view/3667
[6]
Zhuo Chen, Ruizhou Ding, Ting Wu Chin, and Diana Marculescu. 2019. Understanding the impact of label granularity on CNN-based image classification. Technical Report. 895--904 pages. https://doi.org/10.1109/ICDMW.2018.00131
[7]
Charlotte Jacobé de Naurois, Christophe Bourdin, Anca Stratulat, Emmanuelle Diaz, and Jean-Louis Vercher. 2019. Detection and prediction of driver drowsiness using artificial neural network models. Accident Analysis & Prevention, Vol. 126 (2019), 95 -- 104. https://doi.org/10.1016/j.aap.2017.11.038
[8]
EuroNCAP. 2017. EuroNCAP 2025 Roadmap. (2017), 1--17.
[9]
F. Friedrichs and B. Yang. 2010. Drowsiness monitoring by steering and lane data based features under real driving conditions. In 2010 18th European Signal Processing Conference. 209--213.
[10]
Rongrong Fu, Hong Wang, and Wenbo Zhao. 2016. Dynamic driver fatigue detection using hidden Markov model in real driving condition. Expert Systems with Applications, Vol. 63 (2016), 397--411. https://doi.org/10.1016/j.eswa.2016.06.042
[11]
Jasper Gielen and Jean-Marie Aerts. 2019. Feature Extraction and Evaluation for Driver Drowsiness Detection Based on Thermoregulation. Applied Sciences, Vol. 9, 17 (2019). https://doi.org/10.3390/app9173555
[12]
IPG Automotive GmbH. 2018. CarMaker: Virtual testing of automobiles and light-duty vehicles. https://ipg-automotive.com/products-services/simulation-software/carmaker/ (accesssed on 21 April 2020).
[13]
National Highway Traffic Safety Administration and Us Department of Transportation. 2015. TRAFFIC SAFETY FACTS Critical Reasons for Crashes Investigated in the National Motor Vehicle Crash Causation Survey. (2015). https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812115
[14]
E. Hoddes, V. Zarcone, H. Smythe, R. Phillips, and W. C. Dement. 1973. Quantification of Sleepiness: A New Approach. Psychophysiology, Vol. 10, 4 (1973), 431--436. https://doi.org/10.1111/j.1469--8986.1973.tb00801.x
[15]
Murray Johns. 1998. Rethinking the assessment of sleepiness. Sleep Medicine Reviews, Vol. 2, 1 (1998), 3 -- 15. https://doi.org/10.1016/S1087-0792(98)90050--8
[16]
Murray W. Johns. 1991. A New Method for Measuring Daytime Sleepiness: The Epworth Sleepiness Scale. Sleep, Vol. 14, 6 (1991), 540--545. https://doi.org/10.1093/sleep/14.6.540
[17]
Sang-Joong Jung, Heung-Sub Shin, and Wan-Young Chung. 2014. Driver fatigue and drowsiness monitoring system with embedded electrocardiogram sensor on steering wheel. IET Intelligent Transport Systems, Vol. 8, 1 (2014), 43--50. https://doi.org/10.1049/iet-its.2012.0032
[18]
Katja Karrer-Gauß. 20110. Prospektive Bewertung von Systemen zur Mü digkeitserkennung Ableitung von Gestaltungsempfehlungen zur Vermeidung von Risikokompensation aus empirischen Untersuchungen. Ph.D. Dissertation. Verkehrs- und Maschinensysteme, Technische Universitat Berlin, Berlin, Germany.
[19]
Thomas Kundinger and Andreas Riener. 2020. The Potential of Wrist-Worn Wearables for Driver Drowsiness Detection: A Feasibility Analysis (UMAP '20). in press. https://doi.org/10.1145/3340631.3394852
[20]
Thomas Kundinger, Andreas Riener, Nikoletta Sofra, and Klemens Weigl. 2018. Drowsiness Detection and Warning in Manual and Automated Driving: Results from Subjective Evaluation (AutomotiveUI '18). 229--236. https://doi.org/10.1145/3239060.3239073
[21]
Thomas Kundinger, Nikoletta Sofra, and Andreas Riener. 2020 a. Assessment of the Potential of Wrist-Worn Wearable Sensors for Driver Drowsiness Detection. Sensors, Vol. 20, 4 (2020). https://doi.org/10.3390/s20041029
[22]
Thomas Kundinger, Phani Krishna Yalavarthi, Andreas Riener, Philipp Wintersberger, and Clemens Schartmüller. 2020 b. Feasibility of smart wearables for driver drowsiness detection and its potential among different age groups. International Journal of Pervasive Computing and Communications, Vol. 16, 1 (2020), 1--23. https://doi.org/10.1108/IJPCC-03--2019-0017
[23]
Boon-leng Lee, Boon-giin Lee, G. Li, and W.-Y. Chung. 2014. Wearable Driver Drowsiness Detection System Based on Smartwatch. Korea Institute of Signal Processing and Systems, Vol. 15 (2014), 134--146.
[24]
Hyeonjeong Lee, Jaewon Lee, and Miyoung Shin. 2019. Using Wearable ECG/PPG Sensors for Driver Drowsiness Detection Based on Distinguishable Pattern of Recurrence Plots. Electronics, Vol. 8, 2 (2019). https://doi.org/10.3390/electronics8020192
[25]
L. B. Leng, L. B. Giin, and W. Chung. 2015. Wearable driver drowsiness detection system based on biomedical and motion sensors. In 2015 IEEE Sensors Journal. 1--4. https://doi.org/10.1109/ICSENS.2015.7370355
[26]
Gang Li and Wan-Young Chung. 2015. A Context-Aware EEG Headset System for Early Detection of Driver Drowsiness. Sensors, Vol. 15, 8 (2015), 20873--20893. https://doi.org/10.3390/s150820873
[27]
Gang Li, Boon Leng Lee, and Wan Young Chung. 2015a. Smartwatch-Based Wearable EEG System for Driver Drowsiness Detection. IEEE Sensors Journal, Vol. 15, 12 (2015), 7169--7180. https://doi.org/10.1109/JSEN.2015.2473679
[28]
Quanzhe Li, Juan Wu, Shin-Dug Kim, and Cheong-Ghil Kim. 2015b. Hybrid Driver Fatigue Detection System Based on Data Fusion with Wearable Sensor Devices.
[29]
Zuojin Li, Shengbo Eben Li, Renjie Li, Bo Cheng, and Jinliang Shi. 2017. Online Detection of Driver Fatigue Using Steering Wheel Angles for Real Driving Conditions. Sensors, Vol. 17, 3 (2017). https://doi.org/10.3390/s17030495
[30]
Todd Litman. 2017. Autonomous Vehicle Implementation Predictions. Technical Report.
[31]
Alina Mashko. 2017. Subjective Methods for Assessment of Driver Drowsiness. Acta Polytechnica CTU Proceedings, Vol. 12, 64. https://doi.org/10.14311/APP.2017.12.0064
[32]
Anthony D. McDonald, Chris Schwarz, John D Lee, and Timothy L Brown. 2012. Real-time detection of drowsiness related lane departures using steering wheel angle. In Proceedings of the Human Factors and Ergonomics Society. 2201--2205. https://doi.org/10.1177/1071181312561464
[33]
Aqsa Mehreen, Syed Muhammad Anwar, Muhammad Haseeb, Muhammad Majid, and Muhammad Obaid Ullah. 2019. A Hybrid Scheme for Drowsiness Detection Using Wearable Sensors. IEEE Sensors Journal, Vol. 19, 13 (2019), 5119--5126. https://doi.org/10.1109/JSEN.2019.2904222
[34]
Lars Michael, Sven Passmann, and Ruth Becker. 2012. Electrodermal lability as an indicator for subjective sleepiness during total sleep deprivation. Journal of Sleep Research, Vol. 21, 4 (2012), 470--478. https://doi.org/10.1111/j.1365--2869.2011.00984.x
[35]
J. M. Owens, T. A. Dingus, F. Guo, Y. Fang, M. Perez, J. McClafferty, and B Tefft. 2018. Prevalence of Drowsy Driving Crashes: Estimates from a Large-Scale Naturalistic Driving Study. (Research Brief.). Washington, D.C.: AAA Foundation for Traffic Safety (February 2018).
[36]
Mohsen Poursadeghiyan, Adel Mazloumi, Gebraeil Nasl Saraji, Ali Niknezhad, Arash Akbarzadeh, and Mohammad Hossein Ebrahimi. 2017. Determination the Levels of Subjective and Observer Rating of Drowsiness and Their Associations with Facial Dynamic Changes. Iranian Journal of Public Health, Vol. 46, 1 (2017), 93--102.
[37]
Arun Sahayadhas, Kenneth Sundaraj, and Murugappan Murugappan. 2012. Detecting Driver Drowsiness Based on Sensors: A Review. Sensors, Vol. 12, 12 (2012), 16937--16953. https://doi.org/10.3390/s121216937
[38]
J. Schmidt, C. Braunagel, W. Stolzmann, and K. Karrer-Gauß. 2016. Driver drowsiness and behavior detection in prolonged conditionally automated drives. In 2016 IEEE Intelligent Vehicles Symposium (IV). 400--405. https://doi.org/10.1109/IVS.2016.7535417
[39]
Azmeh Shahid, Kate Wilkinson, Shai Marcu, and Colin M. Shapiro. 2012. Stanford Sleepiness Scale (SSS). Springer New York, 369--370. https://doi.org/10.1007/978--1--4419--9893--4_91
[40]
G. Sikander and S. Anwar. 2019. Driver Fatigue Detection Systems: A Review. IEEE Transactions on Intelligent Transportation Systems, Vol. 20, 6 (June 2019), 2339--2352. https://doi.org/10.1109/TITS.2018.2868499
[41]
Jeremy D. Slater. 2008. A definition of drowsiness: One purpose for sleep? Medical Hypotheses, Vol. 71, 5 (November 2008), 641--644. https://doi.org/10.1016/j.mehy.2008.05.035
[42]
Sleep Health Foundation. 2015. Sleep Needs Across The Lifespan. http://www.sleephealthfoundation.org.au/files/pdfs/Sleep-Needs-Across-Lifespan.pdf (accesssed on 21 April 2020).
[43]
Society of Automotive Engineers (SAE) International. 2018. Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles.
[44]
Udo Trutschel, Bill Sirois, David Sommer, Martin Golz, and Dave Edwards. 2011. PERCLOS: An Alertness Measure of the Past. Proceedings of the 6th International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design (2011), 172--179. https://doi.org/10.17077/drivingassessment.1394
[45]
F. Wang, S. Wang, X. Wang, Y. Peng, and Y. Yang. 2014. Design of driving fatigue detection system based on hybrid measures using wavelet-packets transform. In 2014 IEEE International Conference on Robotics and Automation (ICRA). 4037--4042. https://doi.org/10.1109/ICRA.2014.6907445
[46]
Veronika Weinbeer, T. Muhr, Klaus Bengler, C. Baur, J. Radlmayr, and J. Bill. 2017. Highly automated driving: How to get the driver drowsy and how does drowsiness influence various take-over-aspects?. In 8. Tagung Fahrerassistenz. Lehrstuhl für Fahrzeugtechnik mit TÜ V SÜ D Akademie, München.
[47]
Douglas M Wiegand, Julie Mcclafferty, Shelby E Mcdonald, and Richard J Hanowski. 2009. Development and Evaluation of a Naturalistic Observer Rating of Drowsiness Protocol Final Report. The National Surface Transportation Safety Center for Excellence (2009).
[48]
Walter W. Wierwille and Lynne A. Ellsworth. 1994. Evaluation of driver drowsiness by trained raters. Accident Analysis and Prevention, Vol. 26, 5 (1994), 571--581.

Cited By

View all
  • (2023)Drowsiness Mitigation Through Driver State Monitoring Systems: A Scoping ReviewHuman Factors: The Journal of the Human Factors and Ergonomics Society10.1177/00187208231208523Online publication date: 20-Nov-2023
  • (2021)Drivers Fatigue Level Prediction Using Facial, and Head Behavior InformationIEEE Access10.1109/ACCESS.2021.31085619(121686-121697)Online publication date: 2021
  • (2020)Research of Driving Fatigue Detection Based on Gaussian Mixture Hidden Markov ModelSAE Technical Paper Series10.4271/2020-01-5158Online publication date: 30-Dec-2020

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction
Proceedings of the ACM on Human-Computer Interaction  Volume 4, Issue EICS
EICS
June 2020
534 pages
EISSN:2573-0142
DOI:10.1145/3407187
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 June 2020
Online AM: 07 May 2020
Published in PACMHCI Volume 4, Issue EICS

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. automated driving
  2. driver drowsiness detection
  3. driver state
  4. machine learning
  5. simulator study
  6. subjective measures

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)31
  • Downloads (Last 6 weeks)6
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Drowsiness Mitigation Through Driver State Monitoring Systems: A Scoping ReviewHuman Factors: The Journal of the Human Factors and Ergonomics Society10.1177/00187208231208523Online publication date: 20-Nov-2023
  • (2021)Drivers Fatigue Level Prediction Using Facial, and Head Behavior InformationIEEE Access10.1109/ACCESS.2021.31085619(121686-121697)Online publication date: 2021
  • (2020)Research of Driving Fatigue Detection Based on Gaussian Mixture Hidden Markov ModelSAE Technical Paper Series10.4271/2020-01-5158Online publication date: 30-Dec-2020

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media