skip to main content
10.1145/3505711.3505719acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicaaiConference Proceedingsconference-collections
research-article

Deep Learning-based EEG Detection of Mental Alertness States from Drivers under Ethical Aspects

Published: 28 March 2022 Publication History

Abstract

One of the most critical factors for a successful road trip is a high degree of alertness while driving. Even a split second of inattention or sleepiness in a crucial moment, will make the difference between life and death. Several prestigious car manufacturers are currently pursuing the aim of automated drowsiness identification to resolve this problem. The path between neuro-scientific research in connection with artificial intelligence and the preservation of the dignity of human individual’s and its inviolability, is very narrow. The key contribution of this work is a system of data analysis for EEGs during a driving session, which draws on previous studies analyzing heart rate (ECG), brain waves (EEG), and eye function (EOG). The gathered data is hereby treated as sensitive as possible, taking ethical regulations into consideration. Obtaining evaluable signs of evolving exhaustion includes techniques that obtain sleeping stage frequencies, problematic are hereby the correlated interference’s in the signal. This research focuses on a processing chain for EEG band splitting that involves band-pass filtering, principal component analysis (PCA), independent component analysis (ICA) with automatic artefact severance, and fast fourier transformation (FFT). The classification is based on a step-by-step adaptive deep learning analysis that detects theta rhythms as a drowsiness predictor in the pre-processed data. It was possible to obtain an offline detection rate of 89% and an online detection rate of 73%. The method is linked to the simulated driving scenario for which it was developed. This leaves space for more optimization on laboratory methods and data collection during wakefulness-dependent operations.

References

[1]
10.03.2021. Sekundenschlaf am Steuer: Lebensgefährlich | ADAC. https://www.adac.de/verkehr/verkehrssicherheit/verkehrsmedizin/muedigkeit-sekundenschlaf-auto/
[2]
2012. Euro NCAP Roadmap 2025. https://cdn.euroncap.com/media/30700/euroncap-roadmap-2025-v4.pdf. accessed: February 22, 2022.
[3]
J.Todd Arnedt, Gerald J.S Wilde, Peter W. Munt, and Alistair W. MacLean. 2001. How do prolonged wakefulness and alcohol compare in the decrements they produce on a simulated driving task?Accident; analysis and prevention 33, 3 (2001), 337–344. https://doi.org/10.1016/s0001-4575(00)00047-6
[4]
Chris Baynes. 15.01.2019. Chinese schools scanning children’s brains to see if they are concentrating. The Independent(15.01.2019). https://www.independent.co.uk/news/world/asia/china-schools-scan-brains-concentration-headbands-children-brainco-focus-a8728951.html
[5]
L. M. Bergasa, J. Nuevo, M. A. Sotelo, R. Barea, and M. E. Lopez. 2006. Real-Time System for Monitoring Driver Vigilance. IEEE Transactions on Intelligent Transportation Systems 7, 1(2006), 63–77. https://doi.org/10.1109/TITS.2006.869598
[6]
S Boverie, JM Leqellec, and A Hirl. 1998. Intelligent systems for video monitoring of vehicle cockpit. SAE international.
[7]
Sasha Burwell, Matthew Sample, and Eric Racine. 2017. Ethical aspects of brain computer interfaces: a scoping review. BMC Medical Ethics 18, 1 (2017), 60. https://doi.org/10.1186/s12910-017-0220-y
[8]
E. Cherry. 1953. Some Experiments on the Recognition of Speech With One and With Two Ears. Journal of The Acoustical Society of America 25 (09 1953). https://doi.org/10.1121/1.1907229
[9]
clederhouse. 30.10.2019. Too many Americans admit to driving while drowsy. American Academy of Sleep Medicine(30.10.2019). https://aasm.org/too-many-americans-admit-to-driving-while-drowsy/
[10]
Daimler AG. 2020. Der ATTENTION ASSIST und die Konditionssicherheit: Mach mal Pause. https://media.daimler.com/marsMediaSite/de/instance/ko/Der-ATTENTION-ASSIST-und-die-Konditionssicherheit-Mach-mal-Pause.xhtml?oid=9361847
[11]
D. Dawson and K. Reid. 1997. Fatigue, alcohol and performance impairment. Nature 388, 6639 (1997), 235. https://doi.org/10.1038/40775
[12]
M. Duvinage, T. Castermans, M. Petieau, and T. Hoellinger. 2013. Performance of the Emotiv Epoc headset for P300-based applications. BioMedical Engineering OnLine 12 (2013), 56. https://doi.org/10.1186/1475-925X-12-56
[13]
J. Engstroem, E. Johansson, and J. Oestlund. 2015. Effects of visual and cognitive load in real and simulated motorway driving. Elsevier 8(2015), 97–120.
[14]
Center for Disease Control and Prevention. 2019. The Drowsy Driving Problem. accessed: February 22, 2022.
[15]
Alexandre Gramfort, Martin Luessi, Eric Larson, Denis Engemann, Daniel Strohmeier, Christian Brodbeck, and Roman et al. Goj. 2013. MEG and EEG data analysis with MNE-python. Frontiers in neuroscience 7 (12 2013), 267. https://doi.org/10.3389/fnins.2013.00267
[16]
Ruey S. Huang, Chung J. Kuo, Ling-Ling Tsai, and O.T.C. Chen. 1996. EEG pattern recognition-arousal states detection and classification. In The 1996 IEEE international conference on neural networks. IEEE, 641–646. https://doi.org/10.1109/ICNN.1996.548971
[17]
Aapo Hyvärinen, Juha Karhunen, and Erkki Oja. 2001. Independent Component Analysis. Vol. 26. 165–179 pages. https://doi.org/10.1002/0471221317
[18]
Imanuel Kant. 10.03.2021. Critique of Practical Reason. https://www.goodreads.com/quotes/21053-two-things-fill-the-mind-with-ever-new-and-increasing
[19]
H. Jasper. 1958. Report of the committee on methods of clinical examination in electroencephalography. Electroencephalography and Clinical Neurophysiology 10 (1958), 370–375. https://doi.org/10.1016/0013-4694(58)90053-1
[20]
Robin R. Johnson, Djordje P. Popovic, Richard E. Olmstead, Maja Stikic, Daniel J. Levendowski, and Chris Berka. 2011. Drowsiness/alertness algorithm development and validation using synchronized EEG and cognitive performance to individualize a generalized model. Biological psychology 87, 2 (2011), 241–250. https://doi.org/10.1016/j.biopsycho.2011.03.003
[21]
Kosuke Kaida, Masaya Takahashi, Torbjörn Åkerstedt, Akinori Nakata, Yasumasa Otsuka, Takashi Haratani, and Kenji Fukasawa. 2006. Validation of the Karolinska Sleepiness Scale against performance and EEG variables. Clinical neurophysiology : official journal of the International Federation of Clinical Neurophysiology 117 (08 2006), 1574–1581. https://doi.org/10.1016/j.clinph.2006.03.011
[22]
Nico Kaptein, Jan Theeuwes, and Richard van der Horst. 1996. Driving Simulator Validity: Some Considerations. Transportation Research Record 1550 (11 1996), 30–36. https://doi.org/10.3141/1550-05
[23]
K. B. Khalifa, M. H. Bedoui, R. Raytchev, and M. Dogui. 2001. A portable device for alertness detection. In Proceedings, 1st Annual International IEEE-EMBS Special Topic Conference on Microtechnologies in Medicine & Biology, David Beebe and André Dittmar (Eds.). IEEE, Piscataway, NJ, 584–586. https://doi.org/10.1109/MMB.2000.893852
[24]
Rami N. Khushaba, Sarath Kodagoda, Sara Lal, and Gamini Dissanayake. 2011. Driver drowsiness classification using fuzzy wavelet-packet-based feature-extraction algorithm. IEEE transactions on bio-medical engineering 58, 1(2011), 121–131. https://doi.org/10.1109/TBME.2010.2077291
[25]
N. Lamond and D. Dawson. 1999. Quantifying the performance impairment associated with fatigue. Journal of sleep research 8, 4 (1999), 255–262. https://doi.org/10.1046/j.1365-2869.1999.00167.x
[26]
Boon-Leng Lee, Boon-Giin Lee, and Wan-Young Chung. 2016. Standalone Wearable Driver Drowsiness Detection System in a Smartwatch. IEEE Sensors Journal 16, 13 (2016), 5444–5451. https://doi.org/10.1109/JSEN.2016.2566667
[27]
Chin-Teng Lin, Ruei-Cheng Wu, Sheng-Fu Liang, Wen-Hung Chao, Yu-Jie Chen, and Tzyy-Ping Jung. 2005. EEG-based drowsiness estimation for safety driving using independent component analysis. IEEE Transactions on Circuits and Systems I: Regular Papers 52, 12(2005), 2726–2738. https://doi.org/10.1109/TCSI.2005.857555
[28]
Charles C. Liu, Simon G. Hosking, and Michael G. Lenné. 2009. Predicting driver drowsiness using vehicle measures: recent insights and future challenges. Journal of safety research 40, 4 (2009), 239–245. https://doi.org/10.1016/j.jsr.2009.04.005
[29]
Scott Makeig and Julie Onton. 2012. ERP features and EEG dynamics: An ICA perspective. The Oxford Handbook of Event-Related Potential Components (01 2012). https://doi.org/10.1093/oxfordhb/9780195374148.013.0035
[30]
Henry J. Moller, Leonid Kayumov, Eric L. Bulmash, Jonathan Nhan, and Colin M. Shapiro. 2006. Simulator performance, microsleep episodes, and subjective sleepiness: normative data using convergent methodologies to assess driver drowsiness. Journal of psychosomatic research 61, 3 (2006), 335–342. https://doi.org/10.1016/j.jpsychores.2006.04.007
[31]
Hanbit Park, Seungwon Oh, and Minsoo Hahn. 2009. Drowsy driving detection based on human pulse wave by photoplethysmography signal processing. In Proceedings of the 3rd International Universal Communication Symposium(ACM Digital Library), Kazumasa Enami (Ed.). ACM, New York, NY, 89. https://doi.org/10.1145/1667780.1667798
[32]
P. Pasler. 2017. Artefaktkorrektur und verfeinerte Qualitätsmetriken für ein EEG-basiertes System zur Müdigkeitserkennung.(German). Master’s thesis. University of Reutlingen, Reutlingen GER.
[33]
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, and Zeming et al. Lin. 2017. Automatic differentiation in PyTorch. In NIPS-W.
[34]
RobertBoschGmbH. 2015. Notbremssysteme und Spurassistenten sind auf dem Vormarsch. https://www.bosch-presse.de/pressportal/de/de/notbremssysteme-und-spurassistenten-sind-auf-dem-vormarsch-43164.html
[35]
E. Rogado, J. L. Garcia, R. Barea, L. M. Bergasa, and E. Lopez. 2009. Driver fatigue detection system. In IEEE International Conference on Robotics and Biomimetics, 2008. IEEE, Piscataway, NJ, 1105–1110. https://doi.org/10.1109/ROBIO.2009.4913155
[36]
Sakhavi, Guan, and Yan. 2018. Learning Temporal Information for Brain-Computer Interface Using Convolutional Neural Networks. IEEE Transactions on Neural Networks and Learning Systems (2018).
[37]
Gulbadan Sikander and Shahzad Anwar. 2019. Driver Fatigue Detection Systems: A Review. IEEE Transactions on Intelligent Transportation Systems 20, 6(2019), 2339–2352. https://doi.org/10.1109/TITS.2018.2868499
[38]
Michael Simon, Eike Schmidt, Wilhelm Kincses, Martin Fritzsche, Andreas Bruns, Claus Aufmuth, and Martin et al. Bogdan. 2011. EEG alpha spindle measures as indicators of driver fatigue under real traffic conditions. Clinical neurophysiology : official journal of the International Federation of Clinical Neurophysiology 122 (02 2011), 1168–1178. https://doi.org/10.1016/j.clinph.2010.10.044
[39]
Rahul Kumar Singh, Archisman Sarkar, Rupak Kumar Thakur, and C. S. Anoop. 1/8/2016 - 1/10/2016. A real-time heart-rate monitor using non-contact electrocardiogram for automotive drivers. In 2016 IEEE First International Conference on Control, Measurement and Instrumentation (CMI). IEEE, 484–488. https://doi.org/10.1109/CMI.2016.7413795
[40]
IEEE Staff (Ed.). 2011. 2011 Computing in Cardiology Conference. IEEE, [Place of publication not identified].
[41]
A. SUBASI. 2005. Automatic recognition of alertness level from EEG by using neural network and wavelet coefficients. Expert Systems with Applications 28, 4 (2005), 701–711. https://doi.org/10.1016/j.eswa.2004.12.027
[42]
Toyota. 2008. Toyota Enhances Pre-crash Safety System with Eye Monitor. https://web.archive.org/web/20160928085623http://www.toyota.co.jp/en/news/08/0122.html
[43]
H Ueno, M Kaneda, and M Tsukino. 1994. Development of drowsiness detection system. In Proc. of 1994 vehicle navigation and information systems Conference. IEEE.
[44]
Ricardo Nuno Vigaro. 1997. Extraction of ocular artefacts from EEG using independent component analysis. Electroencephalography and Clinical Neurophysiology (1997).
[45]
F. C Viola, B Edmonds, T Schneider, T Eichele, and S Debener. 2009. Semi-automatic identification of independent components representing EEG artifact. Clinical Neurophysiology(2009).
[46]
F. C Viola, J Thorne, B Edmonds, T Schneider, T Eichele, and S Debener. 2009. Semi-automatic identification of independent components representing EEG artifact using independent component analysis. Clinical Neurophysiology(2009).
[47]
Volkswagen. 2016. Müdigkeitserkennung. inside.volkswagen.de/Mach-mal-Pause.html
[48]
Aleksandra Vuckovic, Vlada Radivojevic, Andrew C.N. Chen, and Dejan Popovic. 2002. Automatic recognition of alertness and drowsiness from EEG by an artificial neural network. Medical Engineering & Physics 24, 5 (2002), 349–360. https://doi.org/10.1016/S1350-4533(02)00030-9
[49]
Nathaniel F. Watson, Timothy Morgenthaler, Ronald Chervin, Kelly Carden, Douglas Kirsch, David Kristo, Raman Malhotra, Jennifer Martin, Kannan Ramar, Ilene Rosen, Terri Weaver, and Merrill Wise. 2015. Confronting Drowsy Driving: The American Academy of Sleep Medicine Perspective. Journal of clinical sleep medicine : JCSM : official publication of the American Academy of Sleep Medicine 11, 11 (2015), 1335–1336. https://doi.org/10.5664/jcsm.5200
[50]
W. W. Wierwille, R. J. Fairbanks, United States. National Highway Traffic Safety Administration. Office of Crash Avoidance Research, L. A. Ellsworth, Virginia Polytechnic Institute, State University. Vehicle Analysis, Simulation Laboratory, C. L. Kirn, and S. S. Wreggit. 1994. Research on Vehicle-based Driver Status/performance Monitoring: Development, Validation, and Refinement of Algorithms for Detection of Driver Drowsiness. U.S. Department of Transportation, National Highway Traffic Safety Administration. https://books.google.de/books?id=7mRiPgAACAAJ
[51]
A. M. Williamson and A. M. Feyer. 2000. Moderate sleep deprivation produces impairments in cognitive and motor performance equivalent to legally prescribed levels of alcohol intoxication. Occupational and environmental medicine 57, 10 (2000), 649–655. https://doi.org/10.1136/oem.57.10.649
[52]
Yan, Zhang, and Gong. 2018. Deep Convolutional Neural Network for Decoding Motor Imagery based Brain Computer Interface. In IEEE International Conference on Signal Processing, Communications and Computing.
[53]
Jun-Juh Yan, Hang-Hong Kuo, Ying-Fan Lin, and Teh-Lu Liao. 2016. Real-Time Driver Drowsiness Detection System Based on PERCLOS and Grayscale Image Processing. In 2016 IEEE International Symposium on Computer, Consumer and Control. Conference Publishing Services, IEEE Computer Society, Los Alamitos, California and Washington and Tokyo, 243–246. https://doi.org/10.1109/IS3C.2016.72
[54]
YouTube. 15.03.2021. WATCH: Elon Musk’s Neuralink Presentation (full working demo). https://www.youtube.com/watch?v=sr8hzF3j2fo
[55]
Zhang, Jian, and Dong. 2017. Classification of EEG signal by WT-CNN model in emotion recognition system. In IEEE 16th International Conference on Cognitive Informatics & Cognitive Computing.
[56]
Aihua Zhang and Fenghua Liu. 2012. Drowsiness detection based on wavelet analysis of ECG and pulse signals. In 2012 5th International Conference on Biomedical Engineering and Informatics, Qianbin Chen, Jun Huan, Yong Xu, Tianqi Zhang, and Lipo Wang (Eds.). IEEE, Piscataway, NJ, 491–495. https://doi.org/10.1109/BMEI.2012.6513058

Index Terms

  1. Deep Learning-based EEG Detection of Mental Alertness States from Drivers under Ethical Aspects
        Index terms have been assigned to the content through auto-classification.

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Other conferences
        ICAAI '21: Proceedings of the 5th International Conference on Advances in Artificial Intelligence
        November 2021
        199 pages
        ISBN:9781450390699
        DOI:10.1145/3505711
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 28 March 2022

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. Automated Artefact Separation
        2. CNN and CNN headway
        3. Driver Drowsiness Detection
        4. Driving Simulator
        5. Ethical Consideration
        6. ICA

        Qualifiers

        • Research-article
        • Research
        • Refereed limited

        Conference

        ICAAI 2021

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • 0
          Total Citations
        • 103
          Total Downloads
        • Downloads (Last 12 months)19
        • Downloads (Last 6 weeks)1
        Reflects downloads up to 11 Feb 2025

        Other Metrics

        Citations

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format.

        HTML Format

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media