Skip to main content
SpringerLink
Log in

A Hybrid Vigilance Monitoring Study for Mental Fatigue and Its Neural Activities

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Mental fatigue causes many casualties and economic losses from unexpected accidents. Electroencephalograph (EEG) signals are most commonly used for vigilance estimation. In this paper, we report a novel hybrid vigilance monitoring and warning system based on EEG and eye movement signals to detect mental drowsiness. This system collects eye movement information to quickly detect unsafe driving behavior and also gives real-time warning of driving fatigue by monitoring EEG activity. It also uses Fisher score electrode analysis to locate the cortical regions involved in vigilance and reduces the number of channels required. Fewer channels make the integration of vigilance monitoring technologies easier to implement and use as a vehicle safety aid. The self-adaptive system can provide various online monitoring and warning strategies for adapting to different individual physiological situations and complex external environments. For new users, the non-model module can be used for online monitoring without prior training and analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Brown ID. Driver fatigue. Hum Factors J Hum Factors Ergon Soc. 1994;36(2):298–314.

    CAS  Google Scholar 

  2. Lal SKL, Craig A. A critical review of the psychophysiology of driver fatigue. Biol Psychol. 2001;55(3):173–94.

    Article  CAS  PubMed  Google Scholar 

  3. Wang Q, Yang J, Ren M, Zheng Y. Driver fatigue detection: a survey. In: The sixth world congress on intelligent control and automation, 2006 (WCICA 2006), vol 2. IEEE; 2006. p. 8587–91.

  4. Philip P, Sagaspe P, Moore N, Taillard J, Charles A, Guilleminault C, Bioulac B. Fatigue, sleep restriction and driving performance. Accid Anal Prev. 2005;37(3):473–8.

    Article  PubMed  Google Scholar 

  5. Thiffault P, Bergeron J. Monotony of road environment and driver fatigue: a simulator study. Accid Anal Prev. 2003;35(3):381–91.

    Article  PubMed  Google Scholar 

  6. Lin CT, Wu RC, Jung TP, Liang SF, Huang TY. Estimating driving performance based on EEG spectrum analysis. EURASIP J Appl Signal Process. 2005;2005:3174.

    Article  Google Scholar 

  7. Lin CT, Wu RC, Liang SF, Chao WH, Chen YJ, Jung TP. EEG-based drowsiness estimation for safety driving using independent component analysis. IEEE Trans Circuits Syst. 2005;52:12.

    Google Scholar 

  8. Lal SKL, Craig A, Boord P, Kirkup L, Nguyen H. Development of an algorithm for an EEG-based driver fatigue countermeasure. J Saf Res. 2003;34(3):321–8.

    Article  Google Scholar 

  9. Makeig S, Jung TP. Changes in alertness are a principal component of variance in the EEG spectrum. Neuroreport. 1995;7(1):213–6.

    Article  CAS  PubMed  Google Scholar 

  10. Berka C, Levendowski DJ, Westbrook P, Davis G, Lumicao MN, Ramsey C, Petrovic MM, Zivkovic VT, Olmstead RE. Implementation of a closed-loop real-time EEG-based drowsiness detection system: effects of feedback alarms on performance in a driving simulator. In: Proceedings of the 11th annual conference on human–computer interaction. 2005.

  11. Pal NR, Chuang CY, Ko LW, Chao CF, Jung TP, Liang SF, Lin CT. EEG-based subject-and session-independent drowsiness detection: an unsupervised approach. EURASIP J Adv Signal Process. 2008;2008:192.

    Article  Google Scholar 

  12. Heitmann A, Guttkuhn R, Aguirre A, Trutschel U, Moore-Ede M. Technologies for the monitoring and prevention of driver fatigue. In: Proceedings of the first international driving symposium on human factors in driver assessment, training and vehicle design. 2001. p. 81–6.

  13. Johns MW, Tucker AJ, Chapman RJ, Michael NJ, Beale CA. A new scale of drowsiness based on multiple characteristics of blinks: the Johns Drowsiness Scale. Alcohol. 2006;9:10.

    Google Scholar 

  14. Caffier PP, Erdmann U, Ullsperger P. Experimental evaluation of eye-blink parameters as a drowsiness measure. Eur J Appl Physiol. 2003;89(3):319–25.

    Article  PubMed  Google Scholar 

  15. Morimoto CH, Mimica MRM. Eye gaze tracking techniques for interactive applications. Comput Vis Image Underst. 2005;98(1):4–24.

    Article  Google Scholar 

  16. Ji Q, Yang X. Real-time eye, gaze, and face pose tracking for monitoring driver vigilance. Real-Time Imaging. 2002;8(5):357–77.

    Article  Google Scholar 

  17. Saradadevi M, Bajaj P. Driver fatigue detection using mouth and yawning analysis. IJCSNS. 2008;8(6):183.

    Google Scholar 

  18. Sommer D, Golz M. Evaluation of PERCLOS based current fatigue monitoring technologies. In: Engineering in Medicine and Biology Society (EMBC), 2010 annual international conference of the IEEE. 2010. p. 4456–9.

  19. Bergasa LM, Nuevo J, Sotelo MA, Barea R, Lopez ME. Real-time system for monitoring driver vigilance. IEEE Trans Intell Transp Syst. 2006;7(1):63–77.

    Article  Google Scholar 

  20. Ji Q, Zhu Z, Lan P. Real-time nonintrusive monitoring and prediction of driver fatigue. IEEE Trans Veh Technol. 2004;53(4):1052–68.

    Article  Google Scholar 

  21. Tran Y, Craig A, Wijesuriya N, Nguyen H. Improving classification rates for use in fatigue countermeasure devices using brain activity. In: Engineering in Medicine and Biology Society (EMBC), 2010 annual international conference of the IEEE. 2010. p. 4460–3.

  22. Egelund N. Spectral analysis of heart rate variability as an indicator of driver fatigue. Ergonomics. 1982;25(7):663–72.

    Article  CAS  PubMed  Google Scholar 

  23. Jorna P. Spectral analysis of heart rate and psychological state: a review of its validity as a workload index. Biol Psychol. 1992;34(2–3):237–57.

    Article  CAS  PubMed  Google Scholar 

  24. Allen AP, Jacob TJ, Smith AP. Effects and after-effects of chewing gum on vigilance, heart rate, EEG and mood. Physiol Behav. 2014;133:244–51.

    Article  CAS  PubMed  Google Scholar 

  25. Lenne MG, Triggs TJ, Redman JR. Time of day variations in driving performance. Accid Anal Prev. 1997;29(4):431–7.

    Article  CAS  PubMed  Google Scholar 

  26. Yang G, Lin Y, Bhattacharya P. A driver fatigue recognition model based on information fusion and dynamic Bayesian network. Inf Sci. 2010;180(10):1942–54.

    Article  Google Scholar 

  27. Li F, Wang XW, Lu BL. Detection of driving fatigue based on grip force on steering wheel with wavelet transformation and support vector machine. In: Lee M, Hirose A, Hou Z-G, Kil RM, editors. Neural information processing. Daegu, Korea: Springer; 2013. p. 141–8.

    Chapter  Google Scholar 

  28. Murata A, Uetake A, Takasawa Y. Evaluation of mental fatigue using feature parameter extracted from event-related potential. Int J Ind Ergon. 2005;35(8):761–70.

    Article  Google Scholar 

  29. Simon M, Schmidt EA, Kincses WE, Fritzsche M, Bruns A, Aufmuth C, Bogdan M, Rosenstiel W, Schrauf M. EEG alpha spindle measures as indicators of driver fatigue under real traffic conditions. Clin Neurophysiol. 2011;122(6):1168–78.

    Article  PubMed  Google Scholar 

  30. Lal SKL, Craig A. Reproducibility of the spectral components of the electroencephalogram during driver fatigue. Int J Psychophysiol. 2005;55(2):137–43.

    Article  PubMed  Google Scholar 

  31. Schmidt FM, Schönherr J, Sander C, Kirkby KC, Hegerl U, Himmerich H. Applying EEG-based vigilance measurement in a case of adult attention deficit hyperactivity disorder. Int J Neuropsychopharmacol. 2013;16(05):1169–71.

    Article  PubMed  Google Scholar 

  32. Jödicke J, Olbrich S, Sander C, Minkwitz J, Chittka T, Himmerich H, Hegerl U. Separation of low-voltage EEG-activity during mental activation from that during transition to drowsiness. Brain Topogr. 2013;26(4):538–46.

    Article  PubMed  Google Scholar 

  33. Jalili M. Multivariate synchronization analysis of brain electroencephalography signals: a review of two methods. Cogn Comput. 2013;7(1):3–10.

    Article  Google Scholar 

  34. Xia B, Li X, Xie H, Yang WL, Li J, He LH. Asynchronous brain–computer interface based on steady-state visual-evoked potential. Cogn Comput. 2013;5(2):243–51.

    Article  Google Scholar 

  35. Jap BT, Lal S, Fischer P. Comparing combinations of EEG activity in train drivers during monotonous driving. Expert Syst Appl. 2010;38(1):996–1003.

    Article  Google Scholar 

  36. Kato Y, Endo H, Kizuka T. Mental fatigue and impaired response processes: event-related brain potentials in a go/nogo task. Int J Psychophysiol. 2009;72(2):204–11.

    Article  PubMed  Google Scholar 

  37. Wester AE, Böcker KBE, Volkerts ER, Verster JC, Kenemans JL. Event-related potentials and secondary task performance during simulated driving. Accid Anal Prev. 2008;40(1):1–7.

    Article  CAS  PubMed  Google Scholar 

  38. Lin CT, Ko LW, Chung IF, Huang TY, Chen CY, Jung TP, Liang SF. Adaptive EEG-based alertness estimation system by using ICA-based fuzzy neural networks. IEEE Trans Circuits Syst I Regul Pap. 2006;53(11):2469.

    Article  Google Scholar 

  39. Lin CT, Chen CY, Huang TY, Chiu TT, Ko LW, Liang SF, Hsieh HY, Hsu SH, Duann JR. Development of wireless brain computer interface with embedded multitask scheduling and its application on real-time driver’s drowsiness detection and warning. IEEE Trans Biomed Eng. 2008;55:1582–91.

    Article  PubMed  Google Scholar 

  40. Lin CT, Chuang CH, Huang CS, Tsai SF, Lu SW, Chen YH, Ko LW. Wireless and wearable EEG system for evaluating driver vigilance. IEEE Trans Biomed Circuits Syst. 2014;8(2):165–76.

    Article  PubMed  Google Scholar 

  41. Lin CT, Ko LW, Shen TK. Computational intelligent brain computer interaction and its applications on driving cognition. IEEE Comput Intell Mag. 2009;4(4):32–46.

    Article  Google Scholar 

  42. Shi LC, Lu BL. EEG-based vigilance estimation using extreme learning machines. Neurocomputing. 2013;102:135–43.

    Article  Google Scholar 

  43. Hu SY, Zheng GT, Björn P. Driver fatigue detection from electroencephalogram spectrum after electrooculography artefact removal. IET Intell Transp Syst. 2013;7(1):105–13.

    Article  Google Scholar 

  44. Shou GF, Ding L. Ongoing EEG oscillatory dynamics suggesting evolution of mental fatigue in a color-word matching stroop task. In: 2013 6th international IEEE/EMBS conference on neural engineering (NER). 2013. p. 1339–42.

  45. Bekhtereva V, Sander C, Forschack N, Olbrich S, Hegerl U, Müller MM. Effects of EEG-vigilance regulation patterns on early perceptual processes in human visual cortex. Clin Neurophysiol. 2014;125(1):98–107.

    Article  PubMed  Google Scholar 

  46. Gu J, Lu HT, Lu BL. An integrated gaussian mixture model to estimate vigilance level based on EEG recordings. Neurocomputing. 2014;129:107–13.

    Article  Google Scholar 

  47. Lin CT, Lin KL, Ko LW, Liang SF, Kuo BC, Chung IF. Nonparametric single-trial EEG feature extraction and classification of driver’s cognitive responses. EURASIP J Adv Signal Process. 2008;20:2008.

    Google Scholar 

  48. Yang G, Lin Y, Bhattacharya P. A driver fatigue recognition model using fusion of multiple features. In: IEEE international conference on systems, man and cybernetics, 2005, vol 2. 2005. p. 1777–84.

  49. Xu YF, Zeng JH, Sun YR. Head pose recovery using 3D cross model. In: 2012 4th international conference on intelligent human–machine systems and cybernetics (IHMSC), vol 2. 2012. p. 63–6.

  50. Cao L, Li J, Sun YR, Zhu HP, Yan CG. EEG-based vigilance analysis by using fisher score and pca algorithm. In: Proceedings of the 2010 IEEE international conference on progress in informatics and computing, PIC 2010, vol 1. 2010. p. 175–9.

  51. Gu NJ, Lu HT, Lu BL. An integrated Gaussian mixture model to estimate vigilance level based on EEG recordings. Neurocomputing. 2014;129:107–13.

    Article  Google Scholar 

  52. Sauvet F, Bougard C, Coroenne M, Lely L, Van Beers P, Elbaz M, Guillard M, Leger D, Chennaoui M. In flight automatic detection of vigilance states using a single EEG channel. IEEE Trans Biomed Eng. 2014;61(12):2840–7.

    Article  CAS  PubMed  Google Scholar 

  53. Lin CT, Chuang CH, Huang CS, Chen YH, Ko LW. Real-time assessment of vigilance level using an innovative Mindo4 wireless EEG system. In: 2013 IEEE international symposium on circuits and systems (ISCAS), 2013. p. 1528–31.

Download references

Author information

Authors and Affiliations

  1. Tongji University, Shanghai, 201804, China

    Lei Cao, Jie Li, Yifei Xu, Huaping Zhu & Changjun Jiang

Authors
  1. Lei Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jie Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yifei Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huaping Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Changjun Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Changjun Jiang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, L., Li, J., Xu, Y. et al. A Hybrid Vigilance Monitoring Study for Mental Fatigue and Its Neural Activities. Cogn Comput 8, 228–236 (2016). https://doi.org/10.1007/s12559-015-9351-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-015-9351-y

Keywords

Search

Navigation