Abstract
Mental fatigue is a psychophysiological state that has an intense adverse effect on the quality of life, undermining both the mental and the physical health. As a consequence, detecting this state accurately can be beneficial to delivering prevention and treatment mechanisms. In parallel, advancements in wearable device technologies have reached a maturity level that can support continuous and long-term monitoring of physiological signals accurately and unobtrusively in everyday life. In this paper, a mental fatigue detection methodology is proposed, founded on the use of a wearable consumer device to enable heart rate variability (HRV) analysis and suitable machine learning models to predict the stress state with high accuracy. Even though a lot of studies have attempted to address the same problem in the past by using multiple signals, these approaches are invasive because they require a lot of sensor devices to be attached to the users. The main contributions of this work are three folds: An experimental study with 32 healthy participants demonstrating that mental fatigue caused by cognitive overload can be detected using a wearable commodity device and a single biomarker; Detection models based on eight HRV features which were found to have significant differences after inducing mental fatigue; A methodology, which includes a support vector machine, among other classifiers, and principal component analysis capable to predict cognitive performance degradation in the form of mental fatigue with a high accuracy which can be further improved by applying an ensemble model. The ability to detect mental fatigue unobtrusively and on a regular basis, for example, in workplace environments, could provide awareness on the causes of performance variations which subsequently can navigate improvements on working practices and task planning to prevent accidents or productivity losses.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and materials
The datasets generated during and/or analysed during the current study are not publicly available due to ethical constraints in consideration of participants’ privacy but are available from the corresponding author on reasonable request.
References
Acharya UR, Joseph KP, Kannathal N, Lim CM, Suri JS (2006) Heart rate variability: a review. Med Biol Eng Comput 44(12):1031–1051. https://doi.org/10.1007/s11517-006-0119-0
Athavipach C, Pan-Ngum S, Israsena P (2019) A wearable in-ear EEG device for emotion monitoring. Sensors 19(18):4014. https://doi.org/10.3390/s19184014
Blake M (2015) Development of a bluetooth 4.0 PPG sensor for use in heart rate variability analysis. In: 2015 IEEE International Conference on Consumer Electronics (ICCE), pp 301–304. https://doi.org/10.1109/ICCE.2015.7066423
Boksem MA, Tops M (2008) Mental fatigue: costs and benefits. Brain Res Rev 59(1):125–139. https://doi.org/10.1016/j.brainresrev.2008.07.001
Borghini G, Astolfi L, Vecchiato G, Mattia D, Babiloni F (2014) Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci Biobehav Rev 44:58–75. https://doi.org/10.1016/j.neubiorev.2012.10.003
Castaldo R, Xu W, Melillo P, Pecchia L, Santamaria L, James C (2016) Detection of mental stress due to oral academic examination via ultra-short-term HRV analysis. In: 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp 3805–3808. https://doi.org/10.1109/EMBC.2016.7591557
Chalder T, Berelowitz G, Pawlikowska T, Watts L, Wessely S, Wright D, Wallace EP (1993) Development of a fatigue scale. J Psychosom Res 37(2):147–153. https://doi.org/10.1016/0022-3999(93)90081-P
Chowdhury A, Shankaran R, Kavakli M, Haque MM (2018) Sensor applications and physiological features in drivers’ drowsiness detection: a review. IEEE Sens J 18(8):3055–3067. https://doi.org/10.1109/JSEN.2018.2807245
Cosoli G, Spinsante S, Scalise L (2020) Wrist-worn and chest-strap wearable devices: systematic review on accuracy and metrological characteristics. Measurement 159:107789. https://doi.org/10.1016/j.measurement.2020.107789
Dinh DT, Huynh VN, Sriboonchitta S (2021) Clustering mixed numerical and categorical data with missing values. Inf Sci 571:418–442. https://doi.org/10.1016/j.ins.2021.04.076
Duffy JF, Zitting KM, Czeisler CA (2015) The case for addressing operator fatigue. Rev Hum Factors Ergon 10(1):29–78. https://doi.org/10.1177/1557234X15573949
Duschek S, Muckenthaler M, Werner N, Del Paso GAR (2009) Relationships between features of autonomic cardiovascular control and cognitive performance. Biol Psychol 81(2):110–117. https://doi.org/10.1016/j.biopsycho.2009.03.003
Fairclough SH, Venables L, Tattersall A (2005) The influence of task demand and learning on the psychophysiological response. Int J Psychophysiol 56(2):171–184. https://doi.org/10.1016/j.ijpsycho.2004.11.003
Fan X, Zhou Q, Liu Z, Xie F (2015) Electroencephalogram assessment of mental fatigue in visual search. Biomed Mater Eng 26(s1):S1455–S1463. https://doi.org/10.3233/BME-151444
Fawcett T (2006) An introduction to ROC analysis. Pattern Recognit Lett 27(8):861–874. https://doi.org/10.1016/j.patrec.2005.10.010
Georgiou K, Larentzakis AV, Khamis NN, Alsuhaibani GI, Alaska YA, Giallafos EJ (2018) Can wearable devices accurately measure heart rate variability? A Systematic Review. Folia Med 60(1):7–20. https://doi.org/10.2478/folmed-2018-0012
Goldberger AL, Amaral LA, Glass L et al (2000) PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation 101(23):e215–e220. https://doi.org/10.1161/01.CIR.101.23.e215
Goumopoulos C, Menti E (2019) Stress detection in seniors using biosensors and psychometric tests. Procedia Comput Sci 152:18–27. https://doi.org/10.1016/j.procs.2019.05.022
Haghi M, Thurow K, Stoll R (2017) Wearable devices in medical internet of things: scientific research and commercially available devices. Healthc Inf Res 23(1):4–15. https://doi.org/10.4258/hir.2017.23.1.4
Huang S, Li J, Zhang P, Zhang W (2018) Detection of mental fatigue state with wearable ECG devices. Int J Med Inf 119:39–46. https://doi.org/10.1016/j.ijmedinf.2018.08.010
Kakria P, Tripathi NK, Kitipawang P (2015) A real-time health monitoring system for remote cardiac patients using smartphone and wearable sensors. Int J Telemed Appl 2015:8. https://doi.org/10.1155/2015/373474
Ke DS (2012) Overwork, stroke, and Karoshi-death from overwork. Acta Neurol Taiwan 21(2):54–59
Lam SP, Kao HS, Kao X, Fung MMY, Kao TT (2019) HRV regulation by calligraphic finger-writing and guqin music: a pilot case study. Neuroregulation 6(1):42–51. https://doi.org/10.15540/nr.6.1.42
Laurent F, Valderrama M, Besserve M, Guillard M, Lachaux JP, Martinerie J, Florence G (2013) Multimodal information improves the rapid detection of mental fatigue. Biomed Signal Process Control 8(4):400–408. https://doi.org/10.1016/j.bspc.2013.01.007
Lee KA, Hicks G, Nino-Murcia G (1991) Validity and reliability of a scale to assess fatigue. Psychiatry Res 36(3):291–298. https://doi.org/10.1016/0165-1781(91)90027-M
Lerman SE, Eskin E, Flower DJ et al (2012) Fatigue risk management in the workplace. J Occup Environ Med 54(2):231–258. https://doi.org/10.1097/JOM.0b013e318247a3b0
Liu Y, Lan Z, Khoo HHG, Li KHH, Sourina O, Mueller-Wittig W (2018) EEG-based evaluation of mental fatigue using machine learning algorithms. In: 2018 IEEE International Conference on Cyberworlds (CW), pp 276–279. https://doi.org/10.1109/CW.2018.00056
Malik M, Bigger JT, Camm AJ, Kleiger RE, Malliani A, Moss AJ, Schwartz PJ (1996) Heart rate variability: standards of measurement, physiological interpretation, and clinical use. Eur Heart J 17(3):354–381. https://doi.org/10.1093/oxfordjournals.eurheartj.a014868
Melillo P, Bracale M, Pecchia L (2011a) Nonlinear heart rate variability features for real-life stress detection. Case study: students under stress due to university examination. BioMed Eng OnLine 10(1):96. https://doi.org/10.1186/1475-925X-10-96
Melillo P, Fusco R, Sansone M, Bracale M, Pecchia L (2011b) Discrimination power of long-term heart rate variability measures for chronic heart failure detection. Med Biol Eng Comput 49(1):67–74. https://doi.org/10.1007/s11517-010-0728-5
Murugan S, Selvaraj J, Sahayadhas A (2020) Detection and analysis: driver state with electrocardiogram (ECG). Phys Eng Sci Med 43(2):525–537. https://doi.org/10.1007/s13246-020-00853-8
Noreen EW (1989) Computer-intensive methods for testing hypotheses. Wiley, New York
Orsila R, Virtanen M, Luukkaala T et al (2008) Perceived mental stress and reactions in heart rate variability—a pilot study among employees of an electronics company. Int J Occup Saf Ergon 14(3):275–283. https://doi.org/10.1080/10803548.2008.11076767
Patel M, Lal SK, Kavanagh D, Rossiter P (2011) Applying neural network analysis on heart rate variability data to assess driver fatigue. Expert Syst Appl 38(6):7235–7242. https://doi.org/10.1016/j.eswa.2010.12.028
Ramos G, Vaz JR, Mendonça GV, Pezarat-Correia P, Rodrigues J, Alfaras M, Gamboa H (2020) Fatigue evaluation through machine learning and a global fatigue descriptor. J Healthc Eng 2020:6484129. https://doi.org/10.1155/2020/6484129
Rand J, Hoover A, Fishel S, Moss J, Pappas J, Muth E (2007) Real-time correction of heart interbeat intervals. IEEE Trans Biomed Eng 54(5):946–950. https://doi.org/10.1109/TBME.2007.893491
Sabeti M, Boostani R, Rastgar K (2018) How mental fatigue affects the neural sources of P300 component? J Integr Neurosci 17(1):93–111. https://doi.org/10.3233/JIN-170040
Salahuddin L, Cho J, Jeong MG, Kim D (2007) Ultra short term analysis of heart rate variability for monitoring mental stress in mobile settings. In: 2007 29th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp 4656–4659. https://doi.org/10.1109/IEMBS.2007.4353378
Shahid A, Wilkinson K, Marcu S, Shapiro CM (2011) Chalder fatigue scale. In: Shahid A et al (eds) STOP, THAT and One Hundred Other Sleep Scales. Springer, New York, pp 97–98
Sikander G, Anwar S (2018) Driver fatigue detection systems: a review. IEEE Trans Intell Transp Syst 20(6):2339–2352. https://doi.org/10.1109/TITS.2018.2868499
Tarvainen MP, Niskanen JP, Lipponen JA, Ranta-Aho PO, Karjalainen PA (2014) Kubios HRV–heart rate variability analysis software. Comput Methods Programs Biomed 113(1):210–220. https://doi.org/10.1016/j.cmpb.2013.07.024
Tsunoda K, Chiba A, Yoshida K et al (2017) Predicting changes in cognitive performance using heart rate variability. IEICE Trans Inf Syst 100(10):2411–2419. https://doi.org/10.1587/transinf.2016OFP0002
Tu L, Huang J, Bi C, Xing G (2017) Fitbeat: a lightweight system for accurate heart rate measurement during exercise. In: 2017 IEEE International Conference on Smart Computing (SMARTCOMP), pp 1–8. https://doi.org/10.1109/SMARTCOMP.2017.7947009
Urigüen JA, Garcia-Zapirain B (2015) EEG artifact removal—state-of-the-art and guidelines. J Neural Eng 12(3):031001. https://doi.org/10.1088/1741-2560/12/3/031001
Urrila AS, Stenuit P, Huhdankoski O, Kerkhofs M, Porkka-Heiskanen T (2007) Psychomotor vigilance task performance during total sleep deprivation in young and postmenopausal women. Behav Brain Res 180(1):42–47. https://doi.org/10.1016/j.bbr.2007.02.019
Van der Linden D, Frese M, Meijman TF (2003) Mental fatigue and the control of cognitive processes: effects on perseveration and planning. Acta Psychol 113(1):45–65. https://doi.org/10.1016/S0001-6918(02)00150-6
Wall ME, Rechtsteiner A, Rocha LM (2003) Singular value decomposition and principal component analysis. In: Berrar DP, Dubitzky W, Granzow M (eds) A practical approach to microarray data analysis. Kluwer Academic Publishers, New York, pp 91–109
Wang H, Dragomir A, Abbasi NI, Li J, Thakor NV, Bezerianos A (2018) A novel real-time driving fatigue detection system based on wireless dry EEG. Cogn Neurodyn 12(4):365–376. https://doi.org/10.1007/s11571-018-9481-5
Zanetti M, Mizumoto T, Faes L et al (2021) Multilevel assessment of mental stress via network physiology paradigm using consumer wearable devices. J Ambient Intell Human Comput 12:4409–4418. https://doi.org/10.1007/s12652-019-01571-0
Zhao C, Zhao M, Liu J, Zheng C (2012) Electroencephalogram and electrocardiograph assessment of mental fatigue in a driving simulator. Accid Anal Prev 45:83–90. https://doi.org/10.1016/j.aap.2011.11.019
Zijlstra F (1993). Efficiency in work behavior. A design approach for modern tools. Dissertation, Delft University of Technology, Delft, the Netherlands
Acknowledgements
The authors would like to thank Nikolaos G. Stergiopoulos from the University of the Aegean for its support in implementing the experimental protocol to collect the data and the volunteers that took part in the study.
Funding
Not applicable.
Author information
Authors and Affiliations
Contributions
CG: research idea and design, investigation, data processing, writing, supervision. NP: machine learning methodology, data curation, writing.
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Goumopoulos, C., Potha, N. Mental fatigue detection using a wearable commodity device and machine learning. J Ambient Intell Human Comput 14, 10103–10121 (2023). https://doi.org/10.1007/s12652-021-03674-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-021-03674-z