Detection of mental fatigue state with wearable ECG devices

Highlights

• This paper aims to investigate the possibility of using wearable smart electrocardiogram (ECG) devices to detect the mental fatigue state.

• In total, 35 healthy participants were recruited from a public university in East China. Throughout the entire experiment, each participant wore a wearable device that was further linked to a smartphone to upload the data based on Bluetooth transmission.

• Eight heart rate variability (HRV) indicators were collected at intervals of 5 min throughout the entire experiment.

• Four algorithms, support vector machine (SVM), K-nearest neighbor (KNN), naïve Bayes (NB), and logistic regression (LR), were used to build classifiers that automatically detected the fatigue state. The best performance was achieved by KNN.

• The NN.mean (mean of normal to normal interval), PNN50 (the proportion of NN50 divided by total number of NNs), TP (total spectral power), and LF (low frequency from 0.04 Hz to 0.15 Hz) were the most important HRV indicators for mental fatigue detection.

Abstract

Overwork-related disorders, such as cerebrovascular/cardiovascular diseases (CCVD) and mental disorders due to overwork, are a major occupational and public health issue worldwide, particularly in East Asian countries. Since wearable smart devices are inexpensive, convenient, popular and widely available today, we were interested in investigating the possibility of using wearable smart electrocardiogram (ECG) devices to detect the mental fatigue state. In total, 35 healthy participants were recruited from a public university in East China. Throughout the entire experiment, each participant wore a wearable device that was further linked to a smartphone to upload the data based on Bluetooth transmission. To manipulate the fatigue state, each participant was asked to finish a quiz, which lasted for approximately 80 min, with 30 logical referential and computing problems and 25 memory tests. Eight heart rate variability (HRV) indicators namely NN.mean (mean of normal to normal interval), rMSSD (root mean square of successive differences), PNN50 (the proportion of NN50 divided by total number of NNs), TP (total spectral power), HF (high frequency from 0.15 Hz to 0.4 Hz), LF (low frequency from 0.04 Hz to 0.15 Hz), VLF (very low frequency from 0.0033 Hz to 0.04 Hz) and the LF/HF ratio were collected at intervals of 5 min throughout the entire experiment. After the feature selection was performed, six indicators remained for further analysis, which were the NN.mean, rMSSD, PNN50, TP, LF, and VLF. Four algorithms, support vector machine (SVM), K-nearest neighbor (KNN), naïve Bayes (NB), and logistic regression (LR), were used to build classifiers that automatically detected the fatigue state. The best performance was achieved by KNN, which had a CV accuracy of 75.5%. The NN.mean, PNN50, TP and LF were the most important HRV indicators for mental fatigue detection. KNN performed the best among the four algorithms and had an average CV accuracy of 65.37% for all of the possible feature combinations.

Introduction

Overwork-related disorders, such as cerebrovascular/cardiovascular diseases (CCVD) and mental disorders due to overwork, are a major occupational and public health issue worldwide, particularly in East Asian countries [1]. Japan's work culture is so intense that people in the 1970s invented a word, “karoshi,” which translates to "death by overwork." One example of an employee’s death determined to be karoshi was 31-year-old journalist Miwa Sado [2]. She reportedly logged 159 h of overtime in one month at the news network NHK before dying of heart failure in July 2013. In Japan, the government estimates that 200 people die from overwork every year because of heart attacks or cerebral hemorrhages due to long hours spent at the workplace [3]. However, this estimation does not include deaths from mental depression or suicides. If these deaths were included, the number of work-related deaths would dramatically increase. From January 2010 to March 2015, 368 suicides in Japan, from 352 men and 16 women, were deemed as being karoshi [4]. Overwork is also a serious problem in China. According to the China Youth Daily, approximately 600,000 Chinese people each year die from working too hard [5]. In April 2015, China Radio International reported a toll of 1600 deaths from overwork every day in China [5].

Since overwork is a subjective feeling that varies among people, it is very difficult to measure overwork by simply counting working hours. Therefore, mental fatigue is a better way for detecting potential overwork. Mental fatigue is a subjective feeling of mental tiredness. It is a transient decrease in maximal cognitive performance resulting from prolonged periods of cognitive activity (long working hours, shift work, stressful work, anxiety, etc.) [6]. Research proves that stroke and death by karoshi have a strong association with mental fatigue caused by overwork [7]. In addition, intensive work increases the risks for cardiovascular diseases [8], diabetes [9] and cancer [10]. In addition to inducing damage to human health, mental fatigue also has a variety of effects that impair memory, judgement, decision-making and emotion management [11]. Routinely working long hours leads to stress and strain, which, in turn, can lead to higher accident levels, greater absenteeism, and reduced productivity [12,13]. Therefore, analyzing wearable devices that can monitor a worker’s mental fatigue in a real-time manner and prompt the user to take a rest or leave the office is highly imperative.

However, mental fatigue is elusive and difficult to measure in practice. Extant measures for mental fatigue can be divided into two categories: subjective self-report measures and objective performance measures. Subjective self-report measures require subjects to evaluate their level of mental fatigue typically by a questionnaire [[14], [15], [16]]. Some scales simply involve questions about the participant’s perceptions of experienced fatigue or sleepiness at the moment, such as in the Stanford Sleepiness Scale (SSS), Chalder Fatigue Scale (CFS) and Fatigue Severity Scale (FSS). Meanwhile, other scales assess the participant’s fatigue level by setting detailed scenarios, such as in the Epworth Sleepiness Scale (ESS) [17] and Specific Fatigue Scale (SFS) [18]. Objective performance measures design many mental tasks to assess the subject’s performance of brain function. Some tasks measure the subject’s reaction time, memory and decision-making performance, such as in the Psychomotor Vigilance Task (PVT) [19]. Meanwhile, other tasks assess the subject’s maintenance of wakefulness and resistance of sleepiness, such as in the Multiple Sleep Latency Test (MSLT) and Maintenance of Wakefulness Test (MWT) [20].

The two measures mentioned above are intrusive in nature because the users must stop their work at hand to finish the questionnaires or mental tasks. Therefore, they cannot be used to monitor mental fatigue without interrupting normal daily life. The equipment approach allows mental fatigue to be measured while the daily work is still going on. For example, the electroencephalograph (EEG) is the most widely used equipment for measuring mental fatigue [21,22]. Some other researchers have proposed a variety of EEG-based algorithms that detect fatigue based on a spectrum analysis [[23], [24], [25]]. Four frequency components obtained from the original EEG signal have been proven to be useful for detecting a subject’s brain state, namely, delta ($\delta$) ( ± 0 Hz to 4 Hz), theta ($\theta$) (4–8 Hz), alpha ($\alpha$) (8–13 Hz), and beta ($\beta$) (13–20 Hz) [25]. For a driving fatigue state detection, a group of Australian researchers developed an EEG-based driver-fatigue countermeasure system to monitor driver fatigue [23,24,26]. However, the devices used for EEG-based fatigue detection are usually heavy and large, which is inconvenient for applying to daily life, especially when used in an office space or at home. Since real-time monitoring is very important for helping the users to remain in a healthy state, a convenient wearable device that can ubiquitously monitor the mental fatigue condition is highly desirable [27].

A recent trend in health information technology is the growing popularity of wearable smart devices, such as smart bracelets and wearable ECGs, which makes real-time and distant health monitoring and management possible. According to Gartner’s investigation, in 2016, a total of 265.9 M wearable devices were sold. The global market for wearable electronic devices is forecasted to be worth more than $50 billion in 2021 [28]. Therefore, wearable smart devices are becoming increasingly more widely available. We are interested in investigating the possibility of using wearable smart devices to measure mental fatigue.

A number of smart sensors used to continuously obtain physiological parameters, such as an electrocardiogram (ECG), heart rate and blood pressure, with Bluetooth wireless transmission for health monitoring have been developed [[29], [30], [31], [32]]. Among all of these wearable devices, a wearable ECG is a promising one for real-time mental fatigue monitoring. The device provides a relatively easy way to obtain ECG signals compared with that for complex EEG devices. Since the connection between the autonomic nervous system (ANS) and heart rhythms was discovered a long time ago [33], it is possible to measure the mental fatigue status with ECG signals. Therefore, the research question of this study can be interpreted as follows:

RQ: Can mental fatigue be detected by wearable ECG smart devices? If so, how and with what effect can this be achieved?

To answer this research question, an experiment was designed and executed in this study to test the possibility of measuring mental fatigue with a wearable ECG device. In total, 35 subjects were recruited from a public university in East China. An experiment was carried out to collect self-reported mental fatigue and ECG data.