Research PaperOptimal imaging of multi-channel EEG features based on a novel clustering technique for driver fatigue detection
Introduction
Fatigue decreases drivers’ ability to operate vehicles safely and reduces their alertness level. It is one of the most important contributors to road accidents [1,2]. According to a report from National Highway Traffic Safety Administration (NHTSA), the accidents caused by driver fatigue make up close to 16.5 % of fatal crashes, but more than 1 in 4 drivers (29.4 %) reported having driving experience when they were so tired that they had a hard time keeping their eyes open in the past 30 days [3]. Thus, how to quantify, assess, and mitigate driver fatigue based on objective measurements, is an important challenge for traffic safety research [4,5]. Electrophysiology-based detection extracts the non-visible characteristics including heart rate variability, electrooculogram (EOG), electromyogram (EMG), and electroencephalogram (EEG), as well as other physiological indexes, reflecting the change of the physiological states directly and reliably [1,[6], [7], [8]]. In particular, EEG, which directly reflects the activities of the human brain is sensitive to fluctuations in alertness and has been applied to predicting performance degradation in a prolonged driving task [9].
As a physiological process which is gradual and accumulative, mental fatigue usually involves a number of different stages from the alertness and vigor stage to the tiredness and weakness state [4]. Correspondingly, its promising indicator, EEG, has demonstrated stage changes with state-specific and frequency-specific topographical differences in the fatigue transition [10,11]. According to the EEG differences, driver fatigue can be effectively identified by classification algorithms to make timely warning instructions. Currently, most of the existing methods detect driver fatigue based on the differences in the time and frequency domains. The detection accuracy of state-of-the-art techniques has exceeded 90 %. In the frequency domain, previous studies have shown that increasing drowsiness is typically associated with progressive changes in rhythmic activities, such as theta (4–8 Hz), alpha (8–13 Hz), or beta (13–30 Hz) activity or their combinations (e.g. (theta + alpha)/beta) [8,12]. Hence, these spectral features have been extracted to develop driver fatigue detection systems. For example, Li et al. [13] calculated 12 types of power spectra combinations of the rhythmic activities and fed them into the linear regression modal, which achieved an accuracy of 91.5 % for fatigue detection. At present, alpha-based analysis is a relatively most efficient method for detecting driver fatigue in frequency domain. The power of alpha oscillations predominantly in the central and posterior brain regions (parietal-occipital) is generally increased when the subjects are fatigued or tired [6,12]. According to the burst of alpha activity [8], Lawhern et al. [14] developed a discounted autoregressive (DAR) method just depending on the statistical properties of the alpha frequency band and achieved approximately 95 % accuracy.
Since the degraded performance of fatigue is involved in a variety of brain activity changes, the EEG complexity that can be quantified by different entropy estimators has been regarded as another prominent feature [5,15,16]. Through entropy-based feature extraction, the system realized the nonlinear estimation of the dynamical EEG activity during driving [5]. The typical entropy estimators used in fatigue detection system are fuzzy entropy (FE) [17] and wavelet entropy (WE) [18]. On the basis of FZ analysis, Luo et al. [19] introduced an adaptive scaling factor and proposed adaptive multi-scale FZ. It increased the detection accuracy to 95.4 %. In [5], the researchers explored WE-based real-time feature extraction of driver fatigue by using sliding window and analyzed the fusion effects of WE, approximate entropy (ApEn), sample entropy (SampEn). When these entropy features were applied to the multilayer perceptron (MLP) neural network, it achieved 96.5 % accuracy.
In accordance with the ten-twenty international standards [20], EEG signals are recorded from multiple electrodes attached to the scalp with a fixed spatial arrangement. Among the fatigue detection methods mentioned above, the EEG analyses in the time and frequency domains may neglect the valuable correlation information of the signals from different electrode positions, which can be concretely interpreted as a spatial domain.
As pointed out in the literature [21], a fatigued driver may fail to functionally and fast enough engage multi-perceptual and processing functions (e.g. attention, reasoning, decision making, sensorimotor coordination, and visual processing), resulting in a series of risky driving behaviors. The coherence and interaction between distinct brain resources intuitively appear to be important for these functions. As a technological advancement in neuroscience, functional brain network analysis, measuring coupling of the functional activities in different brain regions, can provide richer information about human cognition than simpler univariate approaches [22,23]. Several studies have reported that the functional brain networks become more integrated during task performance in comparison with the resting state, but linearly decline with ongoing time-on-task [23,24]. In the prolonged visuomotor vigilance task, Gaggioni et al. [25] suggested that decreased propagation of EEG response evoked by transcranial magnetic stimulation within the fronto-parietal cortex was related to the failure of increased vigilance. Under the simulated driving condition, Kong et al. [26] also revealed the degraded performance of small-world features of brain networks under fatigue state, providing further support for the presence of a reshaped global topology in connectivity networks when drivers shifted from the alert to the drowsy stage. Zhao et al. [27] attributed the shift to a more economic but less efficient configuration, or an inability to retrieve these resources related to mental fatigue.
However, when the spatial connectivity analysis are directly applied to fatigue detection, it may encounter challenges, because of signal mixing. Signal mixing also translates to volume conduction in the case of EEG recordings. This problem is caused by that the activity of any single neuronal source is detected by a spatially widespread group of electrodes. Therefore, the brain networks may contain spurious interactions [28,29]. It is difficult to find the related electrodes which are the most sensitive to the degraded performance to optimize fatigue detection.
In this context, we propose a novel EEG signal analysis approach based on clustering on brain networks (CBNs) to get more reliable and sensitive information from both the spatial and temporal dimensions to detect driver fatigue. Our hypothesis is that the introduced spatial information through CBNs can further improve the detection accuracy. The entire framework and its application are shown in Fig. 1, Fig. 2. In order to alleviate the problem of signal mixing, connection cluster is considered as the basic unit of connectivity analysis in CBNs. A connection cluster is defined as a tree composed of a node and the connections linking to it so that we can keep the effective information on fewer significant nodes with the similar links after clustering. Then, the EEG nonlinear dynamical features are extracted on the significant nodes (electrodes). Since the features on the significant nodes have consistent staged changes, the corresponding feature matrices are converted into images and we employ the image edge detection method to distinguish different stages of fatigue.
This paper is organized as follows. In Section 2, we describe the experimental details and EEG data preprocessing (see Sections 2.1 and 2.2), elaborate CBNs applied to suppress signal mixing to find significant nodes (see Section 2.3), add temporal feature extraction on the significant nodes to form imaging feature matrices (see Section 2.4), and illustrate two-dimensional pattern recognition to distinguish different stages of fatigue (see Section 2.5). Results of the study are presented in Section 3 and discussed in 4. Finally, the paper concludes in Section 5.
Section snippets
Experiments and data
This study was reviewed and approved by Ethics Committee, Dalian University of Technology (protocol number: 2018−040). Written informed consents were obtained from all participants before the experiments. Sixteen healthy right-handed subjects (eight males and eight females, age range: 20–35) who had driving licenses were recruited to participate in the simulated driving experiments. All the subjects had normal intelligence and no mental disorders or sleep problems. They have the habit of taking
Clustering on brain networks
Considering fatigue’s impact on the functional interaction, the brain networks in three temporal stages of driving were constructed to extract the spatial features. Fig. 7(a) shows the spatial connectivity averaged across the subjects at the stages A (0 h), B (0.5 h), and C (1 h). The strengths of the edges were represented by the thickness of the edges (the larger the strength the thicker the line). In order to make the results more obvious, all the edge strengths were magnified up to 2 times.
Discussion
The effects of fatigue are multifaceted [40]. It can bring the EEG changes in time-frequency-spatial domain. The results of the present study confirm the detection accuracy of fatigue can be further improved by introducing effective spatial information. The spatial information is extracted by CBNs.
Due to the issues such as signal mixing, the spatial information is vulnerable to interference, which needs to be processed. Especially in the prolonged driving task, the fatigue-related information
Conclusion
In this paper, a new EEG imaging method for automated detection and identification of driver fatigue was proposed. The nonlinear dynamical features of EEG from spatially optimized nodes extracted by CBNs were transformed into spatio-temporal images to extract reliably critical information of fatigue. The PCNN-based image processing method was utilized to recognize the two-dimensional fatigue pattern. The results showed that the temporal features from the spatially optimized nodes keep fatigue
CRediT authorship contribution statement
Chi Zhang: Conceptualization, Methodology, Software, Visualization, Writing - original draft, Writing - review & editing. Lina Sun: Conceptualization, Writing - original draft, Writing - review & editing. Fengyu Cong: Supervision. Tuomo Kujala: Resources, Writing - review & editing, Supervision. Tapani Ristaniemi: Conceptualization, Supervision. Tiina Parviainen: Writing - review & editing, Supervision.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (grant number 61703069), the National Foundation in China (No. JCKY2019110B009), and the Fundamental Research Funds for the Central Universities [grant number DUT18RC(4)035]. It is also to memorize Prof. Tapani Ristaniemi for his great help to Fengyu Cong and Chi Zhang.
References (48)
- et al.
Assessment of early onset of driver fatigue using multimodal fatigue measures in a static simulator
Appl. Ergon.
(2014) - et al.
EEG signal analysis for the assessment and quantification of driver’s fatigue
Transp. Res. Part F Traffic Psychol. Behav.
(2010) - et al.
Dynamic driver fatigue detection using hidden Markov model in real driving condition
Expert Syst. Appl.
(2016) - et al.
EEG alpha spindle measures as indicators of driver fatigue under real traffic conditions
Clin. Neurophysiol.
(2011) - et al.
Antero-posterior EEG changes during the wakefulness–sleep transition
Clin. Neurophysiol.
(2001) - et al.
EEG index for control operators’ mental fatigue monitoring using interactions between brain regions
Expert Syst. Appl.
(2016) - et al.
Evaluation of driver fatigue on two channels of EEG data
Neurosci. Lett.
(2012) - et al.
Automatic detection of alertness/drowsiness from physiological signals using wavelet-based nonlinear features and machine learning
Expert Syst. Appl.
(2015) - et al.
Research on fatigue driving detection using forehead EEG based on adaptive multi-scale entropy
Biomed. Signal Process. Control
(2019) - et al.
An EEG-based perceptual function integration network for application to drowsy driving
Knowledge Based Syst.
(2015)
Default mode network alterations in individuals with high-trait-anxiety: an EEG functional connectivity study
J. Affect. Disord.
Functional cortical connectivity analysis of mental fatigue unmasks hemispheric asymmetry and changes in small-world networks
Brain Cogn.
Neural mechanisms of mental fatigue revisited: new insights from the brain connectome
Engineering
Human fronto-parietal response scattering subserves vigilance at night
NeuroImage
Ghost interactions in MEG/EEG source space: a note of caution on inter-areal coupling measures
Neuroimage
Hyperedge bundling: a practical solution to spurious interactions in MEG/EEG source connectivity analyses
Neuroimage
Brain complex network analysis by means of resting state fMRI and graph analysis: will it be helpful in clinical epilepsy?
Epilepsy Behav.
State dependent properties of epileptic brain networks: comparative graph–theoretical analyses of simultaneously recorded EEG and MEG
Clin. Neurophysiol.
Analysis of autowave characteristics for competitive pulse coupled neural network and its application
Neurocomputing
Automatic design of pulse coupled neurons for image segmentation
Neurocomputing
Pattern recognition using pulse-coupled neural networks and discrete Fourier transforms
Neurocomputing
Topographic distribution of EEG alpha attractor correlation dimension values in wake and drowsy states in humans
Int. J. Psychophysiol.
The clinical and theoretical importance of EEG rhythms corresponding to states of lowered vigilance
Electroencephalogr. Clin. Neurophysiol.
Awareness of sleepiness and ability to predict sleep onset: can drivers avoid falling asleep at the wheel?
Sleep Med.
Cited by (17)
A vigilance estimation method for high-speed rail drivers using physiological signals with a two-level fusion framework
2023, Biomedical Signal Processing and ControlQuantitative evaluation of short-term resting-state brain networks for primary insomnia diagnosis
2022, Biomedical Signal Processing and ControlCitation Excerpt :DWT is an effective method to analyze various components (e.g. approximate and detailed components) of EEG due to its attractive properties such as good local representation in both time and frequency domain and multi-rate filtering [35]. On the basis of DWT, the wavelet-based threshold technique in [36–39] was applied to correct the sub-band EEG waves further. The comparison of the EEG signals before and after preprocessing is shown in Fig. 2.
Applications of brain imaging methods in driving behaviour research
2021, Accident Analysis and PreventionCitation Excerpt :Rather, studies need to have had offered behavioural insights via their brain imaging data. Studies whose primary focus was on clinical aspects (Huizeling et al., 2020), or technological (Kim et al., 2020) or methodological aspects of the brain mapping (Ma et al., 2020b; Wang et al., 2020; Zhang et al., 2020a; Zou et al., 2020) or those in which the brain signal was not particularly recorded while subjects actively driving were excluded. Studies on drivers using non-functional MRI, such as those studying the brain structure of taxi drivers, skilled drivers or car racers (Maguire et al., 1997, 2000; Maguire et al., 2006a, b; Bernardi et al., 2013; Lappi, 2015; Lima et al., 2020) were deemed out of the scope.
Fatigue State Detection Through Multiple Machine Learning Classifiers Using EEG Signal
2023, Applications of Modelling and Simulation