Research Paper
Optimal imaging of multi-channel EEG features based on a novel clustering technique for driver fatigue detection

https://doi.org/10.1016/j.bspc.2020.102103Get rights and content

Highlights

  • A driver fatigue detection method based on brain network clustering is proposed.

  • System performance is improved while considering effective spatial information.

  • EEG data with fatigue and accidents are used to evaluate the proposed method.

  • Most Fatigue was detected before accidents and subjective feelings of fatigue.

Abstract

Fatigue may cause a decrease in mental and physical performance capacity, which is a serious safety risk for the drivers in the transportation system. Recently, various studies have demonstrated the deviations of electroencephalogram (EEG) indicators from normal vigilant state during fatigue in time and frequency domains. However, when considering spatial information, these feature descriptors are not satisfying the demand for reliable detection due to the well-known challenge of signal mixing. In this paper, we propose a novel approach based on clustering on brain networks (CBNs) to alleviate the problem to improve the performance of driver fatigue detection. The clustering algorithm was employed to extract the spatial nodes with distinct connectivity attributes throughout the EEG-based brain networks. Then, the temporal features of wavelet entropy from the extracted nodes were transformed to spatio-temporal images so that the image edge detection method (pulse-coupled neural networks) to distinguish different stages of fatigue can be used. The experimental results demonstrated the temporal features from the extracted nodes reduced signal mixing and showed clearer deviations. The detected fatigue based on the imaging method was to an extent consistent with self-reported subjective feelings and most of the critical fatigue was detected before the subjective feelings of fatigue. For all the subjects, 21 of 29 accidents happened after detected fatigue in the simulated driving task. Therefore, the proposed method owns potential value for early warning and avoidance of traffic accidents caused by driver fatigue.

Introduction

Fatigue decreases drivers’ ability to operate vehicles safely and reduces their alertness level. It is one of the most important contributors to road accidents [1,2]. According to a report from National Highway Traffic Safety Administration (NHTSA), the accidents caused by driver fatigue make up close to 16.5 % of fatal crashes, but more than 1 in 4 drivers (29.4 %) reported having driving experience when they were so tired that they had a hard time keeping their eyes open in the past 30 days [3]. Thus, how to quantify, assess, and mitigate driver fatigue based on objective measurements, is an important challenge for traffic safety research [4,5]. Electrophysiology-based detection extracts the non-visible characteristics including heart rate variability, electrooculogram (EOG), electromyogram (EMG), and electroencephalogram (EEG), as well as other physiological indexes, reflecting the change of the physiological states directly and reliably [1,[6], [7], [8]]. In particular, EEG, which directly reflects the activities of the human brain is sensitive to fluctuations in alertness and has been applied to predicting performance degradation in a prolonged driving task [9].

As a physiological process which is gradual and accumulative, mental fatigue usually involves a number of different stages from the alertness and vigor stage to the tiredness and weakness state [4]. Correspondingly, its promising indicator, EEG, has demonstrated stage changes with state-specific and frequency-specific topographical differences in the fatigue transition [10,11]. According to the EEG differences, driver fatigue can be effectively identified by classification algorithms to make timely warning instructions. Currently, most of the existing methods detect driver fatigue based on the differences in the time and frequency domains. The detection accuracy of state-of-the-art techniques has exceeded 90 %. In the frequency domain, previous studies have shown that increasing drowsiness is typically associated with progressive changes in rhythmic activities, such as theta (4–8 Hz), alpha (8–13 Hz), or beta (13–30 Hz) activity or their combinations (e.g. (theta + alpha)/beta) [8,12]. Hence, these spectral features have been extracted to develop driver fatigue detection systems. For example, Li et al. [13] calculated 12 types of power spectra combinations of the rhythmic activities and fed them into the linear regression modal, which achieved an accuracy of 91.5 % for fatigue detection. At present, alpha-based analysis is a relatively most efficient method for detecting driver fatigue in frequency domain. The power of alpha oscillations predominantly in the central and posterior brain regions (parietal-occipital) is generally increased when the subjects are fatigued or tired [6,12]. According to the burst of alpha activity [8], Lawhern et al. [14] developed a discounted autoregressive (DAR) method just depending on the statistical properties of the alpha frequency band and achieved approximately 95 % accuracy.

Since the degraded performance of fatigue is involved in a variety of brain activity changes, the EEG complexity that can be quantified by different entropy estimators has been regarded as another prominent feature [5,15,16]. Through entropy-based feature extraction, the system realized the nonlinear estimation of the dynamical EEG activity during driving [5]. The typical entropy estimators used in fatigue detection system are fuzzy entropy (FE) [17] and wavelet entropy (WE) [18]. On the basis of FZ analysis, Luo et al. [19] introduced an adaptive scaling factor and proposed adaptive multi-scale FZ. It increased the detection accuracy to 95.4 %. In [5], the researchers explored WE-based real-time feature extraction of driver fatigue by using sliding window and analyzed the fusion effects of WE, approximate entropy (ApEn), sample entropy (SampEn). When these entropy features were applied to the multilayer perceptron (MLP) neural network, it achieved 96.5 % accuracy.

In accordance with the ten-twenty international standards [20], EEG signals are recorded from multiple electrodes attached to the scalp with a fixed spatial arrangement. Among the fatigue detection methods mentioned above, the EEG analyses in the time and frequency domains may neglect the valuable correlation information of the signals from different electrode positions, which can be concretely interpreted as a spatial domain.

As pointed out in the literature [21], a fatigued driver may fail to functionally and fast enough engage multi-perceptual and processing functions (e.g. attention, reasoning, decision making, sensorimotor coordination, and visual processing), resulting in a series of risky driving behaviors. The coherence and interaction between distinct brain resources intuitively appear to be important for these functions. As a technological advancement in neuroscience, functional brain network analysis, measuring coupling of the functional activities in different brain regions, can provide richer information about human cognition than simpler univariate approaches [22,23]. Several studies have reported that the functional brain networks become more integrated during task performance in comparison with the resting state, but linearly decline with ongoing time-on-task [23,24]. In the prolonged visuomotor vigilance task, Gaggioni et al. [25] suggested that decreased propagation of EEG response evoked by transcranial magnetic stimulation within the fronto-parietal cortex was related to the failure of increased vigilance. Under the simulated driving condition, Kong et al. [26] also revealed the degraded performance of small-world features of brain networks under fatigue state, providing further support for the presence of a reshaped global topology in connectivity networks when drivers shifted from the alert to the drowsy stage. Zhao et al. [27] attributed the shift to a more economic but less efficient configuration, or an inability to retrieve these resources related to mental fatigue.

However, when the spatial connectivity analysis are directly applied to fatigue detection, it may encounter challenges, because of signal mixing. Signal mixing also translates to volume conduction in the case of EEG recordings. This problem is caused by that the activity of any single neuronal source is detected by a spatially widespread group of electrodes. Therefore, the brain networks may contain spurious interactions [28,29]. It is difficult to find the related electrodes which are the most sensitive to the degraded performance to optimize fatigue detection.

In this context, we propose a novel EEG signal analysis approach based on clustering on brain networks (CBNs) to get more reliable and sensitive information from both the spatial and temporal dimensions to detect driver fatigue. Our hypothesis is that the introduced spatial information through CBNs can further improve the detection accuracy. The entire framework and its application are shown in Fig. 1, Fig. 2. In order to alleviate the problem of signal mixing, connection cluster is considered as the basic unit of connectivity analysis in CBNs. A connection cluster is defined as a tree composed of a node and the connections linking to it so that we can keep the effective information on fewer significant nodes with the similar links after clustering. Then, the EEG nonlinear dynamical features are extracted on the significant nodes (electrodes). Since the features on the significant nodes have consistent staged changes, the corresponding feature matrices are converted into images and we employ the image edge detection method to distinguish different stages of fatigue.

This paper is organized as follows. In Section 2, we describe the experimental details and EEG data preprocessing (see Sections 2.1 and 2.2), elaborate CBNs applied to suppress signal mixing to find significant nodes (see Section 2.3), add temporal feature extraction on the significant nodes to form imaging feature matrices (see Section 2.4), and illustrate two-dimensional pattern recognition to distinguish different stages of fatigue (see Section 2.5). Results of the study are presented in Section 3 and discussed in 4. Finally, the paper concludes in Section 5.

Section snippets

Experiments and data

This study was reviewed and approved by Ethics Committee, Dalian University of Technology (protocol number: 2018−040). Written informed consents were obtained from all participants before the experiments. Sixteen healthy right-handed subjects (eight males and eight females, age range: 20–35) who had driving licenses were recruited to participate in the simulated driving experiments. All the subjects had normal intelligence and no mental disorders or sleep problems. They have the habit of taking

Clustering on brain networks

Considering fatigue’s impact on the functional interaction, the brain networks in three temporal stages of driving were constructed to extract the spatial features. Fig. 7(a) shows the spatial connectivity averaged across the subjects at the stages A (0 h), B (0.5 h), and C (1 h). The strengths of the edges were represented by the thickness of the edges (the larger the strength the thicker the line). In order to make the results more obvious, all the edge strengths were magnified up to 2 times.

Discussion

The effects of fatigue are multifaceted [40]. It can bring the EEG changes in time-frequency-spatial domain. The results of the present study confirm the detection accuracy of fatigue can be further improved by introducing effective spatial information. The spatial information is extracted by CBNs.

Due to the issues such as signal mixing, the spatial information is vulnerable to interference, which needs to be processed. Especially in the prolonged driving task, the fatigue-related information

Conclusion

In this paper, a new EEG imaging method for automated detection and identification of driver fatigue was proposed. The nonlinear dynamical features of EEG from spatially optimized nodes extracted by CBNs were transformed into spatio-temporal images to extract reliably critical information of fatigue. The PCNN-based image processing method was utilized to recognize the two-dimensional fatigue pattern. The results showed that the temporal features from the spatially optimized nodes keep fatigue

CRediT authorship contribution statement

Chi Zhang: Conceptualization, Methodology, Software, Visualization, Writing - original draft, Writing - review & editing. Lina Sun: Conceptualization, Writing - original draft, Writing - review & editing. Fengyu Cong: Supervision. Tuomo Kujala: Resources, Writing - review & editing, Supervision. Tapani Ristaniemi: Conceptualization, Supervision. Tiina Parviainen: Writing - review & editing, Supervision.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (grant number 61703069), the National Foundation in China (No. JCKY2019110B009), and the Fundamental Research Funds for the Central Universities [grant number DUT18RC(4)035]. It is also to memorize Prof. Tapani Ristaniemi for his great help to Fengyu Cong and Chi Zhang.

References (48)

  • C. Imperatori et al.

    Default mode network alterations in individuals with high-trait-anxiety: an EEG functional connectivity study

    J. Affect. Disord.

    (2019)
  • Y. Sun et al.

    Functional cortical connectivity analysis of mental fatigue unmasks hemispheric asymmetry and changes in small-world networks

    Brain Cogn.

    (2014)
  • P. Qi et al.

    Neural mechanisms of mental fatigue revisited: new insights from the brain connectome

    Engineering

    (2019)
  • G. Gaggioni et al.

    Human fronto-parietal response scattering subserves vigilance at night

    NeuroImage

    (2018)
  • J.M. Palva et al.

    Ghost interactions in MEG/EEG source space: a note of caution on inter-areal coupling measures

    Neuroimage

    (2018)
  • S.H. Wang et al.

    Hyperedge bundling: a practical solution to spurious interactions in MEG/EEG source connectivity analyses

    Neuroimage

    (2018)
  • H. Onias et al.

    Brain complex network analysis by means of resting state fMRI and graph analysis: will it be helpful in clinical epilepsy?

    Epilepsy Behav.

    (2014)
  • M.-T. Horstmann et al.

    State dependent properties of epileptic brain networks: comparative graph–theoretical analyses of simultaneously recorded EEG and MEG

    Clin. Neurophysiol.

    (2010)
  • D. Zhou et al.

    Analysis of autowave characteristics for competitive pulse coupled neural network and its application

    Neurocomputing

    (2009)
  • H. Berg et al.

    Automatic design of pulse coupled neurons for image segmentation

    Neurocomputing

    (2008)
  • R.C. Mureşan

    Pattern recognition using pulse-coupled neural networks and discrete Fourier transforms

    Neurocomputing

    (2003)
  • A. Kalauzi et al.

    Topographic distribution of EEG alpha attractor correlation dimension values in wake and drowsy states in humans

    Int. J. Psychophysiol.

    (2015)
  • B. Roth

    The clinical and theoretical importance of EEG rhythms corresponding to states of lowered vigilance

    Electroencephalogr. Clin. Neurophysiol.

    (1961)
  • K.A. Kaplan et al.

    Awareness of sleepiness and ability to predict sleep onset: can drivers avoid falling asleep at the wheel?

    Sleep Med.

    (2007)
  • Cited by (17)

    • Quantitative evaluation of short-term resting-state brain networks for primary insomnia diagnosis

      2022, Biomedical Signal Processing and Control
      Citation Excerpt :

      DWT is an effective method to analyze various components (e.g. approximate and detailed components) of EEG due to its attractive properties such as good local representation in both time and frequency domain and multi-rate filtering [35]. On the basis of DWT, the wavelet-based threshold technique in [36–39] was applied to correct the sub-band EEG waves further. The comparison of the EEG signals before and after preprocessing is shown in Fig. 2.

    • Applications of brain imaging methods in driving behaviour research

      2021, Accident Analysis and Prevention
      Citation Excerpt :

      Rather, studies need to have had offered behavioural insights via their brain imaging data. Studies whose primary focus was on clinical aspects (Huizeling et al., 2020), or technological (Kim et al., 2020) or methodological aspects of the brain mapping (Ma et al., 2020b; Wang et al., 2020; Zhang et al., 2020a; Zou et al., 2020) or those in which the brain signal was not particularly recorded while subjects actively driving were excluded. Studies on drivers using non-functional MRI, such as those studying the brain structure of taxi drivers, skilled drivers or car racers (Maguire et al., 1997, 2000; Maguire et al., 2006a, b; Bernardi et al., 2013; Lappi, 2015; Lima et al., 2020) were deemed out of the scope.

    View all citing articles on Scopus
    View full text