Estimating cognitive workload using wavelet entropy-based features during an arithmetic task
Introduction
Researchers in cognitive science, psychology, Brain-Computer Interfaces (BCI), and Human-Computer Interaction (HCI), recognize the importance of gaining information about the user's cognitive processing capacity and memory workload or task engagement or a combination of these [1], [2]. This is due to the fact that the human cognitive system has a limited capacity for processing and holding information in the mind [3]. Therefore, if the increasing demand of cognitive activity exceeds the capacity limits, it may lead to cognitive overload, performance breakdown or even failure when accomplishing a cognitive task [3].
This should be prevented, particularly in highly demanding situations and workplaces where focused thinking and sustained performance is a key determinant, such as air traffic control, medical and emergency applications, and military operations or when designing or developing adaptive interfaces. Thus, there is a significant need to measure the amount of cognitive demand, precisely during a cognitive process to maintain efficiency, productivity and also avoid cognitive overload.
Conventional methods for measuring cognitive workload mainly include the measurement of reaction time, performance accuracy and self-assessment [4], [5]. However, these methods are based on the assumption that subjects are able and willing to respond accurately across tasks [6]. Besides, to date they have been measured in a post-hoc manner and are not available as on-line and continuous measurements, during the progress of the cognitive task. Physiological methods (i.e. brain activities, pupil dilatation [7], heartbeat rate, [7], hormone levels [6], galvanic skin response (GSR) [8]) have also been utilized in this field, previously. Although, among the physiological measures, brain activity measurement has been known as the most sensitive and consistent reflector of cognitive workload [2]. Since, it can interface more directly with the brain, which is the seat of cognitive activities using advanced brain-sensing technologies; such as Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) [9]. Behavioral measures (i.e. such as disfluencies in speech [10] and mouse and pen-input movements [11]) can also reflect cognitive workload, but they are the most distant level of measurement from the cognitive activity, and are also unable to measure the load, continuously.
EEG is a noninvasive neuro-imaging technique widely used to monitor and measure various types of cognitive activities, workloads and physiological states of the brain [12]. It is reliable, economical, and easy to use to record the neural electrical fluctuations of the brain, along the scalp. The EEG not only manifests the brain activity characteristics but can also reflect the underlying neural dynamics (using suitable quantifiers), due to its high temporal resolution (about 1 ms) [13], [14]. This is while, neural communications mostly occur in time-scales between 1 ms and 100 ms depending on brain processing time for various stimuli, which make fMRI temporal resolution (about few seconds) unsuitable for assessing many cognitive tasks [14]. On the other hand, it is known that as a result of information processing in the brain, dynamical variables of its electrical activity vary and this can change time, space and frequency characteristics of the EEG signal [13]. Thus, it would be beneficial to find these dynamical variables or features of EEG signals that are indicative of workload variations to successfully measure and classify the cognitive workload.
In previous research, a range of spectral features have been deployed for this purpose using EEG signals, including signal's maximum or average powers extracted from its power spectrum density (PSD) [9], [15], [16], [17]. Entropy-based features such as the wavelet packet entropy, entropy synchronization [18], [19] and approximate entropy [20] have been also used, but mainly in different mental tasks classification not in working memory load/cognitive workload classification.
Applying non-linear or dynamical features, in classifying different mental tasks (or in comparison with the rest condition), have also attracted attention, recently. These features are not only discriminative of some mental tasks but also can reflect the underlying neural dynamics. Therefore, dynamical measures like Correlation Dimension (CD) [21], [22], Hurst Exponent (HE), Approximate Entropy (ApEn) and Largest Lyapunov Exponent (LLE) [20], [23] have been used previously to measure the complexity or irregularity of the underlying brain dynamics in different mental tasks.
On the other hand, spectral coherence [24] and Phase Locking Value (PLV) [25] features have also been used previously to interpret the underlying brain dynamics from a synchronization point of view, when performing different mental tasks. The synchronization is of interest, due to the fact that different regions of the brain must communicate to make the integration of sensory information possible, which is necessary for conducting many cognitive functions. Thus, synchronization of oscillating neural ensembles is an integrative mechanism that may bring a widely distributed neural set together into a coherent ensemble that underlies a cognitive task [25].
Notwithstanding the plethora of previous EEG-based systems for measuring various cognitive states and pathologies proposed to date, the measurement of cognitive workload using EEG signals is still in its early stages, and there is still a need to validate the applicability of new approaches across different tasks, to establish the precision of cognitive load measurement, and investigate the underlying brain dynamics or behavior when performing a cognitive task with varying difficulty levels.
For this study, a cognitive task with seven levels of difficulty was designed, to examine the performance of the entropy-based features for fine load level measurement and discrimination. To our knowledge, five levels are the largest number of cognitive task loads previously induced [26], [27], and our work proceeds this to seven. We also aim to investigate the effect of load on neural regularity or order from the EEG signals when the imposed memory workload varies, which was not investigated prior to our study. In our previous work, features such as spectral entropy, CD, HE, ApEn, wavelet-entropic measures proved to be a good discriminator of imposed memory load and an indicator of higher predictability (less irregularity) in the brain activity, when dealing with higher memory load [28], [29], [30].
The aim of the paper is threefold: first, it compares the performance of our proposed wavelet-entropic (wavelet entropy-based) feature set in precise estimation of working memory load in seven levels, which is the highest proposed, so far. Second, it investigates the features' implications for synchronized and ordered neural activities towards a better understanding of the brain dynamics when dealing with higher loads, as the relationship between cognitive workload with synchronized and ordered neural activity are yet to be fully established. Finally, we compare the performance of this EEG-based method with existing measures of cognitive workload; i.e. performance (accuracy of responses), self-assessment, and reaction time of the subjects.
Section snippets
Technical background
Entropy is a measure of regularity or order [31] and more specifically in the case of EEG signals is a measure of the degree of synchrony of the neural groups participating in different neural responses [32]. When a stimulus is presented to the brain, the neural generators are triggered accordingly, and respond in a coherent way [13]. The EEG can reflect the activity of ensembles of these neural fluctuations in many frequency bands, which are active in a very complicated manner. This activation
Experiment
We designed our cognitive experiment as an arithmetic task with seven levels of difficulty, starting from one digit addition (very easy) to multi-digit addition (extremely difficult). This is because there is a rich literature on the concepts and procedure of mental arithmetic [39], and importantly this task allows the induction of many different workload levels. In [40], it is shown that the manipulation of the number of carry operations and the value of the carry is an important variable in
Preprocessing
The acquired EEG signals were first visually inspected and segments contaminated with EMG and EOG artifacts were removed to obtain artifact-free segments.1 To extract features, the EEG signals were segmented using a rectangular window of length T=5 s. The DC level of the segments was also removed and these segments were band-pass filtered in the frequency band of 0.5–30 Hz (as
Results
The source localization analysis demonstrated that the highest distributions of electrical activities are mainly estimated in the frontal and occipital regions of the brain in all the task difficulty levels, across all twelve subjects. For illustration purposes, the brain map results for two levels of task difficulty for subject 1 are shown in Fig. 1(a) and (b). As seen, the frontal and occipital regions exhibited notable cortical activation compared to other regions of the brain. However,
Discussion
Our objective was to investigate the usefulness of a new feature set namely; the wavelet-entropic features, in classifying working memory load (cognitive workload) on a fine scale. The four entropic measures; that is Shannon, Tsallis, Escort–Tsallis and Renyi entropies were calculated from the wavelet coefficients of the EEG signals to distinguish the memory load imposed during an arithmetic task. Wavelet coefficients have shown their capability for EEG signal classification in pathological
Conflict of interest statement
None.
References (56)
Entropy changes in brain function
Int. J. Psychophysiol.
(2007)EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis
Brain Res. Rev.
(1999)- et al.
Variability of EEG synchronization during a working memory task in healthy subjects
Int. J. Psychophysiol.
(2002) Nonlinear dynamical analysis of EEG and MEG: review of an emerging field
Clinical Neurophysiol.
(2005)- et al.
Mean phase coherence as a measure for phase synchronization and its application to the EEG of epilepsy patients
Physica D: Nonlinear Phenomena
(2000) - et al.
Sensitivity of human EEG alpha band desynchronization to different working memory components and increasing levels of memory load
Neurosci. Lett.
(2003) - et al.
A comparison of quantitative EEG features for neonatal seizure detection
Clinical Neurophysiol.
(2008) - et al.
and et al., EEG analysis using wavelet-based information tools
J. Neuroscie. Methods
(2006) Combined neural network model employing wavelet coefficients for EEG signals classification
Digital Signal Processing
(2009)- et al.
Classification of EEG signals using the wavelet transform
Signal Processing
(1997)
Brain electrical activity analysis using wavelet-based informational tools (II): Tsallis non-extensivity and complexity measures
Physica A: Stat. Mech. Appl.
Computational applications of nonextensive statistical mechanics
J. Comput. Appl. Math
Review of advanced techniques for the estimation of brain connectivity measured with EEG/MEG
Comput. Biol. Med.
What does delta band tell us about cognitive processes: a mental calculation study
Neurosci. Lett.
A supramodal number representation in human intraparietal cortex
Neuron
Automatic recognition of alertness level from EEG by using neural network and wavelet coefficients
Expert Systems with Appl.
Non-linear forecasting measurements of multichannel EEG dynamics
Electroencephalogr. Clin. Neurophysiol.
Wavelet entropy: a new tool for analysis of short duration brain electrical signals
J. Neurosci. Methods
Dynamic activities of the frontal association cortex in calculating and thinking
Neurosci. Res.
Functional roles of the cingulo-frontal network in performance on working memory
NeuroImage
Do specific EEG frequencies indicate different processes during mental calculation?
Neuro. Lett.
EEG delta activity: an indicator of attention to internal processing during performance of mental tasks
Int. J. Psychophysiol.
Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general
J. Neural. Eng.
Using electroencephalography to measure cognitive load
Educ. Psychol. Rev.
Cognitive load measurement as a means to advance cognitive load theory
Educ. Psychol.
The efficiency of instructional conditions: an approach to combine mental effort and performance measures
Hum. Factors: J. Hum. Factors Ergon Soc.
Measurment of cognitive load in instructional research
Percept. Mot. Skills
Psychophysiological assessment of workload in multi-task environments: Multiple-task performance
Cited by (97)
LGNet: Learning local–global EEG representations for cognitive workload classification in simulated flights
2024, Biomedical Signal Processing and ControlGray-level co-occurrence matrix of Smooth Pseudo Wigner-Ville distribution for cognitive workload estimation
2023, Biocybernetics and Biomedical EngineeringA comprehensive decoding of cognitive load
2022, Smart HealthCitation Excerpt :The total number of vertices of the used head model for this analysis was 13 439 vertices in which the number of vertices involved with each brain activation in different parts of the brain is shown in Table 1 and visualized in Fig. 9. These results are consistent with the results obtained at Giri, Kumar, and Gandhi (2020), Zarjam et al. (2013). This evidence asserts our hypothesis 2 that a specific pattern dominates the brain activation related to each cognitive load, which is inflectional in predicting the dynamics of cognitive processes.
Neurocognitive and physiological measurment of STEM learning processes
2022, International Encyclopedia of Education: Fourth EditionRecognition of driver's mental workload based on physiological signals, a comparative study
2022, Biomedical Signal Processing and Control