Elsevier

Neural Networks

Volume 148, April 2022, Pages 23-36
Neural Networks

Multi-dimensional conditional mutual information with application on the EEG signal analysis for spatial cognitive ability evaluation

https://doi.org/10.1016/j.neunet.2021.12.010Get rights and content

Highlights

  • MCMI describes coupling feature of EEG signals for spatial cognitive evaluation well.

  • The classification performance is best under multiple combination frequency bands.

  • MCMI is a valid valuable biomarker for spatial cognitive ability evaluation.

Abstract

This study aims to explore an effective method to evaluate spatial cognitive ability, which can effectively extract and classify the feature of EEG signals collected from subjects participating in the virtual reality (VR) environment; and evaluate the training effect objectively and quantitatively to ensure the objectivity and accuracy of spatial cognition evaluation, according to the classification results. Therefore, a multi-dimensional conditional mutual information (MCMI) method is proposed, which could calculate the coupling strength of two channels considering the influence of other channels. The coupled characteristics of the multi-frequency combination were transformed into multi-spectral images, and the image data were classified employing the convolutional neural networks (CNN) model. The experimental results showed that the multi-spectral image transform features based on MCMI are better in classification than other methods, and among the classification results of six band combinations, the best classification accuracy of Beta1–Beta2–Gamma combination is 98.3%. The MCMI characteristics on the Beta1–Beta2–Gamma band combination can be a biological marker for the evaluation of spatial cognition. The proposed feature extraction method based on MCMI provides a new perspective for spatial cognitive ability assessment and analysis.

Introduction

Spatial cognition refers to the information processing process of the dimension, shape, direction, and distance of people in the physical space or mental space of three-dimensional objects (Xu, Shushan, et al., 2018). Cognitive impairment, such as degeneration of visual–spatial structure and spatiotemporal disorientation (Johnson, 2010), is one of the most difficult problems in the diagnosis and treatment of brain diseases (Hornero et al., 2009, Whitwell et al., 2007). In the measurement and evaluation of spatial cognitive ability (Morris, 1984), the results of the common methods focus more on subjective judgments, and subjects are easily affected by environmental factors, resulting in errors and a lack of validity and reliability of the measurement results. Therefore, it is of great importance to explore spatial cognitive training and assessment methods for the prevention of spatial cognitive decline in healthy subjects. It is a problem to be solved in the field of brain science and cognitive science to analyze and study spatial cognitive ability and find a quick and effective method to evaluate spatial cognitive ability.

Coupling strength calculation is the mainstream method to quantify the change of spatial cognitive activity in EEG signal analysis. At present, only the method based on causality (Granger causality analysis (GCA) (Protopapa, Mylonas, Spiliotis, Siettos, Smyrnis, & Evdokimidis, 2011) and Sugihara causality) and mutual information (Li and Ouyang, 2010, Toppi et al., 2017, Wen et al., 2016, Wen et al., 2019) considers the coupling direction between EEG signals of different electrodes. Granger causality analysis has quantitatively analyzed the degree of linear dependence between different electrode signals (Bischof and Boulanger, 2003, Mcbride et al., 2015, Tarnanas et al., 2014). However, it has a high requirement for the selection of time window width. Sugihara causality describes the deterministic causality that is not easily detected but exists in the system (Tarnanas, Laskaris, Tsolaki, et al., 2015). CMI method considers the coupling strength relationship and coupling direction. Many studies have shown that CMI is superior to Granger causality when considering the coupling direction of neurons (Bashivan et al., 2016, Jaiswal et al., 2010, Kober and Neuper, 2011, Olton et al., 1979). However, CMI is an information theory method of binary time series, which only considers the one-way coupling between two neurons, but does not consider the influence of other related channels on the coupling relationship between two channels (Wen et al., 2016, Wen et al., 2019).

The above methods identify EEG signals by analyzing the characteristics of time domain, frequency domain, coupling strength, coherence strength, and coupling direction of different electrodes, but the spatial characteristics of EEG signals have not been considered. Therefore, Bashivan proposed to transform EEG signals into multispectral images with spatial attributes, which retains the spatial, temporal, and frequency attributes of EEG signals. Considering the influence of all other related channels on the coupling relationship between two channels, a feature extraction method of EEG signals based on multi-dimensional conditional mutual information (MCMI) was proposed to extract the coupled features of EEG signals before and after spatial cognitive training. Considering the advantages of multi-spectral images in feature extraction of EEG signals (Baskaran et al., 2013, Chen et al., 2018, Ieracitano et al., 2019), a convolutional neural network (CNN) model is used for classification after the coupled characteristics of the multi-frequency combination were transformed into multi-spectral images. To explore the performance of MCMI, this study not only compares the performance of MCMI with GCA and CMI but also compares MCMI with the existing coupling methods such as PCMI (Li & Ouyang, 2010) and MPCMI (Wen et al., 2016, Wen et al., 2019) on six band combinations: Theta–Alpha2–Gamma, Alpha2–Beta2–Gamma, Beta1–Beta2–Gamma, Theta–Beta2–Gamma, Theta–Alpha1–Gamma, and Alpha1–Alpha2–Gamma.

Section snippets

The data set

In this paper, we studied the 16 channel EEG signals from 7 subjects without any spiritual system or nerve disease male volunteers, normal eyesight or corrected visual acuity, no color blindness, normal right-handed, and did not take part in similar experiments training, the age distribution in 21–26 before the experiment signed informed consent (Wen, Yuan, Zhou, et al., 2020). The data set included two kinds of EEG signals from 7 subjects participating in the game of Virtual City Walking

Theta-Alpha2-Gamma band combination

The average verification loss of the multi-spectral image data built based on GCA, CMI, PCMI, MCMI, and MPCMI in the CNN model under the combined condition of The Frequency band of Theta–Alpha2–Gamma is shown in Fig. 3. According to Fig. 3, the following conclusions can be drawn: the curve representing the CNN model based on MCMI is the lowest, reaches the lowest point, and is stable after 190 195 iterations, and the average validation loss is stable at 0.09 0.10. The curve of THE CNN model

The value of the MCMI approach

In recent years, the analysis of coupling strength has provided a deeper insight into the mechanism of interaction between different brain regions and has become the focus of EEG signal research. MPCMI method is proposed in cognitive impairment diagnosis (Wen et al., 2016, Wen et al., 2019), and this method can effectively calculate kinetic characteristics of the coupling between different brain areas. It takes into account the effects of all other related channels on the coupling relationship

Conclusion

The MCMI-based EEG signal analysis method proposed in this study can better represent the coupling dynamic relationship between channels, and combining this method with the CNN model can more effectively analyze the spatial characteristics of the coupling relationship. The conclusion shows that the classification model constructed by the MCMI method combined with CNN under the frequency bands of Alpha2–Beta2–Gamma, Theta- Beta2-Gamma, and Beta1–Beta2–Gamma can most effectively evaluate the

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This research was funded by National Natural Science Foundation of China (61876165, 61503326), Natural Science Foundation of Hebei Province in China (F2016203343), China Postdoctoral Science Foundation (2015M581317). The authors have no any potential conflicts.

References (40)

  • SunB. et al.

    Synaptic devices based neuromorphic computing applications in artificialintelligence

    Materials Today Physics

    (2021)
  • TarnanasI. et al.

    Can a novel computerized cognitive screening test provideadditional information for early detection of Alzheimer’s disease?

    Alzheimers & Dementia

    (2014)
  • WenD. et al.

    Resting-state EEG coupling analysis of amnestic mild cognitive impairmentwith type 2 diabetes mellitus by using permutation conditional mutual information

    Clinical Neurophysiology

    (2016)
  • WenD. et al.

    Estimating coupling strength between multivariate neural series withmultivariate permutation conditional mutual information

    Neural Networks

    (2019)
  • BaşarM. et al.

    Investigation of emotional changes using features of EEG-Gamma band and different classifiers

  • Bashivan, P., Rish, I., & Yeasin, M., et al. (2016). learning representations from EEG with deeprecurrent-convolutional...
  • BaskaranA. et al.

    A review of electroencephalographic changes in diabetes mellitus inrelation to major depressive disorder

    Neuropsychiatric Disease and Treatment

    (2013)
  • BischofW.F. et al.

    Spatial navigation in virtual reality environments: An EEG analysis

    Cyberpsychology, Behavior, and Social Networking

    (2003)
  • Brain, Shi L. Inspired computing devices, chips and system. Asia-Pacific magnetic...
  • ChenD. et al.

    Hexadirectional modulation of theta power in human entorhinal cortexduring spatial navigation

    Current Biology

    (2018)
  • Cited by (10)

    • Scalp EEG functional connection and brain network in infants with West syndrome

      2022, Neural Networks
      Citation Excerpt :

      Similarly, McCrimmon et al. (2021) used the scalp high-frequency oscillations to accurately identify West subjects from healthy. Although, the characteristics of EEGs are very useful for clinical diagnosis of West as biomarkers (Cao, Zhu, Hu, & Kummert, 2020; Hu, Cao, Lai, Liu et al., 2020; Hussein, Lee, Ward, & McKeown, 2021; Raghu, Sriraam, Temel, Rao, & Kubben, 2020; Wen et al., 2021; Xu et al., 2021), these objective patterns are usually not conducive to understand the underlying neural mechanism of West. Graph theory has been recently applied in the brain imaging, electrophysiology and epilepsy analysis for its ability in quantifying the characteristics of complex system topologies (Bernhardt, Bonilha, & Gross, 2015; Stacey et al., 2020).

    View all citing articles on Scopus
    View full text