A classification method for EEG motor imagery signals based on parallel convolutional neural network

Abstract

Deep learning has been used popularly and successfully in state of art researches to classify different types of images. However, so far, the applications of deep learning methods for the electroencephalography (EEG) motor imagery classification is very limited. In this study, a pre-processing algorithm is proposed for the EEG signals representation. Then, a parallel convolutional neural network (PCNN) architecture is proposed to classify motor imagery signals. For the raw EEG signals representation, a new form of the images is created to combine spatial filtering and frequency bands extracting together. By feeding the represented images into the PCNN, it stacks three unique sub-models together aiming to optimize the performance of classification. The average accuracy of the proposed method achieves 83.0 $\pm$ 3.4% on BCI Competition IV dataset 2b, which outperforms the compared methods at least 5.2%. The average Kappa value of the proposed method achieves 0.659 $\pm$ 0.067 on dataset 2b, providing at least 20.5% improvement with respect to the compared algorithms. The results show that the proposed method performs better in EEG motor imagery signals classification.

Introduction

The brain computer interface (BCI) is a direct interface from the human intent to external devices like computer, robot, unmanned aerial vehicle, and so on, which provides a novel non-muscular communication method via brain signals [1]. There are various types of the electroencephalography (EEG) signals used for many researches. For instance, P300 evoked potentials is an event related potential (ERP), which shows the sensitivity of subject’s response to stimulation. Steady state visually evoked potential (SSVEP) can recognize a set of characters under visual stimulation by identifying the SSVEP frequencies in the EEG signals which has been used in different applications like control grabbing of the robotic hand and assess visual acuity in adults[2], [3], [4]. Moreover, motor imagery (MI) is another widely used the EEG signals. MI signals can control robot movements via translate MI frequencies into different instructions [5]. Two or more types of the EEG signals can combine together to complete complex actions[5].

The MI is one type of the EEG signal from sensorimotor cortex when experimenter performs imagination of moving a body part without actual movement [6]. The motor behavior results in a change of the ongoing EEG as form an event related desynchronization (ERD) or an event related synchronization (ERS). The ERD represents an amplitude decrease of rhythmic activity and the ERS represents an amplitude increasing [7], [8]. The body parts of imaginary tasks may be left hand, right hand, foot, tongue, fingers, elbows even shoulder [9], [10], [11], [12], and so on. Thus, the classification of different types of MI signals is an important and fundamental problem in BCI researches.

In this study, a new form of image is proposed for representing the EEG signals. The form is not processed with channel by channel but with spatial and frequency filtering method. First, the raw EEG signals are projected into a low-dimensional space by regularized common spatial pattern (RCSP) aiming to maximize the distinction between two classes. Next, a projectional vector is obtained after this projection. Then, the short time fourier transform (STFT) is used to collect the mu and beta bands as frequency features, since the energy of mu and beta bands has a strong correlation with MI task, i.e., energy decreases in the mu band and energy increases in beta band [13]. Finally, mu and beta bands are combined together to form 2D images. We called this method as regularized common spatial pattern with short time fourier transform (RCSP-STFT).

Furthermore, a new convolutional neural network (CNN) architecture i.e., a parallel convolutional neural network (PCNN), which is proposed to classify 2D images after RCSP-STFT. Since the 2D images contain explicit time and frequency information, only simplex 2D or 1D convolutional kernel may not sufficient enough to capture the features from the 2D images. Thus, different 2D-kernel, 1D-kernel across frequency channel and 1D-kernel across time are combined together to extract features. For 1D-kernel across frequency channel, only one convolutional layer is used to mix all frequency channels together to create a whole new signal through convolutional operation. It is not recommended that the size of 1D-kernel across frequency is smaller than the numbers of the representation image channels [14]. For 1D-kernel across time, four convolutional layers are used, while 2D-kernel, eight convolutional layers are used. These three different CNN sub-models are trained synchronously in one model, and the outputs of these three different CNN parts are stacked together before the final fully-connected layers.

The proposed method is evaluated on the BCI Competition IV dataset 2b and dataset 1 [9], [10]. The instructions that measure our results are accuracy and Kappa value metrics [15]. And the results are compared with the current studies.

The rest of this paper is organized as follows: In Section 3, the RCSP-STFT algorithm for the EEG signals representation is described, and the structure of PCNN is proposed. In Section 4, the classification results on two datasets and the discussion are presented. Finally, we draw a conclusion in Section 5.