Skip to main content
SpringerLink
Log in

Multiband decomposition and spectral discriminative analysis for motor imagery BCI via deep neural network

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Human limb movement imagery, which can be used in limb neural disorders rehabilitation and brain-controlled external devices, has become a significant control paradigm in the domain of brain-computer interface (BCI). Although numerous pioneering studies have been devoted to motor imagery classification based on electroencephalography (EEG) signal, their performance is somewhat limited due to insufficient analysis of key effective frequency bands of EEG signals. In this paper, we propose a model of multiband decomposition and spectral discriminative analysis for motor imagery classification, which is called variational sample-long short term memory (VS-LSTM) network. Specifically, we first use a channel fusion operator to reduce the signal channels of the raw EEG signal. Then, we use the variational mode decomposition (VMD) model to decompose the EEG signal into six band-limited intrinsic mode functions (BIMFs) for further signal noise reduction. In order to select discriminative frequency bands, we calculate the sample entropy (SampEn) value of each frequency band and select the maximum value. Finally, to predict the classification of motor imagery, a LSTM model is used to predict the class of frequency band with the largest SampEn value. An open-access public data is used to evaluated the effectiveness of the proposed model. In the data, 15 subjects performed motor imagery tasks with elbow flexion/extension, forearm supination/pronation and hand open/close of right upper limb. The experiment results show that the average classification result of seven kinds of motor imagery was 76.2%, the average accuracy of motor imagery binary classification is 96.6% (imagery vs. rest), respectively, which outperforms the state-of-the-art deep learning-based models. This framework significantly improves the accuracy of motor imagery by selecting effective frequency bands. This research is very meaningful for BCIs, and it is inspiring for end-to-end learning research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Li Y, Pan J, Wang F, Yu Z. A hybrid BCI system combining P300 and SSVEP and its application to wheelchair control. IEEE Transactions on Biomedical Engineering, 2013, 60(11): 3156–3166

    Article  Google Scholar 

  2. Nicolas L F, Gomez J. Brain computer interfaces, a review. Sensors, 2012, 12: 1211–1279

    Article  Google Scholar 

  3. Wolpaw J R, Bedlack R S, Reda D J, Ringer R J, Banks P G, Vaughan T M, Heckman S M, McCane L M, Carmack C S, Winden S, McFarland D J, Sellers E W, Shi H R, Paine T, Higgins D S, Lo A C, Patwa H S, Hill K J, Huang G D, Ruff R L. Independent home use of a brain-computer interface by people with amyotrophic lateral sclerosis. Neurology, 2018, 91(3): 258–267

    Article  Google Scholar 

  4. Delisle D, Cardoso V, Gurve D, Loterio F, Romero-Laiseca M A, Krishnan S, Bastos T. System based on subject-specific bands to recognize pedaling motor imagery: towards a BCI for lower-limb rehabilitation. Journal of Neural Engineering, 2019, 16(5): 056005

    Article  Google Scholar 

  5. Wilmskoetter J, Gaizo J D, Phillip L, Behroozmand R, Gleichgerrcht E, Fridriksson J, Riley E, Bonilha L. Predicting naming responses based on pre-articulatory electrical activity in individuals with aphasia. Clinical Neurophysiology, 2019, 130(11): 2153–2163

    Article  Google Scholar 

  6. Yang C, Zhang H, Zhang S, Han X, Gao S, Gao X. The spatio-temporal equalization for evoked or event-related potential detection in multichannel EEG data. IEEE Transaction on Biomedical Engineering, 2020, 67: 2397–2414

    Google Scholar 

  7. Xu M, Xiao X, Wang Y, Qi H, Jung T P, Ming D. A brain-computer interface based on miniature-event-related potentials induced by very small lateral visual stimuli. IEEE Transaction on Biomedical Engineering, 2018, 65: 1166–1175

    Article  Google Scholar 

  8. Ramele R, Villar A J, Santos J M. EEG waveform analysis of p300 ERP with applications to brain computer interfaces. Brain Sciences, 2018, 8(11): 199

    Article  Google Scholar 

  9. Nakanishi M, Wang Y, Chen X, Wang Y T, Gao X, Jung T P. Enhancing detection of SSVEPs for a high-speed brain speller using task-related component analysis. IEEE Transaction on Biomedical Engineering, 2018, 65: 104–112

    Article  Google Scholar 

  10. Nakanishi M, Wang Y, Chen X, Wang Y T, Gao X, Jung T P. Enhancing detection of SSVEPs for a high-speed brain speller using task-related component analysis. IEEE Transaction on Biomedical Engineering, 2018, 65(1): 104–112

    Article  Google Scholar 

  11. Han X, Lin K, Gao S, Gao X. A novel system of SSVEP-based humanrobot coordination. Journal of Neural Engineering, 2019, 16(1): 016006

    Article  Google Scholar 

  12. Azab A M, Mihaylova L, Ang K K, Arvaneh M. Weighted transfer learning for improving motor imagery-based brain-computer interface. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27: 1352–1359

    Article  Google Scholar 

  13. Jiao Y, Zhang Y, Chen X, Yin E, Jin J, Wang X, Cichocki A. Sparse group representation model for motor imagery EEG classification. IEEE Journal of Biomedical and Health Informatics, 2019, 23: 631–641

    Article  Google Scholar 

  14. Zhang Y, Wang Y, Jin J, Wang X. Sparse Bayesian learning for obtaining sparsity of EEG frequency bands based feature vectors in motor imagery classification. International Journal of Neural Systems, 2017, 27(2): 1650032

    Article  Google Scholar 

  15. Tabar Y R, Halici U. A novel deep learning approach for classification of EEG motor imagery signal. Journal of Neural Engineering, 2017, 14(1): 016003

    Article  Google Scholar 

  16. Edelman B J, Baxter B, He B. EEG source imaging enhances the decoding of complex right-hand motor imagery tasks. IEEE Transaction on Biomedical Engineering, 2016, 63: 4–14

    Article  Google Scholar 

  17. Quiles E, Ferran S, Candela G. Low-cost robotic guide based on a motor imagery brain-computer interface for arm assisted rehabilitation. International Journal of Environmental Research and Public Health, 2020, 17(3): 699

    Article  Google Scholar 

  18. Mane R, Chew E, Phua K S, Kai K A, Guan C. Prognostic and monitory EEG-biomarkers for BCI upper-limb stroke rehabilitation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27: 1654–1664

    Article  Google Scholar 

  19. Nourmohammadi A, Jafari M, Zander T. A survey on unmanned aerial vehicle remote control using brain-computer Interface. IEEE Transaction on Human-Machine Systems, 2018, 48(4): 337–348

    Article  Google Scholar 

  20. Jafarifarmand A, Badamchizadeh M. A. EEG artifacts handling in a real practical brain-computer interface controlled vehicle. IEEE Transaciton on Neural Systems and Rehabilitation Engineering, 2019, 27(6): 1200–1208

    Article  Google Scholar 

  21. Edelman B J, Meng J, Suma D, Suma C, Zurn E. Noninvasive neuroimaging enhances continuous neural tracking for robotic device control. Science Robotics, 2019, 4(31): eaaw6844

    Article  Google Scholar 

  22. Su Y, Li W, Bi N, Lv Z. Adolescents environmental emotion perception by integrating EEG and eye movements. Frontiers in Neurorobotics, 2019, 13: 46

    Article  Google Scholar 

  23. Klimesch W. Alpha-band oscillations, attention, and controlled access to stored information. Trends in Cognitive Sciences, 2012, 16: 606–617

    Article  Google Scholar 

  24. Bartur G, Pratt H, Soroker N. Changes in mu and beta amplitude of the EEG during upper limb movement correlate with motor impairment and structural damage in subacute stroke. Clinical Neurophysiology, 2019, 130: 1644–1651

    Article  Google Scholar 

  25. Uji M, Wilson R, Francis S T, Mullinger K J, Mayhew S D. Exploring the advantages of multiband fMRI with simultaneous EEG to investigate coupling between gamma frequency neural activity and the BOLD response in humans. Human Brain Mapping, 2018, 39: 1673–1687

    Article  Google Scholar 

  26. Milz P, Pascual-Marqui R D, Achermann P, Kochi K, Faber P L. The EEG microstate topography is predominantly determined by intracortical sources in the alpha band. Neuroimage, 2017, 162: 353–361

    Article  Google Scholar 

  27. Zhao Y, Zhao Y, Pholpat D, Chen L, Sarrigiannis P G. Imaging of nonlinear and dynamic functional brain connectivity based on EEG recordings with the application on the diagnosis of alzheimer’s disease. IEEE Transaction on Medical Imaging, 2020, 39: 1571–1581

    Article  Google Scholar 

  28. Li Q, Şentürk D, Sugar C A, Jeste S, DiStefano C, Frohlich J, Telesca D. Inferring brain signals synchronicity from a sample of EEG readings. Journal of the American Statistical Association, 2019, 114: 991–1001

    Article  MathSciNet  MATH  Google Scholar 

  29. Jiang J, Yan Z, Shen T, Xu G, Guan Q, Yu Z. Use of deep belief network model to discriminate mild cognitive impairment and normal controls based on EEG, eye movement signals and neuropsychological tests. Journal of Medical Imaging and Health Informatics, 2019, 9: 1978–1985

    Article  Google Scholar 

  30. Chen X, Chen Q, Zhang Y and Wang Z J. A novel EEMD-CCA approach to removing muscle artifacts for pervasive EEG. IEEE Sensors Journal, 2019, 19: 8420–8431

    Article  Google Scholar 

  31. Zhang D, Yao L, Chen K, Wang S, Chang X, Liu Y. Making sense of Spatio-temporal preserving representations for EEG-based human intention recognition. IEEE Transaction on Cybernetics, 2019, 50: 3033–3044

    Article  Google Scholar 

  32. Aline D, Delorme R, Delanoë C, Amsellem F, Beggiato A, Germanaud D, Bourgeron T, Toro R, Dumas G. Alpha waves as a neuromarker of autism spectrum disorder: the challenge of reproducibility and heterogeneity. Frontiers in Neuroscience, 2018, 12: 662

    Article  Google Scholar 

  33. Su Y, Chen P, Liu X, Li W, Lv Z. A spatial filtering approach to environmental emotion perception based on electroencephalography. Medical Engineering & Physics, 2018, 60: 77–85

    Article  Google Scholar 

  34. Ghaemi A, Rashedi E, Pourrahimi A M, Kamandar M, Rahdari F. Automatic channel selection in EEG signals for classification of left or right hand movement in Brain Computer Interfaces using improved binary gravitation search algorithm. Biomedical Signal Processing and Control, 2017, 33: 109–118

    Article  Google Scholar 

  35. Lee H K, Choi Y S. Application of continuous wavelet transform and convolutional neural network in decoding motor imagery brain-computer interface. Entropy, 2019, 21(12): 1199

    Article  Google Scholar 

  36. Zeng W, Li M, Yuan C, Wang Q, Liu F, Wang Y. Identification of epileptic seizures in EEG signals using time-scale decomposition (ITD), discrete wavelet transform (DWT), phase space reconstruction (PSR) and neural networks. Artificial Intelligence Review, 2020, 53: 3059–3088

    Article  Google Scholar 

  37. Oh S H. Improving the subject independent classification of implicit intention by generating additional training data with PCA and ICA. International Journal of Contents, 2018, 14: 24–29

    Google Scholar 

  38. Jin J, Xiao R, Daly I, Miao Y, Cichocki A. Internal feature selection method of CSP based on L1-norm and dempster-shafer theory. IEEE Transactions on Neural Networks and Learning Systems, 2020

  39. Sachin B T, Sharma V, Siuly D, Sengur S. Features based on analytic IMF for classifying motor imagery EEG signals in BCI applications. Measurement, 2018, 116: 68–76

    Article  Google Scholar 

  40. Balaiah P. Comparative evaluation of adaptive filter and neuro-fuzzy filter in artifacts removal from electroencephalogram signal. American Journal of Applied Sciences, 2012, 9(10): 1583–1593

    Article  Google Scholar 

  41. Ameri R, Pouyan A, Abolghasemi V. Projective dictionary pair learning for EEG signal classification in brain computer interface applications. Neurocomputing, 2016, 218: 382–389

    Article  Google Scholar 

  42. Ma J, Zhang D. EEG signals feature extraction based on DWT and EMD combined with approximate entropy. Brain Sciences, 2019, 9(8): 201

    Article  Google Scholar 

  43. Jin J, Miao Y, Daly I, Zuo C, Cichocki A. Correlation-based channel selection and regularized feature optimization for MI-based BCI. Neural Networks, 2019, 118: 262–270

    Article  Google Scholar 

  44. Zhu X, Li P, Li C. Separated channel convolutional neural network to realize the training free motor imagery BCI systems. Biomedical Signal Processing and Control, 2019, 49: 396–403

    Article  Google Scholar 

  45. Alchalabi B, Faubert J. A comparison between BCI simulation and neurofeedback for forward/backward navigation in virtual reality. Computational Intelligence and Neuroscience, 2019, 4: 1–12

    Article  Google Scholar 

  46. Aljal M, Ridha D, Sutrisno I. Robot navigation using a brain computer interface based on motor imagery. Journal of Medical and Biological Engineering, 2019, 39(4): 508–522

    Article  Google Scholar 

  47. Huang L, Zhao Y, Zeng Y, Lin Z. BHCR: RSVP target retrieval BCI framework coupling with CNN by a Bayesian method. Neurocomputing, 2017, 238: 255–268

    Article  Google Scholar 

  48. Zhang Y, Zhou T, Wu W, Xie H, Cichocki A. Improving EEG decoding via clustering-based multi-task feature learning. IEEE Transactions on Neural Networks and Learning Systems, 2021

  49. Jiao Y, Zhou T, Yao L, Zhou G, Zhang Y. Multi-view multi-scale optimization of feature representation for EEG classification improvement. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28(12): 2589–2597

    Article  Google Scholar 

  50. Zhang X, Yao L, Wang X, Monaghan J, Mcalpine D, Zhang Y. A survey on deep learning-based non-invasive brain signals: recent advances and new frontiers. Journal of Neural Engineering, 2021, 18(3): 1–44

    Article  Google Scholar 

  51. Craik A, He Y, Jose L. Contreras-vidal deep learning for electroencephalogram (EEG) classification tasks: a review. Journal of Neural Engineering, 2019, 16(3): 031001

    Article  Google Scholar 

  52. Wang M, Lian C, Yao D, Shen D. Spatial-temporal dependency modeling and network hub detection for functional MRI analysis via convolutional-recurrent network. IEEE Transactions on Biomedical Engineering, 2020, 67(8): 2241–2252

    Article  Google Scholar 

  53. Acharya U, Oh S L, Hagiwara Y, Tan J H, Adeli H. Deep convolutional neural network for the automated detection and diagnosis of seizure using EEG signals. Computers in Biology and Medicine, 2018, 100: 270–278

    Article  Google Scholar 

  54. Hou Y, Zhou L, Jia S, Lun X. A novel approach of decoding EEG four-class motor imagery tasks via scout ESI and CNN. Journal of Neural Engineering, 2019, 17(1): 016048

    Article  Google Scholar 

  55. Chen J, Yu Z, Gu Z, Li Y. Deep temporal-spatial feature learning for motor imagery-based brain-computer interfaces. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28(11): 2356–2366

    Article  Google Scholar 

  56. Li Y, Zhang X R, Zhang B, Lei M Y, Cui W G, Guo Y Z. A channel-projection mixed-scale convolutional neural network for motor imagery EEG decoding. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27(6): 1170–1180

    Article  Google Scholar 

  57. Ma X, Qiu S, Wei W, Wang S, He H. Deep channel-correlation network for motor imagery decoding from the same limb. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28(1): 297–306

    Article  Google Scholar 

  58. Luo T J, Zhou C L, Luo F. Exploring spatial-frequency-sequential relationships for motor imagery classification with recurrent neural network. BMC Bioinformatics, 2018, 19(1): 1–18

    Article  Google Scholar 

  59. Wang P, Jiang A, Liu X, Shang J, Zhang L. LSTM-based EEG classification in motor imagery tasks. IEEE Transactions Neural Systems and Rehabilitation Engineering, 2018, 26(11): 2086–2095

    Article  Google Scholar 

  60. Tayeb Z, Fedjaev J, Ghaboosi N, Richter C, Everding L, Qu X, Wu Y, Cheng G, Conradt J. Validating deep neural networks for online decoding of motor imagery movements from EEG signals. Sensors, 2019, 19(1): 210

    Article  Google Scholar 

  61. Ofner P, Schwarz A, Pereira J, Müller-Putz G R. Upper limb movements can be decoded from the time-domain of low-frequency EEG. PLoS ONE, 2017, 12(8): e0182578

    Article  Google Scholar 

  62. Tangermann M, Müller K R, Aertsen A, Birbaumer N, Blankertz B. Review of the BCI competition IV. Frontiers on Neuroscience, 2012, 6: 55

    Article  Google Scholar 

  63. Rehman N U, Aftab H. Multivariate variational mode decomposition. IEEE Transaction on Signal Processing, 2019, 67: 6039–6052

    Article  MathSciNet  MATH  Google Scholar 

  64. Cheng Q, Yang W, Liu K, Zhao W, Yang Y. Increased sample entropy in EEGs during the functional rehabilitation of an injured brain. Entropy, 2019, 21(7): 698

    Article  Google Scholar 

  65. Martinez C V, Santamaria V E, Hornero R. Asynchronous control of P300-based brain-computer interfaces using sample entropy. Entropy, 2019, 21(3): 230

    Article  Google Scholar 

  66. Wang M, Zhang D, Huang J, Yap P T, Liu M. Identifying autism spectrum disorder with multi-site fMRI via low-rank domain adaptation. IEEE Transactions on Medical Imaging, 2020, 39(3): 644–655

    Article  Google Scholar 

  67. Fan J H, Sun C L, Chen C, Chen J, Xinyu Liu X, Zhao X, Meng L, Dai C, Chen W. EEG data augmentation: towards class imbalance problem in sleep staging tasks. Journal of Neural Engineering, 2020, 17(5): 056017

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (Grant Nos. 61876082, 61861130366, 61732006) and National Key R&D Program of China (2018YFC2001600, 2018YFC2001602).

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing, 211106, China

    Pengpai Wang, Mingliang Wang, Yueying Zhou, Ziming Xu & Daoqiang Zhang

Authors
  1. Pengpai Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingliang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yueying Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ziming Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Daoqiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daoqiang Zhang.

Additional information

Pengpai Wang received the MS degrees from Nanjing University of Information Science and Technology, China in 2018. He is currently working toward the PhD degree in College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, China. His research interests include machine learning, brain computer interaface and EEG analysis.

Mingliang Wang received the BS degree in Nanjing University of Information Science and Technology, China in 2012, and the MS degree from Shanghai Institute of Computing Technology, China in 2015. He is current working toward the PhD degree in software engineering from Nanjing University of Aeronautics and Astronautics, China. His current research interests include machine learning and medical image analysis.

Yueying Zhou received the BS and MS degrees from School of Mathematics Science, Liaocheng University, China in 2015 and 2019, respectively. She is currently pursuing the PhD degree in College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, China. Her research interests include brain-computer interface, EEG analysis and machine learning.

Ziming Xu received the BS degree from Nanjing University of Posts and Telecommunications, China in 2019. He is currently working toward the master degree in College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, China. His research interests include brain computer interface, EEG analysis and machine learning.

Daoqiang Zhang received the BS and PhD degrees in computer science from Nanjing University of Aeronautics and Astronautics (NUAA), China in 1999 and 2004, respectively. He joined the Department of Computer Science and Engineering of NUAA as a lecturer, in 2004, and is a professor at present, and is IAPR Fellow. His research interests include machine learning, pattern recognition, data mining, and medical image analysis. In these areas, he has published more than 150 scientific articles in refereed international journals such as the IEEE Transactions on Pattern Analysis and Machine Intelligence, the IEEE Transactions on Medical Imaging, Neuroimage, Human Brain Mapping, Medical Image Analysis; and conference proceedings such as IJCAI, AAAI, NIPS, CVPR, MM and MICCAI, with more than 12,000 citations by Google Scholar.

Electronic Supplementary Material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, P., Wang, M., Zhou, Y. et al. Multiband decomposition and spectral discriminative analysis for motor imagery BCI via deep neural network. Front. Comput. Sci. 16, 165328 (2022). https://doi.org/10.1007/s11704-021-0587-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-021-0587-2

Keywords

Search

Navigation