Skip to main content
SpringerLink
Log in

A spatial-frequency-temporal 3D convolutional neural network for motor imagery EEG signal classification

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Motor imagery (MI) EEG signal classification is a critical issue for brain–computer interface (BCI) systems. In traditional MI EEG machine learning algorithms, feature extraction and classification often have different objective functions, thus resulting in information loss. To solve this problem, a novel spatial-frequency-temporal (SFT) 3D CNN model is proposed. Specifically, the energies of EEG signals located in multiple local SFT ranges are extracted to obtain a novel 3D MI EEG feature representation, and a novel 3D CNN model is designed to simultaneously learn the complex MI EEG features in the entire SFT domains and carry out classification. An extensive experimental study is implemented on two public EEG datasets to evaluate the effectiveness of our method. For BCI Competition III Dataset IVa, the average accuracy rate of five subjects obtained by the proposed method reaches 86.6% and yields 4.1% improvement over the state-of-the-art filter band common spatial pattern (FBCSP) method. For BCI Competition III dataset IIIa, by achieving an average accuracy rate of 91.85%, the proposed method outperforms the state-of-the-art dictionary pair learning (DPL) method by 4.44%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Jin, Z.C., Zhou, G.X., Gao, D.Q., Zhang, Y.: EEG classification using sparse Bayesian extreme learning machine for brain–computer interface. Neural Comput. Appl. 32(11), 6601–6609 (2020)

    Article  Google Scholar 

  2. Jin, J., Miao, Y.Y., Daly, I., Zuo, C.L., Hu, D.W., Cichocki, A.: Correlation-based channel selection and regularized feature optimization for MI-based BCI. Neural Netw. 118, 262–270 (2019)

    Article  Google Scholar 

  3. Blankertz, B., Tomioka, R., Lemm, S., Kawanabe, M., Muller, K.R.: Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Process. Mag. 25(1), 41–56 (2008)

    Article  Google Scholar 

  4. Zhang, Y., Nam, C.S., Zhou, G.X., Jin, J., Wang, X.Y., Cichocki, A.: Temporally constrained sparse group spatial patterns for motor imagery BCI. IEEE Trans. Cybern. 49(9), 3322–3332 (2019)

    Article  Google Scholar 

  5. Jiao, Y., Zhang, Y., Chen, X., Yin, E.W., Jin, J., Wang, X.Y., Cichocki, A.: Sparse group representation model for motor imagery EEG classification. IEEE J. Biomed. Health Inf. 23(2), 631–641 (2019)

    Article  Google Scholar 

  6. Blankertz, B., Muller, K.R., Krusienski, D.J., Schalk, G., Wolpaw, J.R., Schlogl, A., Pfurtscheller, G., Millan, J.D.R., Schroder, M., Birbaumer, N.: The BCI competition III: Validating alternative approaches to actual BCI problems. IEEE Trans. Neural Syst. Rehabilit. Eng. 14(2), 153–159 (2006)

    Article  Google Scholar 

  7. Zhang, Y., Zhou, G.X., Jin, J., Wang, X.Y., Cichocki, A.: Optimizing spatial patterns with sparse filter bands for motor-imagery based brain–computer interface. J. Neurosci. Methods 255, 85–91 (2015)

    Article  Google Scholar 

  8. Ang, K.K., Chin, Z.Y., Wang, C.C., Guan, C.T., Zhang, H.H.: Filter bank common spatial pattern algorithm on BCI competition IV Datasets 2a and 2b. Front. Neurosci. 6, 39 (2012)

  9. Kumar, S., Sharma, A.: A new parameter tuning approach for enhanced motor imagery EEG signal classification. Med. Biol. Eng. Comput. 56(10), 1861–1874 (2018)

    Article  Google Scholar 

  10. Ameri, R., Pouyan, A., Abolghasemi, V.: Projective dictionary pair learning for EEG signal classification in brain computer interface applications. Neurocomputing 218, 382–389 (2016)

    Article  Google Scholar 

  11. Zeng, H., Song, A.G.: Optimizing single-trial EEG classification by stationary matrix logistic regression in brain–computer interface. IEEE Trans. Neural Netw. Learn. Syst. 27(11), 2301–2313 (2016)

    Article  MathSciNet  Google Scholar 

  12. Tabar, Y.R., Halici, U.: A novel deep learning approach for classification of EEG motor imagery signals. J. Neural Eng. 14(1), 016003 (2017)

    Article  Google Scholar 

  13. Sakhavi, S., Guan, C.T., Yan, S.C.: Learning temporal information for brain–computer interface using convolutional neural networks. IEEE Trans. Neural Netw. Learn. Syst. 29(11), 5619–5629 (2018)

    Article  MathSciNet  Google Scholar 

  14. Li, Y., Zhang, X.R., Zhang, B., Lei, M.Y., Cui, W.G., Guo, Y.Z.: A channel-projection mixed-scale convolutional neural network for motor imagery EEG decoding. IEEE Trans. Neural Syst. Rehabilit. Eng. 27(6), 1170–1180 (2019)

    Article  Google Scholar 

  15. Zhao, D.Y., Tang, F.Z., Si, B.L., Feng, X.S.: Learning joint space–time–frequency features for EEG decoding on small labeled data. Neural Netw. 114, 67–77 (2019)

    Article  Google Scholar 

  16. Zhu, X.Y., Li, P.Y., Li, C.B., Yao, D.Z., Zhang, R., Xu, P.: Separated channel convolutional neural network to realize the training free motor imagery BCI systems. Biomed. Signal Process. Control 49, 396–403 (2019)

    Article  Google Scholar 

  17. Tang, Z.C., Li, C., Sun, S.Q.: Single-trial EEG classification of motor imagery using deep convolutional neural networks. Optik 130, 11–18 (2017)

    Article  Google Scholar 

  18. Miao, M.M., Zhang, W.B., Hu, W.J., Wang, R.Q.: An adaptive multi-domain feature joint optimization framework based on composite kernels and ant colony optimization for motor imagery EEG classification. Biomed. Signal Process. Control 61, 101994 (2020). https://doi.org/10.1016/j.bspc.2020.101994

    Article  Google Scholar 

  19. Kim, S.J., Koh, K., Lustig, M., Boyd, S., Gorinevsky, D.: An interior-point method for large-scale l(1)-regularized least squares. IEEE J. Sel. Topics Signal Process. 1(4), 606–617 (2007)

    Article  Google Scholar 

  20. Pfurtscheller, G., Neuper, C.: Motor imagery activates primary sensorimotor area in humans. Neurosci. Lett. 239(2–3), 65–68 (1997)

    Article  Google Scholar 

  21. Bhattacharyya, S., Shimoda, S., Hayashibe, M.: A synergetic brain–machine interfacing paradigm for multi-DOF robot control. IEEE Trans. Syst. Man Cybern. Syst. 46(7), 957–968 (2016)

    Article  Google Scholar 

Download references

Acknowledgements

This work has been partially supported by grants from National Natural Science Foundation of China (Nos. 61772198, 61772199) and Zhejiang Province Basic Public Welfare Research Project of China (No. LGN18F020002).

Author information

Authors and Affiliations

  1. School of Information Engineering, Huzhou University, Huzhou, 313000, China

    Minmin Miao & Wenjun Hu

  2. Zhejiang Province Key Laboratory of Smart Management and Application of Modern Agricultural Resources, Huzhou University, Huzhou, 313000, China

    Minmin Miao & Wenjun Hu

  3. School of Instrument Science and Engineering, Southeast University, Nanjing, 210096, China

    Wenbin Zhang

Authors
  1. Minmin Miao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenjun Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenbin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Minmin Miao.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Miao, M., Hu, W. & Zhang, W. A spatial-frequency-temporal 3D convolutional neural network for motor imagery EEG signal classification. SIViP 15, 1797–1804 (2021). https://doi.org/10.1007/s11760-021-01924-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-021-01924-3

Keywords

Search

Navigation