Abstract
In this paper, a novel Electroepncephalography (EEG)-based Brain Computer Interface (BCI) approach is proposed to decode motion intention from EEG signals collected at the scalp of subjects performing motor execution tasks. The impact of such systems, generally based on the ability to discriminate between the imagination of right/left hand movements, would greatly benefit from the ability to decode the intention to perform sub-movements of the same limb like opening or closing the same hand. In this research, a system meant for decoding the intention to open or close the same hand is proposed. To this end, a dataset of EEG segments preceding hand open/close movement initiation as well as segments with no movement preparation (resting) was created from a public database of EEG signals recorded during upper limb motor execution experiments. Time-frequency maps were constructed for every EEG signal and used to build channel\(\,\times \,\)frequency\(\,\times \,\)time volumes. A system based on a custom deep Convolutional Neural Network (CNN), named EEGframeNNET was designed and developed to discriminate between pre-hand-opening, pre-hand-closing and resting. The proposed system outperformed a comparable method in the literature (TTF-NET) achieving an average accuracy of 86.5% against the 76.3% of TTF-NET. The proposed system offers a novel perspective on EEG signals evolution by projecting EEGs into a sequence of channel\(\,\times \,\)frequency frames constructed by means of time-frequency analysis.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Catani, M.: A little man of some importance. Brain 140(11), 3055–3061 (2017)
Cho, J.H., Jeong, J.H., Shim, K.H., Kim, D.J., Lee, S.W.: Classification of hand motions within EEG signals for non-invasive BCI-based robot hand control. In: 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 515–518. IEEE (2018)
Daubechies, I.: Ten lectures on wavelets. In: Society for Industrial and Applied Mathematics (1992)
Ieracitano, C., Mammone, N., Hussain, A., Morabito, F.C.: A novel explainable machine learning approach for EEG-based brain-computer interface systems. Neural Comput. Appl. 34(14), 11347–11360 (2021). https://doi.org/10.1007/s00521-020-05624-w
Ieracitano, C., Morabito, F.C., Hussain, A., Mammone, N.: A hybrid-domain deep learning-based BCI for discriminating hand motion planning from EEG sources. Int. J. Neural Syst. 31(09), 2150038 (2021)
Irimia, D., et al.: recoveriX: a new BCI-based technology for persons with stroke. In: 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 1504–1507. IEEE (2016)
Irimia, D.C., et al.: Brain-computer interfaces with multi-sensory feedback for stroke rehabilitation: a case study. Artif. Organs 41(11), E178–E184 (2017)
Irimia, D.C., Ortner, R., Poboroniuc, M.S., Ignat, B.E., Guger, C.: High classification accuracy of a motor imagery based brain-computer interface for stroke rehabilitation training. Front. Robot. AI 5, 130 (2018)
Jeong, J.H., Kwak, N.S., Guan, C., Lee, S.W.: Decoding movement-related cortical potentials based on subject-dependent and section-wise spectral filtering. IEEE Trans. Neural Syst. Rehabil. Eng. 28(3), 687–698 (2020)
Jeong, J.H., Lee, B.H., Lee, D.H., Yun, Y.D., Lee, S.W.: EEG classification of forearm movement imagery using a hierarchical flow convolutional neural network. IEEE Access 8, 66941–66950 (2020)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Lotte, F., et al.: A review of classification algorithms for EEG-based brain-computer interfaces: a 10 year update. J. Neural Eng. 15(3), 031005 (2018)
Lotte, F., Jeunet, C.: Online classification accuracy is a poor metric to study mental imagery-based BCI user learning: an experimental demonstration and new metrics. In: 7th International BCI Conference, pp. hal-01519478 (2017)
Mammone, N., Ieracitano, C., Morabito, F.C.: A deep CNN approach to decode motor preparation of upper limbs from time-frequency maps of EEG signals at source level. Neural Netw. 124, 357–372 (2020)
Müller-Putz, G.R., Schwarz, A., Pereira, J., Ofner, P.: From classic motor imagery to complex movement intention decoding: the noninvasive Graz-BCI approach. Prog. Brain Res. 228, 39–70 (2016)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML 2010), pp. 807–814 (2010)
Namazi, H., Ala, T.S., Kulish, V.: Decoding of upper limb movement by fractal analysis of electroencephalogram (EEG) signal. Fractals 26(05), 1850081 (2018)
Ofner, P., Schwarz, A., Pereira, J., Müller-Putz, G.R.: Upper limb movements can be decoded from the time-domain of low-frequency EEG. PLoS ONE 12(8), e0182578 (2017)
Ofner, P., Schwarz, A., Pereira, J., Wyss, D., Wildburger, R., Müller-Putz, G.R.: Attempted arm and hand movements can be decoded from low-frequency EEG from persons with spinal cord injury. Sci. Rep. 9(1), 7134 (2019)
Pereira, J., Ofner, P., Schwarz, A., Sburlea, A.I., Müller-Putz, G.R.: EEG neural correlates of goal-directed movement intention. Neuroimage 149, 129–140 (2017)
Powers, D.M.W.: Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. J. Mach. Learn. Technol. 2(1), 37–63 (2011)
Shakeel, A., Navid, M.S., Anwar, M.N., Mazhar, S., Jochumsen, M., Niazi, I.K.: A review of techniques for detection of movement intention using movement-related cortical potentials. Comput. Math. Methods Med. 2015, 346217 (2015)
Spring, J.N., Place, N., Borrani, F., Kayser, B., Barral, J.: Movement-related cortical potential amplitude reduction after cycling exercise relates to the extent of neuromuscular fatigue. Front. Hum. Neurosci. 10, 257 (2016)
Acknowledgment
This work was supported in part by PON 2014–2020, COGITO project - Grant Ref. ARS01\(\_\)00836; by “iCARE” project (CUP J39J14001400007) - action 10.5.12 - funded within POR FESR FSE 2014/2020 of Calabria Region with the participation of European Community Resources of FESR and FSE, of Italy and of Calabria and by the Programma Operativo Nazionale (PON) “Ricerca e Innovazione” 2014–2020 CCI2014IT16M2OP005 (CUP C35F21001220009 code: I05).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mammone, N., Ieracitano, C., Spataro, R., Guger, C., Cho, W., Morabito, F.C. (2022). Decoding Motor Preparation Through a Deep Learning Approach Based on EEG Time-Frequency Maps. In: Mahmud, M., Ieracitano, C., Kaiser, M.S., Mammone, N., Morabito, F.C. (eds) Applied Intelligence and Informatics. AII 2022. Communications in Computer and Information Science, vol 1724. Springer, Cham. https://doi.org/10.1007/978-3-031-24801-6_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-24801-6_12
Publisher Name: Springer, Cham
Online ISBN: 978-3-031-24801-6
eBook Packages: Computer ScienceComputer Science (R0)