Skip to main content
SpringerLink
Log in

A Review of Research on Brain-Computer Interface Based on Imagined Speech

  • Conference paper
  • First Online:
Cognitive Systems and Signal Processing (ICCSIP 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1397))

Included in the following conference series:

  • 1629 Accesses

Abstract

Brain-computer interface is currently a rapidly developing technology. In recent years, it has received extensive attention and high expectations in the fields of biomedical engineering and rehabilitation medicine engineering. Brain-computer interfaces can enable patients with communication skills or physical disabilities to communicate with machines and equipment, and brain-computer interfaces based on imagined speech can provide patients with normal and effective language communication. At present, its related research has achieved certain results. This article introduces the principles, advantages and disadvantages of several common BCI systems, as well as the two most widely used brain signals EEG and EcoG, and then studies some related feature extraction and data classification algorithms used in current research. Finally, the current problems and future development trends of brain-computer interfaces based on imagined speech are discussed.

This work was supported by National Natural Science Foundation of China with Grant No. 91848206 and Natural Science Foundation of university in Anhui Province (No. KJ2019A0086).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Kübler, A., Kotchoubey, B., Hinterberger, T., et al.: The thought translation device: a neurophysiological approach to communication in total motor paralysis. Exp. Brain Res. 124(2), 223–232 (1999)

    Article  Google Scholar 

  2. Yahud, S., Abu Osman, N. A.: Prosthetic hand for the brain-computer interface system. In: Ibrahim, F., Osman, N.A.A., Usman, J., Kadri, N.A. (eds.) 3rd Kuala Lumpur International Conference on Biomedical Engineering 2006. IP, vol. 15, pp. 643–646. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-68017-8_162

    Chapter  Google Scholar 

  3. Rebsamen, B., Burdet, E., Guan, C., et al.: Controlling a wheelchair indoors using thought. IEEE Intell. Syst. 22(2), 18–24 (2007)

    Article  Google Scholar 

  4. Abiri, R., Borhani, S., Sellers, E.W., Jiang, Y., Zhao, X.: A comprehensive review of EEG-based brain-computer interface paradigms. J. Neural Eng. 16(1), 011001 (2019). https://doi.org/10.1088/1741-2552/aaf12e

    Article  Google Scholar 

  5. Fabiani, M., Gratton, G., Karis, D., Donchin, E.: Definition, identification, and reliability of measurement of the P300 component of the event-related brain potential. Adv. Psychophysiol. 2(S 1), 78 (1987).

    Google Scholar 

  6. Polich, J.: Updating P300: an integrative theory of P3a and P3b. Clin. Neurophysiol. 118(10), 2128–2148 (2007)

    Article  Google Scholar 

  7. Chang, M.H., Baek, H.J., Lee, S.M., Park, K.S.: An amplitude-modulated visual stimulation for reducing eye fatigue in SSVEP-based brain–computer interfaces. Clin. Neurophysiol. 125(7), 1380–1391 (2014)

    Article  Google Scholar 

  8. Molina, G.G., Mihajlovic, V.: Spatial filters to detect steady-state visual evoked potentials elicited by high frequency stimulation: BCI application. Biomedizinische Technik/Biomed. Eng. 55(3), 173–182 (2010)

    Article  Google Scholar 

  9. Müller, S.M.T., Diez, P.F., Bastos-Filho, T.F., Sarcinelli-Filho, M., Mut, V., Laciar, E.: SSVEP-BCI implementation for 37–40 Hz frequency range. In: Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE, pp. 6352–6355: IEEE (2011)

    Google Scholar 

  10. Volosyak, I., Valbuena, D., Luth, T., Malechka, T., Graser, A.: BCI demographics II: how many (and what kinds of) people can use a high-frequency SSVEP BCI? IEEE Trans. Neural Syst. Rehabil. Eng. 19(3), 232–239 (2011)

    Article  Google Scholar 

  11. Morash, V., Bai, O., Furlani, S., Lin, P., Hallett, M.: Classifying EEG signals preceding right hand, left hand, tongue, and right foot movements and motor imageries. Clin. Neurophysiol. 119(11), 2570–2578 (2008)

    Article  Google Scholar 

  12. Hochberg, L.R., et al.: Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442(7099), 164–171 (2006)

    Article  Google Scholar 

  13. Kim, S.-P., Simeral, J.D., Hochberg, L.R., Donoghue, J.P., Black, M.J.: Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia. J. Neural Eng. 5(4), 455 (2008)

    Article  Google Scholar 

  14. Yuan, H., He, B.: Brain-computer interfaces using sensorimotor rhythms: current state and future perspectives. IEEE Trans. Biomed. Eng. 61(5), 1425–1435 (2014)

    Google Scholar 

  15. Ibayashi, K., Kunii, N., Matsuo, T., et al.: Decoding speech with integrated hybrid signals recorded from the human ventral motor cortex. Front. Neurosci. 12, 221 (2018). https://doi.org/10.3389/fnins.2018.00221

  16. Song, C., Xu, R., Hong, B.: Decoding of Chinese phoneme clusters using ECoG. In: Conference Proceedings-IEEE Engineering in Medicine and Biology Society 2014, pp. 1278–1281 (2014). https://doi.org/10.1109/EMBC.2014.6943831

  17. Anumanchipalli, G.K., Chartier, J., Chang, E.F.: Speech synthesis from neural decoding of spoken sentences. Nature 568(7753), 493–498 (2019). https://doi.org/10.1038/s41586-019-1119-1

    Article  Google Scholar 

  18. Dash, D., Ferrari, P., Wang, J.: Decoding Imagined and spoken phrases from non-invasive neural (MEG) signals. Front. Neurosci. 14, 290 (2020). https://doi.org/10.3389/fnins.2020.00290

  19. Sereshkeh, A.R., Trott, R., Bricout, A., Chau, T.: Online EEG classification of covert speech for brain-computer interfacing. Int. J. Neural Syst. 27(8), 1750033 (2017). https://doi.org/10.1142/S0129065717500332

    Article  Google Scholar 

  20. Mugler, E.M., Patton, J.L., Flint, R.D., et al.: Direct classification of all American English phonemes using signals from functional speech motor cortex. J. Neural Eng. 11(3), 035015 (2014). https://doi.org/10.1088/1741-2560/11/3/035015

    Article  Google Scholar 

  21. Pei, X., Barbour, D.L., Leuthardt, E.C., Schalk, G.: Decoding vowels and consonants in spoken and imagined words using electrocorticographic signals in humans. J. Neural Eng. 8(4), 046028 (2011). https://doi.org/10.1088/1741-2560/8/4/046028

    Article  Google Scholar 

  22. Balaji, A., Haldar, A., Patil, K., et al.: EEG-based classification of bilingual unspoken speech using ANN. In: Conference Proceedings-IEEE Engineering in Medicine and Biology Society 2017, pp. 1022–1025 (2017). https://doi.org/10.1109/EMBC.2017.8037000

  23. Pawar, D., Dhage, S.: Multiclass covert speech classification using extreme learning machine. Biomed. Eng. Lett. 10(2), 217–226 (2020). https://doi.org/10.1007/s13534-020-00152-x

    Article  Google Scholar 

  24. Tottrup, L., Leerskov, K., Hadsund, J.T., Kamavuako, E.N., Kaseler, R.L., Jochumsen, M.: Decoding covert speech for intuitive control of brain-computer interfaces based on single-trial EEG: a feasibility study. In: IEEE International Conference on Rehabilitation Robotics 2019, pp. 689–693 (2019). https://doi.org/10.1109/ICORR.2019.8779499

  25. Chengaiyan, S., Retnapandian, A., Anandan, K.: Identification of vowels in consonant–vowel–consonant words from speech imagery based EEG signals. Cogn. Neurodyn. 14(1), 1–19 (2019). https://doi.org/10.1007/s11571-019-09558-5

    Article  Google Scholar 

  26. Livezey, J.A., Bouchard, K.E., Chang, E.F.: Deep learning as a tool for neural data analysis: speech classification and cross-frequency coupling in human sensorimotor cortex. PLoS Comput. Biol. 15(9), e1007091 (2019). https://doi.org/10.1371/journal.pcbi.1007091

  27. Bouchard, K.E., Chang, E.F.: Neural decoding of spoken vowels from human sensory-motor cortex with high-density electrocorticography. In: Conference Proceedings-IEEE Engineering in Medicine and Biology Society 2014, pp. 6782–6785 (2014). https://doi.org/10.1109/EMBC.2014.6945185

  28. Makin, J.G., Moses, D.A., Chang, E.F.: Machine translation of cortical activity to text with an encoder-decoder framework. Nat. Neurosci. 23(4), 575–582 (2020). https://doi.org/10.1038/s41593-020-0608-8

    Article  Google Scholar 

  29. Akbari, H., Khalighinejad, B., Herrero, J.L., Mehta, A.D., Mesgarani, N.: Towards reconstructing intelligible speech from the human auditory cortex. Sci. Rep. 9(1), 874 (2019). https://doi.org/10.1038/s41598-018-37359-z

  30. Ahn, J.W., Ku, Y., Kim, H.C.: A novel wearable EEG and ECG recording system for stress assessment. Sensors (Basel) 19(9), 1991 (2019). https://doi.org/10.3390/s19091991

  31. Athavipach, C., Pan-Ngum, S., Israsena, P.: A wearable in-ear EEG device for emotion monitoring. Sensors (Basel). 19(18), 4014 (2019). https://doi.org/10.3390/s19184014

  32. Kawana, T., Yoshida, Y., Kudo, Y., Miki, N.: In: EEG-hat with candle-like microneedle electrode. In: Conference Proceedings-IEEE Engineering in Medicine and Biology Society 2019; pp. 1111–1114 (2019). https://doi.org/10.1109/EMBC.2019.8857477

  33. Shi, Z., Zheng, F., Zhou, Z., et al.: Silk-enabled conformal multifunctional bioelectronics for investigation of spatiotemporal epileptiform activities and multimodal neural encoding/decoding. Adv. Sci. (Weinh) 6(9):1801617 (2019). https://doi.org/10.1002/advs.201801617

  34. Choi, H., Lee, S., Lee, J., et al.: Long-term evaluation and feasibility study of the insulated screw electrode for ECoG recording. J. Neurosci. Methods. 308, 261–268 (2018). https://doi.org/10.1016/j.jneumeth.2018.06.027

    Article  Google Scholar 

  35. Xu, K., Li, S., Dong, S., et al.: Bioresorbable electrode array for electrophysiological and pressure signal recording in the brain. Adv. Healthc. Mater. 8(15), e1801649 (2019). https://doi.org/10.1002/adhm.201801649

    Article  Google Scholar 

  36. Brumberg, J.S., Pitt, K.M., Burnison, J.D.: A noninvasive brain-computer interface for real-time speech synthesis: the importance of multimodal feedback. IEEE Trans. Neural Syst. Rehabil. Eng. 26(4), 874–881 (2018). https://doi.org/10.1109/TNSRE.2018.2808425

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Anhui University of Technology, Maanshan, 243032, Anhui, China

    Chengyin Wang, Wenlong Ding & Jianhua Shan

  2. Tsinghua University, Beijing, 100085, China

    Bin Fang

Authors
  1. Chengyin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenlong Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianhua Shan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bin Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Fang .

Editor information

Editors and Affiliations

  1. Tsinghua University, Beijing, China

    Fuchun Sun

  2. Tsinghua University, Beijing, China

    Huaping Liu

  3. Tsinghua University, Beijing, China

    Bin Fang

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, C., Ding, W., Shan, J., Fang, B. (2021). A Review of Research on Brain-Computer Interface Based on Imagined Speech. In: Sun, F., Liu, H., Fang, B. (eds) Cognitive Systems and Signal Processing. ICCSIP 2020. Communications in Computer and Information Science, vol 1397. Springer, Singapore. https://doi.org/10.1007/978-981-16-2336-3_34

Download citation

  • DOI: https://doi.org/10.1007/978-981-16-2336-3_34

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-16-2335-6

  • Online ISBN: 978-981-16-2336-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation