Skip to main content

Advertisement

SpringerLink
Log in

Effect of functional and effective brain connectivity in identifying vowels from articulation imagery procedures

  • Research Article
  • Published:
Cognitive Processing Aims and scope Submit manuscript

Abstract

Articulation imagery, a form of mental imagery, refers to the activity of imagining or speaking to oneself mentally without an articulation movement. It is an effective domain of research in speech impaired neural disorders, as speech imagination has high similarity to real voice communication. This work employs electroencephalography (EEG) signals acquired from articulation and articulation imagery in identifying the vowel being imagined during different tasks. EEG signals from chosen electrodes are decomposed using the empirical mode decomposition (EMD) method into a series of intrinsic mode functions. Brain connectivity estimators and entropy measures have been computed to analyze the functional cooperation and causal dependence between different cortical regions as well as the regularity in the signals. Using machine learning techniques such as multiclass support vector machine (MSVM) and random forest (RF), the vowels have been classified. Three different training and testing protocols (Articulation-AR, Articulation imagery-AI and Articulation vs Articulation imagery-AR vs AI) were employed for identifying the vowel being imagined of articulating. An overall classification accuracy of 80% was obtained for articulation imagery protocol which was found to be higher than the other two protocols. Also, MSVM techniques outperformed the RF technique in terms of the classification accuracy. The effect of brain connectivity estimators and machine learning techniques seems to be reliable in identifying the vowel from the subjects’ thought and thereby assisting the people with speech impairment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig.23
Fig.24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29

Similar content being viewed by others

References

  • Aleman A, Van’t Wout M, (2004) Subvocalization in auditory-verbal imagery: just a form of motor imagery? Cogn Process 5(4):228–231

    Article  Google Scholar 

  • Amo C, de Santiago L, Barea R, Lopez-Dorado A, Boquete L (2017) Analysis of gamma band activity from human EEG using empirical mode decomposition. Sensors 17(5):989

    Article  PubMed Central  Google Scholar 

  • Anderson RE (1982) Speech imagery is not always faster than visual imagery. Mem Cognit 10(4):371–380

    Article  CAS  PubMed  Google Scholar 

  • Baccala LA, Sameshina K (1999) Using partial directed coherence to describe neuronal ensemble Interactions. J Neurosci Methods 94(1):93–103

    Article  PubMed  Google Scholar 

  • Baccala LA, Sameshima K (2001) Partial directed coherence: a new concept in neural structure determination. Biol Cybern 84(6):463–474

    Article  CAS  PubMed  Google Scholar 

  • Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  Google Scholar 

  • Brodmann K (2007) Brodmann’s: localisation in the cerebral cortex. Springer, New York

    Google Scholar 

  • Callan DE, Callan AM, Honda K, Masaki S (2000) Single-sweep EEG analysis of neural processes underlying perception and production of vowels. Brain Res 10(1–2):173–176

    CAS  Google Scholar 

  • Cortes C, Vapnik V (1995) Support vector networks. Mach Learn 20(3):273–297

    Article  Google Scholar 

  • Chengaiyan S, Anandhan K (2015) Analysis of speech imagery using functional and effective EEG based Brain connectivity parameters. Int J Cog Inform Nat Intell 9(4):33–48

    Article  Google Scholar 

  • Chengaiyan S, Retnapandian AS, Anandan K (2020) Identification of vowels in consonant–vowel–consonant words from speech imagery based EEG signals. Cogn Neurodyn 14(1):1–19

    Article  PubMed  Google Scholar 

  • DaSalla CS, Kambara H, Sato M, Koike Y (2009) Single-trial classification of vowel speech imagery using common spatial patterns. Neural Netw 22(9):1334–1339

    Article  PubMed  Google Scholar 

  • Dogil G, Ackermann H, Grodd W, Haider H, Kamp H, Mayer J, Riecker A, Wildgruber D (2002) The speaking brain: a tutorial introduction to fMRI experiments in the production of speech, prosody and syntax. J Neurolinguistics 15(1):59–90

    Article  Google Scholar 

  • Flandrin P, Goncalves P, Rilling G(2004) Detrending and denoising with Empirical Mode Decompositions. In: Proceedings of the 2004 12th European Signal Processing Conference: pp 1581–1584

  • Granger C (1980) Testing for causality: a personal view point. J Econ Dyn Control 2:329–352

    Article  Google Scholar 

  • Huang NE, Shen Z, Long SR, Wu CM, Shih HH, Zheng Q, Yen NC, Tung CC, Liu HH, (1998) The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proc R Soc A 454(1971):903–995

    Article  Google Scholar 

  • Idrees BM, Farooq O (2016) EEG based Vowel Classification during Speech Imagery. In proceedings of the 2016 In: IEEE 3rd International Conference on Computing for Sustainable Global Development (INDIACom): pp. 1130–1134

  • Kaminski MJ, Blinowska KJ (1991) A new method of the description of the information flow in the brain structures. Biol Cybern 65(3):203–210

    Article  CAS  PubMed  Google Scholar 

  • Kaminski M, Ding M, Truccolo WA, Bressler SL (2001) Evaluating causal relations in neural systems: Granger causality, directed transfer function and statistical assessment of significance. Biol Cybern 85(2):145–157

    Article  CAS  PubMed  Google Scholar 

  • Kosslyn SM (1996) Image and brain: the resolution of the imagery debate. MIT Press, Cambridge

    Google Scholar 

  • Kumar Y, Dewal ML, Anand RS (2012) Features extraction of EEG signals using approximate and sample entropy. In: Proceedings of the 2012 IEEE Students' Conference on Electrical, Electronics and Computer Science: pp. 1–5

  • Liang H, Bressler SL, Desimone R, Fries P (2005) Empirical mode decomposition: a method for analyzing neural data. Neurocomputing 65:801–807

    Article  Google Scholar 

  • Madzarov G, Gjorgjevikj D, Chorbev I (2009) A multi-class SVM classifier utilizing binary decision tree. Informatica 33(2):233–241

    Google Scholar 

  • Mandic DP, Rehman N, Wu Z, Huang NE (2013) Empirical mode decomposition-based time-frequency analysis of multivariate signals. IEEE Signal Process Mag 30(6):74–86

    Article  Google Scholar 

  • Min B, Kim J, Park HJ, Lee B (2016) Vowel imagery decoding toward silent speech BCI using extreme learning machine with electroencephalogram. Bio Med Res Int 15:1–11

    Google Scholar 

  • Eva O, Oskar O (2013) Methodology and application of One-way ANOVA. Am J Mech Eng 1:256–261

    Google Scholar 

  • Perrone-Bertolotti M, Rapin L, Lachaux JP, Baciu M, Loevenbruck H. (2014) What is that little voice inside my head? Inner speech phenomenology, its role in cognitive performance, and its relation to self-monitoring. Behav Brain Res 261:220–239

    Article  CAS  PubMed  Google Scholar 

  • Pincus SM (1991) Approximate entropy as a measure of system complexity. Proc Natl Acad Sci 88(6):2297–2301

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Poeppel D, Hickok G (2007) The cortical organization of speech processing. Nat Rev Neurosci 8(5):393–402

    Article  PubMed  Google Scholar 

  • Price CJ, Crinion JT, MacSweeney MA (2011) Generative model of speech production in Broca’s and Wernicke’s areas. Front Psychol 2:237

    Article  PubMed  PubMed Central  Google Scholar 

  • Richman JS, Moorman JR (2000) Physiological time-series analysis using approximate entropy and sample entropy. Am J Physiol 278(6):2039–2049

    Google Scholar 

  • Rojas DA, Ramos OL (2006) Recognition of Spanish vowels through imagined speech by using spectral analysis and SVM. J Inform Hiding Multimedia Signal Process 7:14

    Google Scholar 

  • Sandhya C, Srinidhi G, Vaishali R, Visali M, Kavitha A(2015a) Analysis of speech imagery using brain connectivity estimators. In: Proceedings of the 2015a IEEE 14th international conference on cognitive informatics and cognitive computing (ICCCI*CC): pp 352–359

  • Sandhya, C; Anandha Sree R; Kavitha, A (2015b). Analysis of speech imagery using consonant-vowel syllable speech pairs and brain connectivity estimators. In: Proceedings of the 2015b 2nd international conference on biomedical systems, Signals and Images.

  • Sandhya C, Kavitha A (2019) Analysis of speech imagery using brain connectivity estimators on consonant-vowel-consonant words. Int J Biomed Eng Technol 30(4):329–343

    Article  Google Scholar 

  • Scheeren AM, Koot HM, Begeer S (2012) Social interaction style of children and adolescents with high-functioning autism spectrum disorder. J Autism Dev Disord 42(10):2046–2055

    Article  PubMed  Google Scholar 

  • Schreiber T (2000) Measuring information transfer. Phys Rev Lett 85(2):461–464

    Article  CAS  PubMed  Google Scholar 

  • Silchenko AN, Adamchica I, Pawelczyka N, Hauptmanna C, Maaroufb M, Sturmb V, Tassa PA (2010) Data-driven approach to the estimation of connectivity and time delays in the coupling of interacting neuronal subsystems. J Neurosci Methods 191(1):32–44

    Article  PubMed  Google Scholar 

  • Thatcher RW, Krause PJ, Hrybyk M (1986) Cortico-cortical associations an EEG coherence: a two-compartmental model. Electroencephalogr Clin Neurophysiol 64(2):123–143

    Article  CAS  PubMed  Google Scholar 

  • Thatcher RW, Biver CJ, North D (2004) EEG coherence and phase delays: comparisons between single reference, average reference and current source density. Neurology in Version 1, College of Medicine, University of South Florida

  • Tian X, Poeppel D (2010) Mental imagery of speech and movement implicates the dynamics of internal forward models. Front Psychol 1:166

    Article  PubMed  PubMed Central  Google Scholar 

  • Tian X, Poeppel D (2012) Mental imagery of speech: linking motor and perceptual systems through internal simulation and estimation. Front Hum Neurosci 6:314

    Article  PubMed  PubMed Central  Google Scholar 

  • Tian X, Zarate JM, Poeppel D (2016) Mental imagery of speech implicates two mechanisms of perceptual reactivation. Cortex 77:1–12

    Article  PubMed  PubMed Central  Google Scholar 

  • Ursino M, Ricci G, Magosso E (2020) Transfer entropy as a measure of brain connectivity: a critical analysis with the help of neural mass models. Front Comp Neurosci 14:45

    Article  Google Scholar 

  • Yoshimura N, Nishimoto A, Belkacem AN, Shin D, Kambara H, Hanakawa T, Koike Y (2016) Decoding of covert vowel articulation using electroencephalography cortical currents. Front Neurosci 10:175

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre for Healthcare Technologies, Department of Biomedical Engineering, SSN College of Engineering, Rajiv Gandhi Salai (OMR), Kalavakkam, Tamil Nadu, 603110, India

    Sandhya Chengaiyan & Kavitha Anandan

Authors
  1. Sandhya Chengaiyan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kavitha Anandan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sandhya Chengaiyan.

Ethics declarations

Conflict of interest

Sandhya Chengaiyan and Kavitha Anandan declare that they have no conflict of interests.

Ethical approval

Ethical approval was given by Sri Sivasubramaniya Nadar College of Engineering, Chennai, Tamil Nadu, India. This work was performed in the Department of Biomedical Engineering as per the guidelines of the Institutional Ethical Committee of SSN College of Engineering, India, for human participants.

Informed consent

Informed consent was obtained from all the individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Editor: Yan Bao (Peking University, & LMU Munich); Reviewers: Si Wu (Peking University) and a second researcher who prefers to remain anonymous.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chengaiyan, S., Anandan, K. Effect of functional and effective brain connectivity in identifying vowels from articulation imagery procedures. Cogn Process 23, 593–618 (2022). https://doi.org/10.1007/s10339-022-01103-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10339-022-01103-3

Keywords

Search

Navigation