Revealing Spatiotemporal Brain Dynamics of Speech Production Based on EEG and Eye Movement

Bin Zhao, Jinfeng Huang, Gaoyan Zhang, Jianwu Dang, Minbo Chen, Yingjian Fu, Longbiao Wang

To understand the neural circuitry associated with speech production in oral reading, it is essential to describe the whole-range spatiotemporal brain dynamics in the processes including visual word recognition, orthography-phonology mapping, semantic accessing, speech planning, articulation, self-monitoring, etc. This has turned out to be extremely difficult because of demanding resolution in both spatial and temporal domains and advanced algorithms to eliminate severe contamination by articulatory movements. To tackle this hard target, we recruited 16 subjects in a sentence reading task and measured multimodal signals of electroencephalography (EEG), eye movement and speech simultaneously. The onset/offset of gazing and utterance were used for segmenting brain activation stages. Cortical modeling of causal interactions among anatomical regions was conducted on EEG signals through (i) independent component analysis to identify cortical regions of interest (ROIs); (ii) multivariate autoregressive modeling of representative cortical activity from each ROI; and (iii) quantification of the dynamic causal interactions among ROIs using the Short-time direct Directed Transfer function. The resulting brain dynamic model reveals a widely connected bilateral organization with left-lateralized semantic, orthographic and phonological sub-networks, right-lateralized prosody and motor sequencing sub-networks and bi-lateralized auditory and multisensory integration sub-networks that cooperate along interlaced and paralleled temporal stages for speech processing.

doi: 10.21437/Interspeech.2018-1908

Cite as: Zhao, B., Huang, J., Zhang, G., Dang, J., Chen, M., Fu, Y., Wang, L. (2018) Revealing Spatiotemporal Brain Dynamics of Speech Production Based on EEG and Eye Movement. Proc. Interspeech 2018, 1427-1431, doi: 10.21437/Interspeech.2018-1908

@inproceedings{zhao18d_interspeech,
  author={Bin Zhao and Jinfeng Huang and Gaoyan Zhang and Jianwu Dang and Minbo Chen and Yingjian Fu and Longbiao Wang},
  title={{Revealing Spatiotemporal Brain Dynamics of Speech Production Based on EEG and Eye Movement}},
  year=2018,
  booktitle={Proc. Interspeech 2018},
  pages={1427--1431},
  doi={10.21437/Interspeech.2018-1908},
  issn={2308-457X}
}