The oscillatory activities and its synchronization in auditory-visual integration as revealed by event-related potentials to bimodal stimuli

Jia Guo; Peng Xu; Li Yao; Hua Shu; Xiaojie Zhao

doi:10.1117/12.905842

20 February 2012 The oscillatory activities and its synchronization in auditory-visual integration as revealed by event-related potentials to bimodal stimuli

Jia Guo, Peng Xu, Li Yao, Hua Shu, Xiaojie Zhao

Author Affiliations +

Proceedings Volume 8291, Human Vision and Electronic Imaging XVII; 82911P (2012) https://doi.org/10.1117/12.905842
Event: IS&T/SPIE Electronic Imaging, 2012, Burlingame, California, United States

Abstract

Neural mechanism of auditory-visual speech integration is always a hot study of multi-modal perception. The articulation conveys speech information that helps detect and disambiguate the auditory speech. As important characteristic of EEG, oscillations and its synchronization have been applied to cognition research more and more. This study analyzed the EEG data acquired by unimodal and bimodal stimuli using time frequency and phase synchrony approach, investigated the oscillatory activities and its synchrony modes behind evoked potential during auditory-visual integration, in order to reveal the inherent neural integration mechanism under these modes. It was found that beta activity and its synchronization differences had relationship with gesture N1-P2, which happened in the earlier stage of speech coding to pronouncing action. Alpha oscillation and its synchronization related with auditory N1-P2 might be mainly responsible for auditory speech process caused by anticipation from gesture to sound feature. The visual gesture changing enhanced the interaction of auditory brain regions. These results provided explanations to the power and connectivity change of event-evoked oscillatory activities which matched ERPs during auditory-visual speech integration.

Citation Download Citation

Jia Guo, Peng Xu, Li Yao, Hua Shu, and Xiaojie Zhao "The oscillatory activities and its synchronization in auditory-visual integration as revealed by event-related potentials to bimodal stimuli", Proc. SPIE 8291, Human Vision and Electronic Imaging XVII, 82911P (20 February 2012); https://doi.org/10.1117/12.905842

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available