Many studies on emotion processing considered little about the issue of ecological validity and insufficient attention has been drawn to uni-sensory and multisensory emotion perception in challenging environments. The current research explored how adding multi-talker babble noise impacts emotion perception and how visual information affects the results in comparison with the audio alone conditions. Forty native Mandarin participants (21 females and 19 males) were asked to identify the emotion according to the auditory or audiovisual information they received. Results showed that the emotion identification accuracy was significantly lower in noisy conditions than in noiseless ones, whether additional visual information was presented simultaneously or not. In noisy environments, providing multisensory emotional information greatly facilitated recognition performances even when the visual information was less reliable. To conclude, multi-talker babble noise had a corrupting effect on emotion identification, which worked in both unisensory and multisensory settings, and emotion perception is a robust multisensory situation that follows the inverse effectiveness principle.
Cite as: Zhang, M., Ding, H. (2022) Impact of Background Noise and Contribution of Visual Information in Emotion Identification by Native Mandarin Speakers. Proc. Interspeech 2022, 1993-1997, doi: 10.21437/Interspeech.2022-10142
@inproceedings{zhang22r_interspeech,
author={Minyue Zhang and Hongwei Ding},
title={{Impact of Background Noise and Contribution of Visual Information in Emotion Identification by Native Mandarin Speakers}},
year=2022,
booktitle={Proc. Interspeech 2022},
pages={1993--1997},
doi={10.21437/Interspeech.2022-10142}
}