Predicting the intelligibility of noisy recordings is difficult and most current algorithms treat all speech energy as equally important to intelligibility. Our previous work on human perception used a listening test paradigm and correlational analysis to show that some energy is more important to intelligibility than other energy. In this paper, we propose a system called the Bubble Cooperative Network (BCN), which aims to predict important areas of individual utterances directly from clean speech. Given such a prediction, noise is added to the utterance in unimportant regions and then presented to a recognizer. The BCN is trained with a loss that encourages it to add as much noise as possible while preserving recognition performance, encouraging it to identify important regions precisely and place the noise everywhere else. Empirical evaluation shows that the BCN can obscure 97.7% of the spectrogram with noise while maintaining recognition accuracy for a simple speech recognizer that compares a noisy test utterance with a clean reference utterance. The masks predicted by a single BCN on several utterances show patterns that are similar to analyses derived from human listening tests that analyze each utterance separately, while exhibiting better generalization and less context-dependence than previous approaches.
Cite as: Trinh, V.A., McFee, B., Mandel, M.I. (2018) Bubble Cooperative Networks for Identifying Important Speech Cues. Proc. Interspeech 2018, 1616-1620, doi: 10.21437/Interspeech.2018-2377
@inproceedings{trinh18_interspeech,
author={Viet Anh Trinh and Brian McFee and Michael I Mandel},
title={{Bubble Cooperative Networks for Identifying Important Speech Cues}},
year=2018,
booktitle={Proc. Interspeech 2018},
pages={1616--1620},
doi={10.21437/Interspeech.2018-2377}
}