Misrecognized Utterance Identification in Support Systems for Daily Human-to-Human Conversations

Abstract

To support smooth conversation between people, we are developing a support system that provides appropriate conversation topics when the speaker cannot immediately think of a topic. The problem with this system is that speech recognition errors occur frequently. The system may not be able to provide appropriate conversation topics due to speech recognition errors. Various methods have been proposed to prevent speech recognition errors, but these methods are still insufficient for domain free daily conversation. Vocalizations become ambiguous across multiple words in human-to-human daily conversation. We believe that it is necessary to understand how acoustic features change due to ambiguous vocalizations, and how changes in acoustic features affect speech recognition results, and to develop countermeasures based on these findings. This paper discusses the speech utterances characteristics that are prone to speech recognition errors to identify speech recognition errors caused by ambiguity in daily speech. In particular, we focus on F0 and mora, and analyze how these factors differ between correctly and incorrectly recognized speech.

We confirmed that the F0 extraction rate tends to be lower for misrecognized speech parts than for correctly recognized speech, and we confirmed that the mora length tends to be shorter for misrecognized speech parts. Regarding the mora, there were a small number of misrecognized utterances in which the mora was extremely long, and the standard deviation of the mor of the misrecognized utterances tended to be larger than that of the misrecognized utterances. By comparing the transcribed sentences with the recognition result sentences, we analyzed the factors contributing to the above tendency, and found that the phenomenon of vowel voicelessness was promoted in daily conversation, with the absence or shortening of vocalized segments including not only vowels but also consonant parts before and after them, and the phenomenon of voiceless nasal sounds at the end of utterances. When the above voicelessness occurred at word-word boundaries, it was found that the words before and after the boundary could be combined and misrecognized as completely different words. In the future, we would like to quantify these features and propose a misrecognized utterances identification method.