Akira Morikawa
ABSTRACT
A major hurdle in achieving a dialogue system that enables smooth dialogue is to determine how to generate an appropriate response to a user's utterance. Previous research has focused mainly on estimating whether to make an utterance backchannel in response to the user's utterance. We go one step further by examining, for the first time, the relationship between the type of utterance backchannel to be used and intent and type of the speaker's utterance, known as a dialogue act (DA). Specifically, we propose a new method for classifying utterance backchannels into nine types. We also created a corpus consisting of the DAs of speaker utterances and the backchannel types of listener utterances then used it to analyze the relationship between a speaker's and listener's utterances. Our findings clarify that the occurrence frequencies of a listener's backchannel types significantly depend on the DAs of the speaker's utterances. Since the goal of our research is to construct a dialogue system that generates a more natural backchannel, this classification method, which determines certain types of aids from the speaker's DA, will be beneficial to such a system.
- Shinya Fujie, Kenta Fukushima, and Tetsunori Kobayashi. 2005. Back-channel feedback generation using linguistic and nonlinguistic information and its application to spoken dialogue system. In INTERSPEECH. 889--892.Google Scholar
- Kohei Hara, Koji Inoue, Katsuya Takanashi, and Tatsuya Kawahara. 2018. Prediction of Turn-taking Using Multitask Learning with Prediction of Backchannels and Fillers. In INTERSPEECH. 991--995.Google Scholar
- Ryuichiro Higashinaka, Kenji Imamura, Toyomi Meguro, Chiaki Miyazaki, Nozomi Kobayashi, Hiroaki Sugiyama, Toru Hirano, Toshiro Makino, and Yoshihiro Matsuo. 2014. Towards an open-domain conversational system fully based on natural language processing. In International conference on Computational linguistics. 928--939.Google Scholar
- Lixing Huang, Louis-Philippe Morency, and Jonathan Gratch. 2010. Parasocial consensus sampling: Combining multiple perspectives to learn virtual human behavior. AAMAS 2, 1265--1272.Google Scholar
- Ryo Ishii, Ryuichiro Higashinaka, and Junji Tomita. 2018. Predicting Nods by using Dialogue Acts in Dialogue. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).Google Scholar
- Ryo Ishii, Xutong Ren, Michal Muszynski, and Louis-Philippe Morency. 2021. Multimodal and Multitask Approach to Listener's Backchannel Prediction: Can Prediction of Turn-Changing and Turn-Management Willingness Improve Backchannel Modeling?. In Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents. 131--138.Google ScholarDigital Library
- Hanae Koiso, Yasuo Horiuchi, Syun Tutiya, Akira Ichikawa, and Yasuharu Den. 1998. An Analysis of Turn-Taking and Backchannels Based on Prosodic and Syntactic Features in Japanese Map Task Dialogs. In Language and Speech, Vol. 41. 295--321.Google ScholarCross Ref
- SENKO K. MAYNARD. 1986. On back-channel behavior in Japanese and English casual conversation. 24, 6 (1986), 1079--1108. Google ScholarCross Ref
- Toyomi Meguro, Ryuichiro Higashinaka, Yasuhiro Minami, and Kohji Dohsaka. 2010. Controlling listening-oriented dialogue using partially observable Markov decision processes. In International conference on computational linguistics. 761--769.Google ScholarDigital Library
- Louis-Philippe Morency, Iwan de Kok, and Jonathan Gratch. 2008. Predicting Listener Backchannels: A Probabilistic Multimodal Approach. In IVA. 176--190.Google Scholar
- Markus Mueller, David Leuschner, Lars Briem, Maria Schmidt, Kevin Kilgour, Sebastian Stueker, and Alex Waibel. 2015. Using Neural Networks for Data-Driven Backchannel Prediction: A Survey on Input Features and Training Techniques. In Human-Computer Interaction: Interaction Technologies. 329--340.Google Scholar
- Chiharu Mukai. 1999. The Use of Back-channels by Advanced Learners of Japanese : Its Qualitative and Quantitative Aspects.Google Scholar
- Kazumi Ohira. 1998. Have you changed? Pragmatic transfer of back-channel behavior by Japanese bilingual speakers. University of Illinois at Urbana-Champaign.Google Scholar
- Robin Ruede, Markus Müller, Sebastian Stüker, and Alex Waibel. 2019. Yeah, Right, Uh-Huh: A Deep Learning Backchannel Predictor. 247--258.Google Scholar
- Khiet P. Truong, Ronald Poppe, and Dirk Heylen. 2010. A rule-based backchannel prediction model using pitch and pause information.. In INTERSPEECH. ISCA.Google Scholar
- N. Ward. 1996. Using prosodic clues to decide when to produce back-channel utterances. In Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96, Vol. 3. 1728--1731 vol.3.Google ScholarCross Ref
- Nigel Ward and Wataru Tsukahara. 2000. Prosodic features which cue backchannel responses in English and Japanese. Journal of Pragmatics 32, 8 (2000), 1177--1207.Google ScholarCross Ref
- Takashi Yamaguchi, Koji Inoue, Koichiro Yoshino, Katsuya Takanashi, Nigel G Ward, and Tatsuya Kawahara. 2016. Analysis and prediction of morphological patterns of backchannels for attentive listening agents. In Proc. 7th International Workshop on Spoken Dialogue Systems. 1--12.Google Scholar
Index Terms
- Determining most suitable listener backchannel type for speaker's utterance
Recommendations
Multimodal and Multitask Approach to Listener's Backchannel Prediction: Can Prediction of Turn-changing and Turn-management Willingness Improve Backchannel Modeling?
IVA '21: Proceedings of the 21st ACM International Conference on Intelligent Virtual AgentsThe listener's backchannel has the important function of encouraging a current speaker to hold their turn and continue to speak, which enables smooth conversation. The listener monitors the speaker's turn-management (a.k.a. speaking and listening) ...
Prediction of Various Backchannel Utterances Based on Multimodal Information
IVA '23: Proceedings of the 23rd ACM International Conference on Intelligent Virtual AgentsThe listener's backchannels are an important part of dialogues. With appropriate backchannels, people are able to smoothly promote dialogues. Thus, backchannels are considered to be important in dialogues between not only humans but also humans and ...
(Semi-)Automatic Analysis of Dialogues
ICAART 2014: Proceedings of the 6th International Conference on Agents and Artificial Intelligence - Volume 1We study human-human dialogues and human-computer dialogues with the aim to determine which dialogue acts and communicative strategies do the participants of interaction use, and which structural parts does a dialogue include. In order to simplify the ...
Comments