Abstract
Because dialogue system development involves a variety of factors and requires multifaceted consideration, design guidelines for such development would be helpful. Although a neural-based approach can be used, it requires a vast amount of dialogue data, which would take too much effort to collect in the case of a system for a specific, fixed-length dialogue. Furthermore, the system design should explicitly consider errors in automatic speech recognition and language understanding, because they degrade the user impression and are inevitable when the system talks with general users. Accordingly, we propose design guidelines for developing such dialogue systems. Systems developed with the aid of these guidelines took first place in two dialogue system competitions: the situation track of the second Dialogue System Live Competition and a pre-preliminary contest of the Dialogue Robot Competition. Our proposed design guidelines are to: (1) make the system take initiative, (2) prevent dialogue flows from relying too much on user utterances, and (3) include in utterances that the system understands what the user said. We describe details and examples for the systems designed for each of the two competitions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
https://dialog-system-live-competition.github.io/dslc2/ (written in Japanese).
- 2.
- 3.
A similar guideline, “Avoid system utterances that may induce user questions,” was also listed as a design guideline in our previous framework for developing closed-domain chat dialogue systems [14].
- 4.
This approach is from a lecture given by Dr. Iio before DSLC2. The video (in Japanese) is available at https://dialog-system-live-competition.github.io/dslc2/lecture.html. It is part of the know-how shared in Prof. Ishiguro’s Laboratory at Osaka University, where he previously worked and where several talking robots were developed.
- 5.
- 6.
This is similar to the “MoveOn” strategy [3].
- 7.
The remaining four items with lower correlations were as follows: “The dialogue went well,” “The system was polite,” “The system was friendly,” and “The system did not often change the topic.” These items were of relatively lower importance, given the current system performance.
References
Adiwardana D, Luong M, So DR, Hall J, Fiedel N, Thoppilan R, Yang Z, Kulshreshtha A, Nemade G, Lu Y, Le QV (2020) Towards a human-like open-domain chatbot. CoRR arXiv:2001.09977
Bernsen NO, Dybkjaer H, Dybkjaer L (1996) Principles for the design of cooperative spoken human-machine dialogue. In: Proceedings of international conference on spoken language processing (ICSLP), vol 2, pp 729–732. https://doi.org/10.1109/ICSLP.1996.607465
Bohus D, Rudnicky AI (2005) Sorry, I didn’t catch that! - an investigation of non-understanding errors and recovery strategies. In: Proceedings of SIGdial workshop on discourse and dialogue, pp 128–143
Clark HH (1996) Using language. Cambridge University Press
Deibel D, Evanhoe R (2021) Conversations with things: UX design for chat and voice. Conversations with things: UX design for chat and voice, Rosenfeld media. https://books.google.co.jp/books?id=BlouzgEACAAJ
Grice HP (1975) Logic and conversation. In: Cole P, Morgan JL (eds) Syntax and semantics: Vol. 3: speech acts. Academic, New York, pp 41–58. http://www.ucl.ac.uk/ls/studypacks/Grice-Logic.pdf
Hardy A, Paranjape A, Manning C (2021) Effective social chatbot strategies for increasing user initiative. In: Proceedings of annual meeting of the special interest group on discourse and dialogue (SIGDIAL), pp 99–110. https://aclanthology.org/2021.sigdial-1.11
Higashinaka R, Funakoshi K, Inaba M, Tsunomori Y, Takahashi T, Akama R (2019) Dialogue system live competition: identifying problems with dialogue systems through live event. In: Marchi E, Siniscalchi SM, Cumani S, Salerno VM, Li H (eds) Proceedings of international workshop on spoken dialogue system technology (IWSDS). Lecture notes in electrical engineering, vol 714. Springer, pp 185–199. https://doi.org/10.1007/978-981-15-9323-9_16
Hu S, Liu Y, Gottardi A, Hedayatnia B, Khatri A, Chadha A, Chen Q, Rajan P, Binici A, Somani V, Lu Y, Dwivedi P, Hu L, Shi H, Sahai S, Eric M, Gopalakrishnan K, Kim S, Gella S, Papangelis A, Lange P, Jin D, Chartier N, Namazifar M, Padmakumar A, Ghazarian S, Oraby S, Narayan-Chen A, Du Y, Stubell L, Stiff S, Bland K, Mandal A, Ghanadan R, Hakkani-Tur D (2021) Further advances in open domain dialog systems in the fourth alexa prize socialbot grand challenge. In: Alexa prize proceedings
Kobori T, Nakano M, Nakamura T (2016) Small talk improves user impressions of interview dialogue systems. In: Proc. Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pp 370–380, https://www.aclweb.org/anthology/W16-3646
Komatani K, Ueno S, Kawahara T, Okuno HG (2005) User modeling in spoken dialogue systems to generate flexible guidance. User Model User-Adap Interact 15(1):169–183
Komeili M, Shuster K, Weston J (2021) Internet-augmented dialogue generation. CoRR arXiv:2107.07566
Langevin R, Lordon RJ, Avrahami T, Cowan BR, Hirsch T, Hsieh G (2021) Heuristic evaluation of conversational agents. In: Proceedings of conference on human factors in computing systems (CHI). https://doi.org/10.1145/3411764.3445312
Nakano M, Komatani K (2020) A framework for building closed-domain chat dialogue systems. Knowl-Based Syst 204:106212. https://doi.org/10.1016/j.knosys.2020.106212
Niculescu A (2011) Conversational interfaces for task-oriented spoken dialogues: design aspects influencing interaction quality. PhD thesis, University of Twente. https://research.utwente.nl/en/publications/conversational-interfaces-for-task-oriented-spoken-dialogues-desi, sIKS Dissertation Series ; no. 2011-49
Nielsen J, Molich R (1990) Heuristic evaluation of user interfaces. In: Proceedings of conference on human factors in computing systems (CHI), pp 249–256. https://doi.org/10.1145/97243.97281
Pearl C (2016) Designing voice user interfaces: principles of conversational experiences, 1st edn. O’Reilly Media, Inc
Roller S, Dinan E, Goyal N, Ju D, Williamson M, Liu Y, Xu J, Ott M, Smith EM, Boureau YL, Weston J (2021) Recipes for building an open-domain chatbot. In: Proceedings of European chapter of the association for computational linguistics (EACL), pp 300–325. https://aclanthology.org/2021.eacl-main.24
Sturm J, den Os E, Boves L (1999) Issues in spoken dialogue systems: Experiences with the Dutch ARISE system. In: Proceedings ESCA workshop on interactive dialogue in multi-modal systems, Kloster Irsee, Germany, pp 1–4
Xu J, Szlam A, Weston J (2021) Beyond goldfish memory: Long-term open-domain conversation. CoRR arXiv:2107.07567
Yang X, Aurisicchio M (2021) Designing conversational agents: a self-determination theory approach. In: Proceedings of conference on human factors in computing systems (CHI). https://doi.org/10.1145/3411764.3445445
Acknowledgements
This work was partly supported by JSPS KAKENHI Grant Numbers JP19H05692 and JP19H04171, and JST, PRESTO Grant Number JPMJPR1857, Japan.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Komatani, K., Takeda, R., Nakashima, K., Nakano, M. (2022). Design Guidelines for Developing Systems for Dialogue System Competitions. In: Stoyanchev, S., Ultes, S., Li, H. (eds) Conversational AI for Natural Human-Centric Interaction. Lecture Notes in Electrical Engineering, vol 943. Springer, Singapore. https://doi.org/10.1007/978-981-19-5538-9_11
Download citation
DOI: https://doi.org/10.1007/978-981-19-5538-9_11
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-5537-2
Online ISBN: 978-981-19-5538-9
eBook Packages: Computer ScienceComputer Science (R0)