Abstract
This chapter introduces an end user development (EUD) approach for handling common types of failures encountered by goal-oriented conversational agents. We start with identifying three common sources of failures in human-agent conversations: unknown concepts, out-of-domain tasks and wrong fulfillment means or level of generalization in task execution. To handle these failures, it is useful to enable the end user to program the agent and to “teach” the agent what to do as a fallback strategy. Showing examples for this approach, we walk through our two integrated systems: Sugilite and Lia. Sugilite uses the programming by demonstration (PBD) technique, allowing the user to program the agent by demonstrating new tasks or new means for completing a task using the GUIs of third-party smartphone apps, while Lia learns new tasks from verbal instructions, enabling the user to teach the agent through breaking down the procedure verbally. Lia also enables the user to verbally define unknown concepts used in the commands and adds those concepts into the agent’s ontology. Both Sugilite and Lia can generalize what they have learned from the user across related entities and perform a task with new parameters in a different context.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Allen J, Chambers N, Ferguson G, Galescu L, Jung H, Swift M, Taysom W (2007) PLOW: a collaborative task learning agent. In: Proceedings of the 22nd national conference on artificial intelligence. AAAI Press, Vancouver, British Columbia, Canada, vol 2, pp 1514–1519
Armstrong G (2017a) Helping your baby bot learn to chat like a grown up bot. In: Chatbots Magazine
Armstrong G (2017b) The UX of not making your users shout at your Chatbot. In: Chatbots Magazine, Aug 8, 2017. https://chatbotsmagazine.com/difficult-conversations-1-voice-recognition-error-handling-74d93056ddce
Azaria A, Hong J (2016) Recommender systems with personality. In: Proceedings of the 10th ACM conference on recommender systems. ACM, New York, pp 207–210
Azaria A, Krishnamurthy J, Mitchell TM (2016) Instructable intelligent personal agent. In: Proceedings of the 30th AAAI conference on artificial intelligence (AAAI)
Barkin J (2016) When bots fail at conversation. In: Being Janis, Aug 19, 2016. https://medium.com/janis/when-bots-fail-at-conversation-d7419605f5cc
Bohus D, Rudnicky AI (2005a) Constructing accurate beliefs in spoken dialog systems. In: IEEE workshop on automatic speech recognition and understanding, pp 272–277
Bohus D, Rudnicky AI (2005b) Error handling in the RavenClaw dialog management framework. In: Proceedings of the conference on human language technology and empirical methods in natural language processing. Association for Computational Linguistics, pp 225–232
Bohus D, Rudnicky AI (2005c) Sorry, I didn’t catch that!—An investigation of non-understanding errors and recovery strategies. In: 6th SIGdial workshop on discourse and dialogue
Chen J-H, Weld DS (2008) Recovering from errors during programming by demonstration. In: Proceedings of the 13th international conference on intelligent user interfaces. ACM, New York, pp 159–168
Cypher A (1993) Eager: programming repetitive tasks by demonstration. In: Watch what I do. MIT Press, pp 205–217
Cypher A, Halbert DC (1993) Watch what I do: programming by demonstration. MIT Press
Dzikovska MO, Allen JF, Swift MD (2003) Integrating linguistic and domain knowledge for spoken dialogue systems in multiple domains. In: Proceedings of IJCAI-03 workshop on knowledge and reasoning in practical dialogue systems
Gajos K, Weld DS (2004) SUPPLE: automatically generating user interfaces. In: Proceedings of the 9th international conference on Intelligent user interfaces. ACM, pp 93–100
Gruber TR, Cheyer AJ, Kittlaus D, Guzzoni DR, Brigham CD, Giuli RD, Bastea-Forte M, Saddler HJ (2017) Intelligent automated assistant. U.S. Patent 9,548,050
Koedinger KR, Aleven V, Heffernan N, McLaren B, Hockenberry M (2004) Opening the door to non-programmers: authoring intelligent tutor behavior by demonstration. Intelligent tutoring systems. Springer, Berlin, pp 162–174
Leshed G, Haber EM, Matthews T, Lau T (2008) CoScripter: automating & sharing how-to knowledge in the enterprise. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, pp 1719–1728
Li TJ-J (2017) End user mobile task automation using multimodal programming by demonstration. In: 2017 IEEE symposium on Visual Languages and Human-Centric Computing (VL/HCC), pp 323–324
Li TJ-J, Azaria A, Myers BA (2017a) SUGILITE: Creating multimodal smartphone automation by demonstration. In: Proceedings of the 2017 CHI conference on human factors in computing systems. ACM, New York, pp 6038–6049
Li TJ-J, Li Y, Chen F, Myers BA (2017b) Programming IoT devices by demonstration using mobile apps. End-user development. Springer, Cham, pp 3–17
Li TJ-J, Myers BA, Azaria A, Labutov I, Rudnicky AI, Mitchell TM (2017c) Designing a conversational interface for a multimodal smartphone programming-by-demonstration agent. In: Conversational UX Design CHI 2017 Workshop. Denver
Lieberman H (2001) Your wish is my command: programming by example. Morgan Kaufmann
Mitman93 Customer review for Amazon.com: Pizza Hut: Alexa Skills. https://www.amazon.com/Pizza-Hut/dp/B01MSXEBMC. Accessed 5 Oct 2017
Myers BA, Ko AJ, Scaffidi C, Oney S, Yoon Y, Chang K, Kery MB, Li TJ-J (2017) Making end user development more natural. New perspectives in end-user development. Springer, Cham, pp 1–22
Pappu A, Rudnicky A (2014) Knowledge acquisition strategies for goal-oriented dialog systems. In: Proceedings of the 15th annual meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pp 194–198
Srivastava S, Labutov I, Mitchell T (2017) Joint concept learning and semantic parsing from natural language explanations. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 1527–1536
Witten IH (1993) A predictive calculator. In: Watch what I do. MIT Press, pp 67–76
Acknowledgements
This work was supported by Yahoo! through CMU’s InMind project and by Samsung GRO 2015.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Li, T.JJ., Labutov, I., Myers, B.A., Azaria, A., Rudnicky, A.I., Mitchell, T.M. (2018). Teaching Agents When They Fail: End User Development in Goal-Oriented Conversational Agents. In: Moore, R., Szymanski, M., Arar, R., Ren, GJ. (eds) Studies in Conversational UX Design. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-319-95579-7_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-95579-7_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-95578-0
Online ISBN: 978-3-319-95579-7
eBook Packages: Computer ScienceComputer Science (R0)