research-article

Miscommunication Detection and Recovery in Situated Human–Robot Dialogue

Authors:

Alexander I. RudnickyAuthors Info & Claims

ACM Transactions on Interactive Intelligent Systems (TiiS), Volume 9, Issue 1

Article No.: 3, Pages 1 - 40

https://doi.org/10.1145/3237189

Published: 17 February 2019 Publication History

Abstract

Even without speech recognition errors, robots may face difficulties interpreting natural-language instructions. We present a method for robustly handling miscommunication between people and robots in task-oriented spoken dialogue. This capability is implemented in TeamTalk, a conversational interface to robots that supports detection and recovery from the situated grounding problems of referential ambiguity and impossible actions. We introduce a representation that detects these problems and a nearest-neighbor learning algorithm that selects recovery strategies for a virtual robot. When the robot encounters a grounding problem, it looks back on its interaction history to consider how it resolved similar situations. The learning method is trained initially on crowdsourced data but is then supplemented by interactions from a longitudinal user study in which six participants performed navigation tasks with the robot. We compare results collected using a general model to user-specific models and find that user-specific models perform best on measures of dialogue efficiency, while the general model yields the highest agreement with human judges. Our overall contribution is a novel approach to detecting and recovering from miscommunication in dialogue by including situated context, namely, information from a robot’s path planner and surroundings.

References

[1]

Anne H. Anderson, Miles Bader, Ellen Gurman Bard, Elizabeth Boyle, Gwyneth Doherty, Simon Garrod, Stephen Isard, Jacqueline Kowtko, Jan McAllister, Jim Miller, Catherine Sotillo, Henry S. Thompson, and Regina Weinert. 1991. The HCRC map task corpus. Language and Speech 34, 4 (1991), 351--366.

[2]

Giulio Antoniol, Bruno Caprile, Alessandro Cimatti, and Roberto Fiutem. 1994. Experiencing real-life interactions with the experimental platform of MAIA. In Proceedings of the 1st European Workshop on Human Comfort and Security.

[3]

Yoav Artzi and Luke Zettlemoyer. 2013. Weakly supervised learning of semantic parsers for mapping instructions to actions. Transactions of the Association for Computational Linguistics 1 (2013), 49--62.

[4]

Hideki Asoh, Yoichi Motomura, Futoshi Asano, Isao Hara, Satoru Hayamizu, Katsunobu Itou, Takio Kurita, Toshihiro Matsui, Nikos Vlassis, Roland Bunschoten, and Ben Kröse. 2001. Jijo-2: An office robot that communicates and learns. IEEE Intelligent Systems 16, 5 (2001), 46--55.

Digital Library

[5]

Stephen Balakirsky, Chris Scrapper, and Elena Messina. 2005. Mobility open architecture simulation and tools environment. In Proceedings of the Knowledge Intensive Multi-Agent Systems Conference (KIMAS’05).

[6]

Dhananjay Bansal and Mosur K. Ravishankar. 1998. New features for confidence annotation. In Proceedings of the 5th International Conference on Spoken Language Processing (ICSLP’98).

[7]

Emanuele Bastianelli, Giuseppe Castellucci, Danilo Croce, Roberto Basili, and Daniele Nardi. 2017. Structured learning for spoken language understanding in human-robot interaction. The International Journal of Robotics Research 36, 5--7 (2017), 660--683.

Digital Library

[8]

Samuel Bayer, Christine Doran, and Bryan George. 2001. Exploring speech-enabled dialogue with the Galaxy communicator infrastructure. In Proceedings of the 1st International Conference on Human Language Technology Research (HLT’01).

Digital Library

[9]

Yonatan Bisk, Deniz Yuret, and Daniel Marcu. 2016. Natural language communication with robots. In Proceedings of NAACL-HLT’16.

[10]

Dan Bohus. 2007. Error Awareness and Recovery in Conversational Spoken Language Interfaces. Ph.D. Dissertation. Carnegie Mellon University, Pittsburgh, PA.

Digital Library

[11]

Dan Bohus and Eric Horvitz. 2010. On the challenges and opportunities of physically situated dialog. In Proceedings of the AAAI Fall Symposium on Dialog with Robots.

[12]

Dan Bohus, Antoine Raux, Thomas K. Harris, Maxine Eskenazi, and Alexander I. Rudnicky. 2007. Olympus: An open-source framework for conversational spoken language interface research. In Proceedings of the NAACL-HLT Workshop on Bridging the Gap: Academic and Industrial Research in Dialog Technologies.

Digital Library

[13]

Dan Bohus and Alexander I. Rudnicky. 2002. Integrating Multiple Knowledge Sources for Utterance-Level Confidence Annotation in the CMU Communicator Spoken Dialog System. Technical Report. Carnegie Mellon University, Pittsburgh, PA.

[14]

Dan Bohus and Alexander I. Rudnicky. 2009. The RavenClaw dialog management framework: Architecture and systems. Computer Speech and Language 23, 3 (2009), 332--361.

Digital Library

[15]

Dan Bohus, Chit W. Saw, and Eric Horvitz. 2014. Directions robot: In-the-wild experiences and lessons learned. In Proceedings of the 13th International Conference on Autonomous Agents and Multiagent Systems (AAMAS’14).

Digital Library

[16]

Abdeslam Boularias, Felix Duvallet, Jean Oh, and Anthony Stentz. 2016. Learning qualitative spatial relations for robotic navigation. In Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI’16).

Digital Library

[17]

Gies Bouwman, Janienke Sturm, and Louis Boves. 1999. Incorporating confidence measures in the Dutch train timetable information system developed in the ARISE project. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’99).

Digital Library

[18]

Jack E. Bresenham. 1965. Algorithm for computer control of a digital plotter. IBM Systems Journal 4, 1 (1965), 25--30.

Digital Library

[19]

Alexander Broad, Jacob Arkin, Nathan Ratliff, Thomas Howard, and Brenna Argall. 2017. Real-time natural language corrections for assistive robotic manipulators. The International Journal of Robotics Research 36, 5--7 (2017), 684--698.

Digital Library

[20]

Guido Bugmann, Ewan Klein, Stanislao Lauria, and Theocharis Kyriacou. 2004. Corpus-based robotics: A route instruction example. In Proceedings of Intelligent Autonomous Systems.

[21]

Cesar Cadena, Luca Carlone, Henry Carrillo, Yasir Latif, Davide Scaramuzza, José Neira, Ian Reid, and John J. Leonard. 2016. Past, present, and future of simultaneous localization and mapping: Towards the robust-perception age. IEEE Transactions on Robotics 32, 6 (2016), 1309--1332.

Digital Library

[22]

Stefano Carpin, Mike Lewis, Jijun Wang, Stephen Balakirsky, and Chris Scrapper. 2007. USARSim: A robot simulator for research and education. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’07).

[23]

Iñigo Casanueva, Thomas Hain, Heidi Christensen, Ricard Marxer, and Phil Green. 2015. Knowledge transfer between speakers for personalised dialogue management. In Proceedings of the 16th Annual SIGdial Meeting on Discourse and Dialogue (SIGdial’15).

[24]

Joyce Y. Chai, Rui Fang, Changsong Liu, and Lanbo She. 2016. Collaborative language grounding toward situated human-robot dialogue. AI Magazine 37, 4 (2016), 32--45.

[25]

Lin Chase. 1997. Error-Responsive Feedback Mechanisms for Speech Recognizers. Ph.D. Dissertation. Carnegie Mellon University, Pittsburgh, PA.

Digital Library

[26]

David L. Chen and Raymond J. Mooney. 2011. Learning to interpret natural language navigation instructions from observations. In Proceedings of the 25th AAAI Conference on Artificial Intelligence (AAAI’11).

Digital Library

[27]

Herbert H. Clark. 1996. Using Language. Cambridge University Press, New York, NY.

[28]

Herbert H. Clark and Susan E. Brennan. 1991. Grounding in communication. Perspectives on Socially Shared Cognition. 127--149.

[29]

Stephen Cox and Richard Rose. 1996. Confidence measures for the SWITCHBOARD database. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’96).

Digital Library

[30]

Robin Deits, Stefanie Tellex, Pratiksha Thaker, Dimitar Simeonov, Thomas Kollar, and Nicholas Roy. 2013. Clarifying commands with information-theoretic human-robot dialog. Journal of Human-Robot Interaction 2, 2 (2013), 58--79.

Digital Library

[31]

Felix Duvallet, Matthew R. Walter, Thomas Howard, Sachithra Hemachandra, Jean Oh, Seth Teller, Nicholas Roy, and Anthony Stentz. 2016. Inferring maps and behaviors from natural language instructions. In Proceedings of the International Symposium on Experimental Robotics (ISER’16).

[32]

Juraj Dzifcak, Matthias Scheutz, Chitta Baral, and Paul Schermerhorn. 2009. What to do and how to do it: Translating natural language directives into temporal and dynamic logic representation for goal management and action execution. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’09).

Digital Library

[33]

Juan Fasola and Maja J. Matarić. 2013. Using semantic fields to model dynamic spatial relations in a robot architecture for natural language instruction of service robots. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’13).

[34]

Terrence Fong, Charles Thorpe, and Charles Baur. 2003. Collaboration, dialogue, and human-robot interaction. In Robotics Research. 255--266.

[35]

Mary Ellen Foster, Rachid Alami, Olli Gestranius, Oliver Lemon, Marketta Niemelä, Jean-Marc Odobez, and Amit Kumar Pandey. 2016. The MuMMER project: Engaging human-robot interaction in real-world public spaces. In Proceedings of the International Conference on Social Robotics (ICSR’16).

[36]

Mary Ellen Foster, Manuel Giuliani, Amy Isard, Colin Matheson, Jon Oberlander, and Alois Knoll. 2009. Evaluating description and reference strategies in a cooperative human-robot dialogue system. In Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI’09).

Digital Library

[37]

Benjamin Fransen, Vlad Morariu, Eric Martinson, Samuel Blisard, Matthew Marge, Scott Thomas, Alan Schultz, and Dennis Perzanowski. 2007. Using vision, acoustics, and natural language for disambiguation. In Proceedings of the 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI’07).

Digital Library

[38]

Lei Gao. 2005. Latin Squares in Experimental Design. Technical Report. Michigan State University, East Lansing, MI.

[39]

Genevieve Gorrell, Ian Lewin, and Manny Rayner. 2002. Adding intelligent help to mixed-initiative spoken dialogue systems. In Proceedings of the 7th International Conference on Spoken Language Processing (ICSLP’02).

[40]

Edward T. Hall. 1966. The Hidden Dimension. Doubleday 8 Co., New York, NY.

[41]

Eui-Hong Han, George Karypis, and Vipin Kumar. 2001. Text categorization using weight adjusted k-nearest neighbor classification. In Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD’01): Advances in Knowledge Discovery and Data Mining.

Digital Library

[42]

Stevan Harnad. 1990. The symbol grounding problem. Physica D: Nonlinear Phenomena 42 (1990), 335--346.

Digital Library

[43]

Thomas K. Harris and Alexander I. Rudnicky. 2007. TeamTalk: A platform for multi-human-robot dialog research in coherent real and virtual spaces. In Proceedings of the 22nd National Conference on Artificial Intelligence (AAAI’07).

Digital Library

[44]

David A. Harville. 1977. Maximum likelihood approaches to variance component estimation and to related problems. J. Amer. Statist. Assoc. 72, 358 (1977), 320--338.

[45]

Sachithra Hemachandra, Felix Duvallet, Thomas M. Howard, Nicholas Roy, Anthony Stentz, and Matthew R. Walter. 2015. Learning models for following natural language directions in unknown environments. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’15).

[46]

Julian Hough and David Schlangen. 2016. Investigating fluidity for human-robot interaction with real-time, real-world grounding strategies. In Proceedings of the 17th Annual SIGdial Meeting on Discourse and Dialogue (SIGdial’16).

[47]

David Huggins-Daines, Mohit Kumar, Arthur Chan, Alan W. Black, Mosur Ravishankar, and Alexander I. Rudnicky. 2006. PocketSphinx: A free, real-time continuous speech recognition system for hand-held devices. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’06).

[48]

Ross A. Knepper, Stefanie Tellex, Adrian Li, Nicholas Roy, and Daniela Rus. 2015. Recovering from failure by asking for help. Autonomous Robots 39, 3 (2015), 347--362.

Digital Library

[49]

Holger Knublauch, Ray W. Fergerson, Natalya F. Noy, and Mark A. Musen. 2004. The Protégé OWL plugin: An open development environment for semantic web applications. In Proceedings of the International Semantic Web Conference (ISWC’04).

Digital Library

[50]

Thomas Kollar, Stefanie Tellex, Deb Roy, and Nicholas Roy. 2010. Toward understanding natural language directions. In Proceedings of the 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI’10).

Digital Library

[51]

Theodora Koulouri and Stanislao Lauria. 2009. A corpus-based analysis of route instructions in human-robot interaction. In Proceedings of Towards Autonomous Robotic Systems (TAROS’09).

[52]

Emiel Krahmer, Marc Swerts, Mariet Theune, and Mieke Weegels. 2001. Error detection in spoken human-machine interaction. International Journal of Speech Technology 4 (2001), 19--30.

[53]

Jayant Krishnamurthy and Thomas Kollar. 2013. Jointly learning to parse and perceive: Connecting natural language to the physical world. Transactions of the Association for Computational Linguistics 1 (2013), 193--206.

[54]

Geert-Jan M. Kruijff, Pierre Lison, Trevor Benjamin, Henrik Jacobsson, and Nick A. Hawes. 2007. Incremental, multi-level processing for comprehending situated dialogue in human-robot interaction. In Proceedings of the Symposium on Language and Robots.

[55]

Geert-Jan M. Kruijff, Pierre Lison, Trevor Benjamin, Henrik Jacobsson, Hendrik Zender, Ivana Kruijff-Korbayová, and Nick Hawes. 2010. Situated dialogue processing for human-robot interaction. Cognitive Systems (2010), 311--364.

[56]

Geert-Jan M. Kruijff, Hendrik Zender, Patric Jensfelt, and Henrik I. Christensen. 2007. Situated dialogue and spatial organization: What, where…and why? International Journal of Advanced Robotic Systems 4, 1 (2007), 125--138.

[57]

Stanislao Lauria, Guido Bugmann, Theocharis Kyriacou, Johan Bos, and Ewan Klein. 2002. Converting natural language route instructions into robot executable procedures. In Proceedings of the 11th IEEE International Workshop on Robot and Human Interactive Communication (RO-MAN’02).

[58]

Séverin Lemaignan, Raquel Ros, Lorenz Mösenlechner, Rachid Alami, and Michael Beetz. 2010. ORO, a knowledge management platform for cognitive architectures in robotics. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2’0).

[59]

Séverin Lemaignan, Raquel Ros, E. Akin Sisbot, Rachid Alami, and Michael Beetz. 2012. Grounding the interaction: Anchoring situated discourse in everyday human-robot interaction. International Journal of Social Robotics 4, 2 (2012), 181--199.

[60]

Oliver Lemon, Anne Bracy, Alexander Gruenstein, and Stanley Peters. 2001. A multi-modal dialogue system for human-robot conversation. In Proceedings of the 2nd North American Chapter of the Association for Computational Linguistics Conference (NAACL’01).

[61]

Patrick Lester. 2005. A* Pathfinding for Beginners. https://www.gamedev.net/articles/programming/artificial-intelligence/a-pathfinding-for-beginners-r2003/.

[62]

Michael Levit and Deb Roy. 2007. Interpretation of spatial language in a map navigation task. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 37, 3 (2007), 667--679.

Digital Library

[63]

Diane J. Litman, Julia B. Hirschberg, and Marc Swerts. 2000. Predicting automatic speech recognition performance using prosodic cues. In Proceedings of the 1st North American Chapter of the Association for Computational Linguistics Conference (NAACL’00).

Digital Library

[64]

Lorenzo Lucignano, Francesco Cutugno, Silvia Rossi, and Alberto Finzi. 2013. A dialogue system for multimodal human-robot interaction. In Proceedings of the 15th ACM International Conference on Multimodal Interaction (ICMI’13).

Digital Library

[65]

James MacGlashan, Monica Babeş-Vroman, Marie desJardins, Michael L. Littman, Smaranda Muresan, Shawn Squire, Stefanie Tellex, Dilip Arumugam, and Lei Yang. 2015. Grounding English commands to reward functions. In Proceedings of Robotics: Science and Systems (RSS’15).

[66]

Matt MacMahon, Brian Stankiewicz, and Benjamin Kuipers. 2006. Walk the talk: Connecting language, knowledge, and action in route instructions. In Proceedings of the 21st National Conference on Artificial Intelligence (AAAI’06).

Digital Library

[67]

Matthew Marge. 2015. Miscommunication Detection and Recovery for Spoken Dialogue Systems in Physically Situated Contexts. Ph.D. Dissertation. Carnegie Mellon University, Pittsburgh, PA.

[68]

Matthew Marge, Aasish Pappu, Benjamin Frisch, Thomas K. Harris, and Alexander I. Rudnicky. 2009. Exploring spoken dialog interaction in human-robot teams. In Proceedings of Robots, Games, and Research: Success Stories in USARSim IROS Workshop.

Digital Library

[69]

Matthew Marge, Aaron Powers, Jonathan Brookshire, Trevor Jay, Odest C. Jenkins, and Christopher Geyer. 2011. Comparing heads-up, hands-free operation of ground robots to teleoperation. In Proceedings of Robotics: Science and Systems (RSS’11).

[70]

Matthew Marge and Alexander I. Rudnicky. 2010. Comparing spoken language route instructions for robots across environment representations. In Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGdial’10).

Digital Library

[71]

Matthew Marge and Alexander I. Rudnicky. 2011. The TeamTalk corpus: Route instructions in open spaces. In Proceedings of the RSS Workshop on Grounding Human-Robot Dialog for Spatial Tasks.

[72]

Matthew Marge and Alexander I. Rudnicky. 2011. Towards overcoming miscommunication in situated dialogue by asking questions. In Proceedings of the AAAI Fall Symposium on Building Representations of Common Ground with Intelligent Agents.

[73]

Matthew Marge and Alexander I. Rudnicky. 2015. Miscommunication recovery in physically situated dialogue. In Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGdial’15).

[74]

Cynthia Matuszek, Dieter Fox, and Karl Koscher. 2010. Following directions using statistical machine translation. In Proceedings of the 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI’10).

Digital Library

[75]

Cynthia Matuszek, Evan Herbst, Luke Zettlemoyer, and Dieter Fox. 2013. Learning to parse natural language commands to a robot control system. In Proceedings of the International Symposium on Experimental Robotics (ISER’13).

[76]

Nikolaos Mavridis. 2015. A review of verbal and non-verbal human--robot interactive communication. Robotics and Autonomous Systems 63 (2015), 22--35.

Digital Library

[77]

Nikolaos Mavridis and Deb Roy. 2006. Grounded situation models for robots: Where words and percepts meet. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’06).

[78]

Raveesh Meena, José Lopes, Gabriel Skantze, and Joakim Gustafson. 2015. Automatic detection of miscommunication in spoken dialogue systems. In Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGdial’15).

[79]

Raveesh Meena, Gabriel Skantze, and Joakim Gustafson. 2012. A data-driven approach to understanding spoken route directions in human-robot dialogue. In Proceedings of the 13th Annual Conference of the International Speech Communication Association (Interspeech’12).

[80]

Raveesh Meena, Gabriel Skantze, and Joakim Gustafson. 2014. Data-driven models for timing feedback responses in a map task dialogue system. Computer Speech 8 Language 28, 4 (2014), 903--922.

[81]

Hongyuan Mei, Mohit Bansal, and Matthew R. Walter. 2016. Listen, attend, and walk: Neural mapping of navigational instructions to action sequences. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI’16).

Digital Library

[82]

Tom M. Mitchell. 1997. Machine Learning. McGraw Hill, New York, NY.

Digital Library

[83]

Marius Muja and David G. Lowe. 2014. Scalable nearest neighbor algorithms for high dimensional data. IEEE Transactions on Pattern Analysis and Machine Intelligence 36, 11 (2014), 2227--2240.

[84]

Aishwarya Padmakumar, Jesse Thomason, and Raymond J. Mooney. 2017. Integrated learning of dialog strategies and semantic parsing. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL’17).

[85]

Tim Paek. 2003. Toward a taxonomy of communication errors. In Proceedings of the ISCA Workshop on Error Handling in Spoken Dialogue Systems.

[86]

Ioannis Papaioannou and Oliver Lemon. 2017. Combining chat and task-based multimodal dialogue for more engaging HRI: A scalable method using reinforcement learning. In Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction (HRI LBR’17).

Digital Library

[87]

Aasish Pappu and Alexander I. Rudnicky. 2012. The structure and generality of spoken route instructions. In Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGdial’12).

Digital Library

[88]

Vittorio Perera and Manuela Veloso. 2015. Handling complex commands as service robot task requests. In Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI’15).

Digital Library

[89]

Dennis Perzanowski, Derek Brock, William Adams, Magdalena Bugajska, Alan C. Schultz, J. Gregory Trafton, Samuel Blisard, and Marjorie Skubic. 2003. Finding the FOO: A pilot study for a multimodal interface. In Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (SMC’03).

[90]

Vasumathi Raman, Constantine Lignos, Cameron Finucane, Kenton C. T. Lee, Mitch Marcus, and Hadas Kress-Gazit. 2013. Sorry Dave, I’m afraid I can’t do that: Explaining unachievable robot tasks using natural language. In Proceedings of Robotics: Science and Systems (RSS’13).

[91]

Antoine Raux and Maxine Eskenazi. 2004. Non-native users in the Let’s Go&excl;&excl; Spoken dialogue system: Dealing with linguistic mismatch. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics (HLT-NAACL’04).

[92]

Antoine Raux and Maxine Eskenazi. 2008. Optimizing endpointing thresholds using dialogue features in a spoken dialogue system. In Proceedings of the 9th SIGdial Workshop on Discourse and Dialogue (SIGdial’08).

Digital Library

[93]

Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. In Proceedings of the Neural Information Processing Systems Conference (NIPS’15).

Digital Library

[94]

Stephanie Rosenthal, Joydeep Biswas, and Manuela Veloso. 2010. An effective personal mobile robot agent through symbiotic human-robot interaction. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems (AAMAS’10).

Digital Library

[95]

Alexander I. Rudnicky, Aasish Pappu, Peng Li, and Matthew Marge. 2010. Instruction taking in the TeamTalk system. In Proceedings of the AAAI Fall Symposium on Dialog with Robots.

[96]

Rubén San-Segundo, Bryan Pellom, Wayne Ward, and José M. Pardo. 2000. Confidence measures for dialogue management in the CU communicator system. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’00).

Digital Library

[97]

Matthias Scheutz, Rehj Cantrell, and Paul Schermerhorn. 2011. Toward humanlike task-based dialogue processing for human robot interaction. AI Magazine 32, 4 (2011), 77--84.

Digital Library

[98]

David Schlangen. 2004. Causes and strategies for requesting clarification in dialogue. In Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue (SIGdial’04).

[99]

Michael F. Schober. 2009. Spatial dialogue between partners with mismatched abilities. Spatial Language and Dialogue. 23--39.

[100]

Stephanie Schuldes, Michael Roth, Anette Frank, and Michael Strube. 2009. Creating an annotated corpus for generating walking directions. In Proceedings of the ACL-IJCNLP Workshop on Language Generation and Summarisation.

Digital Library

[101]

Iulian Vlad Serban, Ryan Lowe, Peter Henderson, Laurent Charlin, and Joelle Pineau. 2018. A survey of available corpora for building data-driven dialogue systems: The journal version. Dialogue and Discourse 9, 1 (2018).

[102]

Hui Shi and Bernd Krieg-Brückner. 2008. Modelling route instructions for robust human-robot interaction on navigation tasks. International Journal of Software and Informatics 2, 1 (2008), 33--60.

[103]

Hui Shi and Thora Tenbrink. 2009. Telling Rolland where to go: HRI dialogues on route navigation. Spatial Language and Dialogue (2009), 177--189.

[104]

Nobuyuki Shimizu and Andrew Haas. 2009. Learning to follow navigational route instructions. In Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI’09).

Digital Library

[105]

Gabriel Skantze. 2005. Exploring human error recovery strategies: Implications for spoken dialogue systems. Speech Communication 45, 3 (2005), 325--341.

[106]

Gabriel Skantze. 2007. Error Handling in Spoken Dialogue Systems: Managing Uncertainty, Grounding and Miscommunication. Ph.D. Dissertation. KTH Royal Institute of Technology, Stockholm, Sweden.

[107]

Gabriel Skantze. 2007. Making grounding decisions: Data-driven estimation of dialogue costs and confidence thresholds. In Proceedings of the 8th SIGdial Workshop on Discourse and Dialogue (SIGdial’07).

[108]

Marjorie Skubic, Dennis Perzanowski, Samuel Blisard, Alan Schultz, William Adams, Magda Bugajska, and Derek Brock. 2004. Spatial language for human-robot dialogs. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 34, 2 (2004), 154--167.

Digital Library

[109]

Dimitris Spiliotopoulos, Ion Androutsopoulos, and Constantine D. Spyropoulos. 2001. Human-robot interaction based on spoken natural language dialogue. In Proceedings of the European Workshop on Service and Humanoid Robots.

[110]

Laura Stoia, Darla Magdalene Shockley, Donna K. Byron, and Eric Fosler-Lussier. 2008. SCARE: A situated corpus with annotated referring expressions. In Proceedings of the Language Resources and Evaluation Conference (LREC’08).

[111]

Stefanie Tellex, Thomas Kollar, Steven Dickerson, Matthew R. Walter, Ashis Gopal Banerjee, Seth Teller, and Nicholas Roy. 2011. Understanding natural language commands for robotic navigation and mobile manipulation. In Proceedings of the 25th AAAI Conference on Artificial Intelligence (AAAI’11).

Digital Library

[112]

Thora Tenbrink, Robert J. Ross, Kavita E. Thomas, Nina Dethlefs, and Elena Andonova. 2010. Route instructions in map-based human-human and human-computer dialogue: A comparative analysis. Journal of Visual Languages 8 Computing 21, 5 (2010), 292--309.

Digital Library

[113]

Moritz Tenorth and Michael Beetz. 2013. KnowRob: A knowledge processing infrastructure for cognition-enabled robots. The International Journal of Robotics Research 32, 5 (2013), 566--590.

Digital Library

[114]

Jesse Thomason, Shiqi Zhang, Raymond Mooney, and Peter Stone. 2015. Learning to interpret natural language commands through human-robot dialog. In Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI’15).

Digital Library

[115]

Andrea Vanzo, Danilo Croce, Emanuele Bastianelli, Guglielmo Gemignani, Roberto Basili, and Daniele Nardi. 2017. Dialogue with robots to support symbiotic autonomy. In Dialogues with Social Robots. 331--342.

[116]

Leslie Versweyveld. 1998. Voice-controlled surgical robot ready to assist in minimally invasive heart surgery. Virtual Medical Worlds Monthly.

[117]

Adam Vogel and Dan Jurafsky. 2010. Learning to follow navigational directions. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL’10).

Digital Library

[118]

Marilyn A. Walker, Diane J. Litman, Candace A. Kamm, and Alicia Abella. 1998. Evaluating spoken dialogue agents with PARADISE: Two case studies. Computer Speech 8 Language 12, 4 (1998), 317--347.

[119]

Marilyn A. Walker, Stephen J. Whittaker, Amanda Stent, Preetam Maloor, Johanna Moore, Michael Johnston, and Gunaranjan Vasireddy. 2004. Generation and evaluation of user tailored responses in multimodal dialogue. Cognitive Science 28 (2004), 811--840.

[120]

Marilyn A. Walker, Jerry Wright, and Irene Langkilde. 2000. Using natural language processing and discourse features to identify understanding errors in a spoken dialogue system. In Proceedings of the 17th International Conference on Machine Learning (ICML’00).

Digital Library

[121]

Tom Williams and Matthias Scheutz. 2017. Resolution of referential ambiguity in human-robot dialogue using Dempster-Shafer theoretic pragmatics. In Proceedings of Robotics: Science and Systems (RSS’17).

[122]

Tom Williams, Fereshta Yazdani, Prasanth Suresh, Matthias Scheutz, and Michael Beetz. 2018. Dempster-Shafer theoretic resolution of referential ambiguity. Autonomous Robots (2018), 1--26.

[123]

Teresa Zollo. 1999. A study of human dialogue strategies in the presence of speech recognition errors. In Proceedings of the AAAI Fall Symposium on Psychological Models of Communication in Collaborative Systems.

Cited By

Antony VStiber MHuang CJohal WLemaignan SBrščić DVázquez MCharisi V(2025)Xpress: A System For Dynamic, Context-Aware Robot Facial Expressions using Language ModelsProceedings of the 2025 ACM/IEEE International Conference on Human-Robot Interaction10.5555/3721488.3721605(958-967)Online publication date: 4-Mar-2025
https://dl.acm.org/doi/10.5555/3721488.3721605
Bennett CBae YYoon JKim SWeiss B(2025)Enhancing Human–Agent Interaction via Artificial Agents That Speculate About the FutureFuture Internet10.3390/fi1702005217:2(52)Online publication date: 21-Jan-2025
https://doi.org/10.3390/fi17020052
Vadivukarasi LBabu LHasan DR. RDevi NKarthick L(2024)Nurturing Trust in Human-Robot Interaction and the Crucial Role of Dialogue and Explicit AITrends and Applications in Mechanical Engineering, Composite Materials and Smart Manufacturing10.4018/979-8-3693-1966-6.ch015(237-252)Online publication date: 14-Aug-2024
https://doi.org/10.4018/979-8-3693-1966-6.ch015
Show More Cited By

Index Terms

Miscommunication Detection and Recovery in Situated Human–Robot Dialogue
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Discourse, dialogue and pragmatics
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Natural language interfaces

Recommendations

Applying politeness maxims in social robotics polite dialogue
HRI '12: Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction

An important element of human-robot interaction, as with inter-human interaction, is conversation. Having previously suggested the Gricean maxims as suitable guidelines for social robotics dialogue, we discovered that a preferable alternative set of ...
Human-robot collaborative tutoring using multiparty multimodal spoken dialogue
HRI '14: Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction

In this paper, we describe a project that explores a novel experimental setup towards building a spoken, multi-modally rich, and human-like multiparty tutoring robot. A human-robot interaction setup is designed, and a human-human dialogue corpus is ...
Situated Open World Reference Resolution for Human-Robot Dialogue
HRI '16: The Eleventh ACM/IEEE International Conference on Human Robot Interaction

A robot participating in natural dialogue with a human interlocutor may need to discuss, reason about, or initiate actions concerning dialogue-referenced entities. To do so, the robot must first identify or create new representations for those entities, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Interactive Intelligent Systems

ACM Transactions on Interactive Intelligent Systems Volume 9, Issue 1

March 2019

168 pages

ISSN:2160-6455

EISSN:2160-6463

DOI:10.1145/3312745

Editor:
Michelle X. Zhou
Juji, Inc., USA

Issue’s Table of Contents

Copyright © 2019 ACM.

Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 February 2019

Accepted: 01 October 2018

Revised: 01 August 2018

Received: 01 April 2018

Published in TIIS Volume 9, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

U.S. Army Research Laboratory
Boeing Company, a National Science Foundation Graduate Research Fellowship

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
1,219
Total Downloads

Downloads (Last 12 months)74
Downloads (Last 6 weeks)10

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Antony VStiber MHuang CJohal WLemaignan SBrščić DVázquez MCharisi V(2025)Xpress: A System For Dynamic, Context-Aware Robot Facial Expressions using Language ModelsProceedings of the 2025 ACM/IEEE International Conference on Human-Robot Interaction10.5555/3721488.3721605(958-967)Online publication date: 4-Mar-2025
https://dl.acm.org/doi/10.5555/3721488.3721605
Bennett CBae YYoon JKim SWeiss B(2025)Enhancing Human–Agent Interaction via Artificial Agents That Speculate About the FutureFuture Internet10.3390/fi1702005217:2(52)Online publication date: 21-Jan-2025
https://doi.org/10.3390/fi17020052
Vadivukarasi LBabu LHasan DR. RDevi NKarthick L(2024)Nurturing Trust in Human-Robot Interaction and the Crucial Role of Dialogue and Explicit AITrends and Applications in Mechanical Engineering, Composite Materials and Smart Manufacturing10.4018/979-8-3693-1966-6.ch015(237-252)Online publication date: 14-Aug-2024
https://doi.org/10.4018/979-8-3693-1966-6.ch015
Lee CPraveena PMutlu B(2024)REX: Designing User-centered Repair and Explanations to Address Robot FailuresProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661559(2911-2925)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661559
Zhang RFlathmann CMusick GSchelble BMcNeese NKnijnenburg BDuan W(2023)I Know This Looks Bad, But I Can Explain: Understanding When AI Should Explain Actions In Human-AI TeamsACM Transactions on Interactive Intelligent Systems10.1145/363547414:1(1-23)Online publication date: 2-Dec-2023
https://dl.acm.org/doi/10.1145/3635474
Özkan EHealey PGurion THough JJamone L(2023)Speakers Raise Their Hands and Head During Self-Repairs in Dyadic ConversationsIEEE Transactions on Cognitive and Developmental Systems10.1109/TCDS.2023.325480815:4(1993-2003)Online publication date: Dec-2023
https://doi.org/10.1109/TCDS.2023.3254808
Xiao Y(2023)Application of Multimodal Intelligent Dialogue Robot in Diabetes Health Management Service Platform2023 5th International Conference on Decision Science & Management (ICDSM)10.1109/ICDSM59373.2023.00021(49-52)Online publication date: 3-Mar-2023
https://doi.org/10.1109/ICDSM59373.2023.00021
Bassetti CBlanzieri EBorgo SMarangon S(2023)Towards socially-competent and culturally-adaptive artificial agentsInteraction Studies. Social Behaviour and Communication in Biological and Artificial SystemsInteraction Studies / Social Behaviour and Communication in Biological and Artificial SystemsInteraction Studies10.1075/is.22021.bas23:3(469-512)Online publication date: 21-Apr-2023
https://doi.org/10.1075/is.22021.bas
Guerrero ETewari MKalmi PLindgren H(2023)Forming We-intentions under breakdown situations in human-robot interactionsComputer Methods and Programs in Biomedicine10.1016/j.cmpb.2023.107817242(107817)Online publication date: Dec-2023
https://doi.org/10.1016/j.cmpb.2023.107817
Natarajan MSeraj EAltundas BPaleja RYe SChen LJensen RChang KGombolay M(2023)Human-Robot Teaming: Grand ChallengesCurrent Robotics Reports10.1007/s43154-023-00103-14:3(81-100)Online publication date: 8-Aug-2023
https://doi.org/10.1007/s43154-023-00103-1
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Issue’s Table of Contents