research-article

Show me how to win: a robot that uses dialog management to learn from demonstrations

Authors:

Alan R. Wagner,

Rebecca J. PassonneauAuthors Info & Claims

FDG '19: Proceedings of the 14th International Conference on the Foundations of Digital Games

Article No.: 78, Pages 1 - 7

https://doi.org/10.1145/3337722.3341866

Published: 26 August 2019 Publication History

Abstract

We present an approach for robot learning from demonstration and communication applied to simple board games like Connect Four. In such games, a visual representation of a winning condition on the board can be converted to an extensive form representation that can then support computation of a winning strategy. We present a robot that can learn simple games from responses to visual questions based on synthesized images, or to verbal questions. We illustrate how reliance on both modalities leads to more efficient learning.

References

[1]

2013. Connect Four Demo: Rethink Robotics. (2013). http://sdk.rethinkrobotics.com/wiki/Connect_Four_Demo.

[2]

Ali Ayub and Alan R Wagner. 2018. Learning to Win Games in a Few Examples: Using Game-Theory and Demonstrations to Learn the Win Conditions of a Connect Four Game. In International Conference on Social Robotics. Springer, 349--358.

[3]

Conway H. J. Berlekamp, E. and R. Guy. 1982. Winning Ways for your Mathematical Plays: Games in general. Academic Press (1982).

[4]

M. Cakmak and A. L. Thomaz. 2012. Designing robot learners that ask good questions. Proceedings of the seventh annual ACM/IEEE International conference on Human-Robot Interaction.

Digital Library

[5]

Joyce Y. Chai, Qiaozi Gao, Lanbo She, Shaohua Yang, Sari Saba-Sadiya, and Guangyue Xu. 2018. Language to Action: Towards Interactive Task Learning with Physical Agents. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18. International Joint Conferences on Artificial Intelligence Organization, 2--9.

Digital Library

[6]

Borghoff U. M. Dobrovsky, A. and M. Hofmann. 2016. An approach to interactive deep reinforcement learning for serious games. 7th IEEE International Conference on Cognitive Infocommunications (CogInfoCom).

[7]

T. R. Hinrichs and K. D. Forbus. 2014. X Goes First : Teaching Simple Games through Multimodal Interaction. Advances in Cognitive Systems 3 (2014), 31--46.

[8]

Francisco S. Melo, Carla Guerra, and Manuel Lopes. 2018. Interactive Optimal Teaching with Unknown Learners. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18. International Joint Conferences on Artificial Intelligence Organization, 2567--2573.

Digital Library

[9]

Shiwali Mohan and John E. Laird. 2014. Learning Goal-oriented Hierarchical Tasks from Situated Interactive Instruction. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI'14). AAAI Press, 387--394. http://dl.acm.org/citation.cfm?id=2893873.2893934

Digital Library

[10]

Kober J. Kroemer O. Mulling, K. and J. Peters. 2013. Learning to select and generalize striking movements in robot table tennis. The International Journal of Robotics Research (IJRR) (2013).

Digital Library

[11]

Mattia Racca and Ville Kyrki. 2018. Active robot learning for temporal task models. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction. ACM, 123--131.

Digital Library

[12]

Mukadam M. Ahmadzadeh S. R. Chernova S. Rana, M. A. and B. Boots. 2017. Towards robust skill generalization: Unifying learning from demonstration and motion planning. Conference on Robot Learning(CoRL).

[13]

Matthias Scheutz, Evan Krause, Brad Oosterveld, Tyler Frasca, and Robert Platt. 2017. Spoken Instruction-Based One-Shot Object and Action Learning in a Cognitive Robotic Architecture. In Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems (AAMAS '17). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 1378--1386. http://dl.acm.org/citation.cfm?id=3091282.3091315

Digital Library

[14]

Pararth Shah, Dilek Hakkani-Tur, and Larry Heck. 2016. Interactive reinforcement learning for task-oriented dialogue management. (2016).

[15]

Huang A. Maddison C. J. Guez A. Sifre L. Driessche G. v. d. Schrittwieser J. Antonoglou I. Panneershelvam V. Lanctot M. Dieleman S. Grewe D. Nham J. Kalchbrenner N. Silver, D. and I. Sutskever. 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529 (2016), 484--489.

[16]

Hubert T. Schrittwieser J. Antonoglou I. Lai M. Guez A. Lanctot M. L. Sifre Kumaran D. Graepel T. Lillicrap T. Simonyan K. Silver, D. and D. Hassabis. 2017. Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm. arXiv Reprint arXiv:1712.01815 (2017).

[17]

Zhuoran Wang and Oliver Lemon. 2013. A simple and generic belief tracking mechanism for the dialog state tracking challenge: On the believability of observed information. In Proceedings of the SIGDIAL 2013 Conference. 423--432.

[18]

Rosen E. MacGlashan J. Wong L. L. Whitney, D. and S. Tellex. 2017. Reducing Errors in Object-Fetching Interactions through Social Feedback. IEEE International Conference on Robotics and Automation (ICRA).

[19]

Steve Young, Milica Gašić, Blaise Thomson, and Jason D Williams. 2013. Pomdp-based statistical spoken dialog systems: A review. Proc. IEEE 101, 5 (2013), 1160--1179.

[20]

Maryam Zare, Ali Ayub, Alan Wagner, and Rebecca Passonneau. 2019. In Review, Learning Board Games. (2019).

Cited By

Liu RGuo YJin RZhang X(2024)A Review of Natural-Language-Instructed Robot Execution SystemsAI10.3390/ai50300485:3(948-989)Online publication date: 26-Jun-2024
https://doi.org/10.3390/ai5030048
Ayub AWagner A(2020)Teach Me What You Want to Play: Learning Variants of Connect Four Through Human-Robot InteractionSocial Robotics10.1007/978-3-030-62056-1_42(502-515)Online publication date: 6-Nov-2020
https://doi.org/10.1007/978-3-030-62056-1_42

Index Terms

Show me how to win: a robot that uses dialog management to learn from demonstrations
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Discourse, dialogue and pragmatics
  2. Machine learning
    1. Learning paradigms
      1. Reinforcement learning
    2. Learning settings
      1. Learning from demonstrations

Recommendations

Ping to Win?: Non-Verbal Communication and Team Performance in Competitive Online Multiplayer Games
CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

Non-verbal communication plays a large role in online competitive multiplayer games, as team members attempt to coordinate with each other without distraction to achieve victory. Some games enable this communication through "pings," alerts that are easy ...
Win at Video Poker: A Guide to Beating the Poker Machines
How fast can Maker win in fair biased games?

We study (a:a) MakerBreaker games played on the edge set of the complete graph on n vertices. In the following four gamesperfect matching game, Hamilton cycle game, star factor game and path factor game, our goal is to determine the least number of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

FDG '19: Proceedings of the 14th International Conference on the Foundations of Digital Games

August 2019

822 pages

ISBN:9781450372176

DOI:10.1145/3337722

Conference Chair:
Sebastian Deterding
University of York
,
General Chair:
Foaad Khosmood
California Polytechnic State University
,
Program Chairs:
Johanna Pirker
Graz University of Technology
,
Thomas Apperley
CEGCS Tampere University

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 August 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Penn State

Conference

FDG '19

FDG '19: The Fourteenth International Conference on the Foundations of Digital Games

August 26 - 30, 2019

California, San Luis Obispo, USA

Acceptance Rates

FDG '19 Paper Acceptance Rate 46 of 124 submissions, 37%;

Overall Acceptance Rate 152 of 415 submissions, 37%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
143
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu RGuo YJin RZhang X(2024)A Review of Natural-Language-Instructed Robot Execution SystemsAI10.3390/ai50300485:3(948-989)Online publication date: 26-Jun-2024
https://doi.org/10.3390/ai5030048
Ayub AWagner A(2020)Teach Me What You Want to Play: Learning Variants of Connect Four Through Human-Robot InteractionSocial Robotics10.1007/978-3-030-62056-1_42(502-515)Online publication date: 6-Nov-2020
https://doi.org/10.1007/978-3-030-62056-1_42

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten