skip to main content
10.1145/3337722.3341866acmotherconferencesArticle/Chapter ViewAbstractPublication PagesfdgConference Proceedingsconference-collections
research-article

Show me how to win: a robot that uses dialog management to learn from demonstrations

Published: 26 August 2019 Publication History

Abstract

We present an approach for robot learning from demonstration and communication applied to simple board games like Connect Four. In such games, a visual representation of a winning condition on the board can be converted to an extensive form representation that can then support computation of a winning strategy. We present a robot that can learn simple games from responses to visual questions based on synthesized images, or to verbal questions. We illustrate how reliance on both modalities leads to more efficient learning.

References

[1]
2013. Connect Four Demo: Rethink Robotics. (2013). http://sdk.rethinkrobotics.com/wiki/Connect_Four_Demo.
[2]
Ali Ayub and Alan R Wagner. 2018. Learning to Win Games in a Few Examples: Using Game-Theory and Demonstrations to Learn the Win Conditions of a Connect Four Game. In International Conference on Social Robotics. Springer, 349--358.
[3]
Conway H. J. Berlekamp, E. and R. Guy. 1982. Winning Ways for your Mathematical Plays: Games in general. Academic Press (1982).
[4]
M. Cakmak and A. L. Thomaz. 2012. Designing robot learners that ask good questions. Proceedings of the seventh annual ACM/IEEE International conference on Human-Robot Interaction.
[5]
Joyce Y. Chai, Qiaozi Gao, Lanbo She, Shaohua Yang, Sari Saba-Sadiya, and Guangyue Xu. 2018. Language to Action: Towards Interactive Task Learning with Physical Agents. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18. International Joint Conferences on Artificial Intelligence Organization, 2--9.
[6]
Borghoff U. M. Dobrovsky, A. and M. Hofmann. 2016. An approach to interactive deep reinforcement learning for serious games. 7th IEEE International Conference on Cognitive Infocommunications (CogInfoCom).
[7]
T. R. Hinrichs and K. D. Forbus. 2014. X Goes First : Teaching Simple Games through Multimodal Interaction. Advances in Cognitive Systems 3 (2014), 31--46.
[8]
Francisco S. Melo, Carla Guerra, and Manuel Lopes. 2018. Interactive Optimal Teaching with Unknown Learners. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18. International Joint Conferences on Artificial Intelligence Organization, 2567--2573.
[9]
Shiwali Mohan and John E. Laird. 2014. Learning Goal-oriented Hierarchical Tasks from Situated Interactive Instruction. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI'14). AAAI Press, 387--394. http://dl.acm.org/citation.cfm?id=2893873.2893934
[10]
Kober J. Kroemer O. Mulling, K. and J. Peters. 2013. Learning to select and generalize striking movements in robot table tennis. The International Journal of Robotics Research (IJRR) (2013).
[11]
Mattia Racca and Ville Kyrki. 2018. Active robot learning for temporal task models. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction. ACM, 123--131.
[12]
Mukadam M. Ahmadzadeh S. R. Chernova S. Rana, M. A. and B. Boots. 2017. Towards robust skill generalization: Unifying learning from demonstration and motion planning. Conference on Robot Learning(CoRL).
[13]
Matthias Scheutz, Evan Krause, Brad Oosterveld, Tyler Frasca, and Robert Platt. 2017. Spoken Instruction-Based One-Shot Object and Action Learning in a Cognitive Robotic Architecture. In Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems (AAMAS '17). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 1378--1386. http://dl.acm.org/citation.cfm?id=3091282.3091315
[14]
Pararth Shah, Dilek Hakkani-Tur, and Larry Heck. 2016. Interactive reinforcement learning for task-oriented dialogue management. (2016).
[15]
Huang A. Maddison C. J. Guez A. Sifre L. Driessche G. v. d. Schrittwieser J. Antonoglou I. Panneershelvam V. Lanctot M. Dieleman S. Grewe D. Nham J. Kalchbrenner N. Silver, D. and I. Sutskever. 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529 (2016), 484--489.
[16]
Hubert T. Schrittwieser J. Antonoglou I. Lai M. Guez A. Lanctot M. L. Sifre Kumaran D. Graepel T. Lillicrap T. Simonyan K. Silver, D. and D. Hassabis. 2017. Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm. arXiv Reprint arXiv:1712.01815 (2017).
[17]
Zhuoran Wang and Oliver Lemon. 2013. A simple and generic belief tracking mechanism for the dialog state tracking challenge: On the believability of observed information. In Proceedings of the SIGDIAL 2013 Conference. 423--432.
[18]
Rosen E. MacGlashan J. Wong L. L. Whitney, D. and S. Tellex. 2017. Reducing Errors in Object-Fetching Interactions through Social Feedback. IEEE International Conference on Robotics and Automation (ICRA).
[19]
Steve Young, Milica Gašić, Blaise Thomson, and Jason D Williams. 2013. Pomdp-based statistical spoken dialog systems: A review. Proc. IEEE 101, 5 (2013), 1160--1179.
[20]
Maryam Zare, Ali Ayub, Alan Wagner, and Rebecca Passonneau. 2019. In Review, Learning Board Games. (2019).

Cited By

View all
  • (2024)A Review of Natural-Language-Instructed Robot Execution SystemsAI10.3390/ai50300485:3(948-989)Online publication date: 26-Jun-2024
  • (2020)Teach Me What You Want to Play: Learning Variants of Connect Four Through Human-Robot InteractionSocial Robotics10.1007/978-3-030-62056-1_42(502-515)Online publication date: 6-Nov-2020

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
FDG '19: Proceedings of the 14th International Conference on the Foundations of Digital Games
August 2019
822 pages
ISBN:9781450372176
DOI:10.1145/3337722
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 August 2019

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article

Funding Sources

  • Penn State

Conference

FDG '19

Acceptance Rates

FDG '19 Paper Acceptance Rate 46 of 124 submissions, 37%;
Overall Acceptance Rate 152 of 415 submissions, 37%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)3
  • Downloads (Last 6 weeks)0
Reflects downloads up to 25 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)A Review of Natural-Language-Instructed Robot Execution SystemsAI10.3390/ai50300485:3(948-989)Online publication date: 26-Jun-2024
  • (2020)Teach Me What You Want to Play: Learning Variants of Connect Four Through Human-Robot InteractionSocial Robotics10.1007/978-3-030-62056-1_42(502-515)Online publication date: 6-Nov-2020

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media