Andreea Bobu
ABSTRACT
Learning from human input has enabled autonomous agents to perform increasingly more complex tasks that are otherwise difficult to carry out automatically. To this end, recent works have studied how robots can incorporate such input - like demonstrations or corrections - into objective functions describing the desired behaviors. While these methods have shown progress in a variety of settings, from semi-autonomous driving, to household robotics, to automated airplane control, they all suffer from the same crucial drawback: they implicitly assume that the person's intentions can always be captured by the robot's hypothesis space. We call attention to the fact that this assumption is often unrealistic, as no model can completely account for every single possible situation ahead of time. When the robot's hypothesis space is misspecified, human input can be unhelpful - or even detrimental - to the way the robot is performing its tasks. Our work tackles this issue by proposing that the robot should first explicitly reason about how well its hypothesis space can explain human inputs, then use that situational confidence to inform how it should incorporate them.
- Pieter Abbeel and Andrew Y Ng. 2004. Apprenticeship learning via inverse reinforcement learning. In Machine Learning (ICML), International Conference on. ACM.Google Scholar
- Andrea Bajcsy, Dylan P. Losey, Marcia Kilchenman O'Malley, and Anca D. Dragan. 2017. Learning Robot Objectives from Physical Human Interaction. In CoRL .Google Scholar
- Chris L Baker, Joshua B Tenenbaum, and Rebecca R Saxe. 2007. Goal inference as inverse planning. In Proceedings of the Annual Meeting of the Cognitive Science Society, Vol. 29.Google Scholar
- Andreea Bobu, Andrea Bajcsy, Jaime F. Fisac, Sampada Deglurkar, and Anca D. Dragan. 2019. Quantifying Hypothesis Space Misspecification in Learning from Human-Robot Demonstrations and Physical Corrections. (2019). To appear in Transactions on Robotics.Google Scholar
- Andreea Bobu, Andrea Bajcsy, Jaime F. Fisac, and Anca D. Dragan. 2018. Learning under Misspecified Objective Spaces. In 2nd Annual Conference on Robot Learning, CoRL 2018, Zü rich, Switzerland, 29--31 October 2018, Proceedings . 796--805.Google Scholar
- Paul Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. (06 2017).Google Scholar
- Justin Fu, Avi Singh, Dibya Ghosh, Larry Yang, and Sergey Levine. [n.d.]. Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definition. arXiv preprint , Vol. arXiv:1805.11686 ( [n.,d.]).Google Scholar
- Ashesh Jain, Shikhar Sharma, Thorsten Joachims, and Ashutosh Saxena. 2015. Learning preferences for manipulation tasks from online coactive feedback. The International Journal of Robotics Research , Vol. 34, 10 (2015), 1296--1313.Google ScholarDigital Library
- Shervin Javdani, Siddhartha S Srinivasa, and J Andrew Bagnell. 2015. Shared autonomy via hindsight optimization. arXiv preprint arXiv:1503.07619 (2015).Google Scholar
- John Von Neumann and Oskar Morgenstern. 1945. Theory of games and economic behavior .Princeton University Press Princeton, NJ.Google Scholar
Index Terms
- Detecting Hypothesis Space Misspecification in Robot Learning from Human Input
Recommendations
Human Intention Prediction in Human-Robot Collaborative Tasks
HRI '18: Companion of the 2018 ACM/IEEE International Conference on Human-Robot InteractionEnabling the robot to predict human intentions in human-robot collaborative hand-over tasks is a challenging but important issue to address. We develop a novel and effective teaching-learning-prediction (TLP) model for the robot to online learn from ...
Physical interaction as communication: Learning robot objectives online from human corrections
When a robot performs a task next to a human, physical interaction is inevitable: the human might push, pull, twist, or guide the robot. The state of the art treats these interactions as disturbances that the robot should reject or avoid. At best, these ...
Compact Real-time Avoidance on a Humanoid Robot for Human-robot Interaction
HRI '18: Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot InteractionWith robots leaving factories and entering less controlled domains, possibly sharing the space with humans, safety is paramount and multimodal awareness of the body surface and the surrounding environment is fundamental. Taking inspiration from ...
Comments