Abstract
Meta-imitation learning has been applied to enable robots to quickly generalize the learned tasks to perform new tasks. The basic idea is to encode tasks into meaningful embeddings and perform the task specified by the given embedding. When the robot is tasked to perform a new, unseen task, it is given a single or just a few demonstrations, from which a new task embedding is obtained by generalizing from existing embeddings. Therefore, task encoding is key to the generalizability of meta-imitation learning. The problem is that most meta-imitation learning methods directly encode the whole task. This will take in too many irrelevant details into the task embeddings, resulting in poor generalization abilities. In this work, we propose to encode a task separately based on its different features, e.g., the required skills and the target objects. As a result, the robot can have a clearer understanding of the tasks and perform new tasks better. We compare the proposed method with typical meta-imitation learning methods on performing a set of robot tasks and test their performances of adapting to new, unseen tasks. The experimental results indicate that the proposed method can better encode tasks into meaningful embeddings and thus lead to better generalization.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Argall, B.D., Chernova, S., Veloso, M., Browning, B.: A survey of robot learning from demonstration. Robotics and Autonomous Systems (2009)
Zhang, T., McCarthy, Z., Jow, O., Lee, D., Chen, X., Goldberg, K., Abbeel, P.: Deep imitation learning for complex manipulation. In: International Conference on Robotics And Automation (ICRA) (2018)
Ng, A.Y., Russell, S.: Algorithms for inverse reinforcement. In: International Conference on Machine Learning (ICML) (2000)
Abbeel, P., Ng, A.Y.: Apprenticeship learning via inverse reinforcement learning. In: International Conference on Machine Learning (ICML) (2004)
Gupta, A., Kumar, V., Lynch, C., Levine, S., Hausman, K.: Relay policy learning: solving long-horizon tasks via imitation and reinforcement learning. In: Conference on Robot Learning (CoRL) (2019)
M. Bojarski, et al.: End to end learning for self-driving cars. arXiv:1604.07316 (2016)
Merel, J., et al.: Learning human behaviors from motion capture by adversarial imitation. arXiv:1707.022s01 (2017)
Y. Zhu, Z. Wang, J. Merel, A. Rusu, T. Erez, S. Cabi, S. Tunyasuvunakool, J. Kram´ar, R. Hadsell, N. de Freitas, and N. Heess, “Reinforcement and imitation learning for diverse visuomotor skills,” arXiv:1802.09564, 2018
Hausman, K., Chebotar, Y., Schaal, S., Sukhatme, G., Lim, J.: Multimodal imitation learning from unstructured demonstrations using generative adversarial nets. In: Neural Information Processing Systems (NIPS) (2017)
Rahmatizadeh, R., Abolghasemi, P., B¨ol¨oni, L., Levine, S.: Vision-based multi-task manipulation for inexpensive robots using end-to-end learning from demonstration, In: International Conference on Robotics and Automation (ICRA), pp. 3758–3765 (2018)
Lynch, C., et al.: Learning latent plans from play. In: Conference on Robot Learning (CoRL) (2019)
C. Devin, A. Gupta, T. Darrell, P. Abbeel, and S. Levine, “Learning modular neural network policies for multi-task and multi-robot transfer,” International Conference on Robotics and Automation (ICRA), 2017
Devin, C., Abbeel, P., Darrell, T., Levine, S.: Deep object-centric representations for generalizable robot learning. In: International Conference on Robotics and Automation (ICRA) (2018)
Finn, C., Yu, T., Zhang, T., Abbeel, P., Levine, S.: One-shot visual imitation learning via meta-learning. In: Conference on Robot Learning (CoRL) (2017)
James, S., Bloesch, M., Davison, A.J.: Task-embedded control networks for few-shot imitation learning. In: Conference on Robot Learning (CoRL) (2018)
Singh, A., et al.: Scalable multi-task imitation learning with autonomous improvement. In: International Conference on Robotics and Automation (ICRA), pp. 2167–2173 (2020)
Duan, Y., et al.: One-shot imitation learning. In: Neural Information Processing Systems (NIPS), pp. 1087–1098 (2017)
Wang, Z., Merel, J., Reed, S., Wayne, G., de Freitas, N., Heess, N.: Robust imitation of diverse behaviors. In: Neural Information Processing Systems (NIPS) (2017)
Levine, S., Finn, C., Darrell, T., Abbeel, P.: End-to-end training of deep visuomotor policies. J. Mach. Learn. Res. (2016)
Yen-Chen, L., Bauza, M., Isola, P.: Experience-embedded visual foresight. In: Conference on Robot Learning (CoRL) (2019)
Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., Wierstra, D.: Matching networks for one shot. In: Neural Information Processing Systems (NIPS), pp. 3630–3638 (2016)
Liu, W., et al.: Ssd: Single shot multibox detector. In: Proceedings of the European Conference on Computer Vision, pp. 21–37 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lin, YF., Ho, C., King, CT. (2023). Improving Meta-imitation Learning with Focused Task Embedding. In: Arai, K. (eds) Intelligent Systems and Applications. IntelliSys 2022. Lecture Notes in Networks and Systems, vol 544. Springer, Cham. https://doi.org/10.1007/978-3-031-16075-2_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-16075-2_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16074-5
Online ISBN: 978-3-031-16075-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)