Robot Motion Learning via Visual Imitation and Language-Conditioned Reward | IEEE Conference Publication | IEEE Xplore

Robot Motion Learning via Visual Imitation and Language-Conditioned Reward


Abstract:

In recent years, there has been an increase in the use of robots that coexist with humans in society and homes. These robots are required to perform various motions in su...Show More

Abstract:

In recent years, there has been an increase in the use of robots that coexist with humans in society and homes. These robots are required to perform various motions in such environments. Research has been conducted on imitation learning for robots to easily learn various motions. However, conventional imitation learning methods require converting the motion information of the instructor into the joint angles of the robot and preparing data in advance for pairs of images that uniquely correspond to human-motion and robot images, which are costly for data collection. To reduce the cost of data collection and learning, we propose a method of imitation learning that consists of two stages: an imitation process using CycleGAN and a convolutional neural network, and a correction process using a policy gradient and contrastive language–image pretraining. During imitation, CycleGAN learns the visual correspondence between images of the human body and the robot body, and the convolutional neural network learns the correspondence between the image and joint angles of the robot. During correction, the convolutional neural network is fine-tuned using a policy gradient, which uses a reward function based on the contrastive language–image pretraining that is easily designed using language, and motions are corrected to achieve desired tasks. In the experiments, the robot arm reproduced human motions by learning from the data of the human and robot arms, which were moved randomly. The effectiveness of the proposed method was demonstrated by completing the assigned task after correcting the imitated motions. Experimental results show that the proposed method enabled the arm robot to perform imitation behavior that captured the outlines of three types of unlearned behaviors and enabled it to accomplish meaningful tasks.
Date of Conference: 20-23 May 2024
Date Added to IEEE Xplore: 27 August 2024
ISBN Information:
Conference Location: Austin, TX, USA

References

References is not available for this document.