Bidan Huang
ABSTRACT
This paper proposes a user-friendly teleoperation interface for manipulation. We provide the user with a view of the scene augmented with depth information from local cameras to provide visibility in occluded areas during manipulation tasks. This gives an improved sense of the 3D environment which results in better task performance. Further, we monitor the pose of the robot's end effector in real-time so that we are able to superimpose a virtual representation into the scene when parts are occluded. The integration of these features enables the user to perform difficult manipulation tasks when the action environment is normally occluded in the main camera view.
We performed preliminary studies with this new setup and users provided positive feedback regarding the proposed augmented reality (AR) system.
- Gloria L Calhoun, Mark H Draper, Jeremy T Nelson, and Heath A Ruff. 2006. Advanced display concepts for UAV sensor operations: Landmark cues and picture-in-picture. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 50. SAGE Publications Sage CA: Los Angeles, CA, 121--125.Google Scholar
Cross Ref
- Soichiro Iwataki, Hiromitsu Fujii, Alessandro Moro, Atsushi Yamashita, Hajime Asama, and Hiroshi Yoshinada. 2015. Visualization of the surrounding environment and operational part in a 3DCG model for the teleoperation of construction machines. In 2015 IEEE/SICE International Symposium on System Integration (SII). IEEE, 81--87.Google ScholarCross Ref
- Leonid Keselman, John Iselin Woodfill, Anders Grunnet-Jepsen, and Achintya Bhowmik. 2017. Intel realsense stereoscopic depth cameras. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 1--10.Google ScholarCross Ref
- Fumio Okura, Yuko Ueda, Tomokazu Sato, and Naokazu Yokoya. 2013. Teleoperation of mobile robots by generating augmented free-viewpoint images. In 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 665--671.Google ScholarCross Ref
- Luis Enrique Ortiz, Elizabeth V Cabrera, and Luiz M Goncc alves. 2018. Depth data error modeling of the ZED 3D vision sensor from stereolabs. ELCVIA: electronic letters on computer vision and image analysis , Vol. 17, 1 (2018), 0001--15.Google Scholar
- A Pitman, Curtis M Humphrey, and Julie A Adams. 2007. A picture-in-picture interface for a multiple robot system.Google Scholar
- Long Qian, Anton Deguet, Zerui Wang, Yun-Hui Liu, and Peter Kazanzides. 2019. Augmented Reality Assisted Instrument Insertion and Tool Manipulation for the First Assistant in Robotic Surgery. In 2019 International Conference on Robotics and Automation (ICRA). IEEE, 5173--5179.Google Scholar
- Thomas B Sheridan. 1992. Musings on telepresence and virtual presence. Presence: Teleoperators & Virtual Environments , Vol. 1, 1 (1992), 120--126.Google ScholarCross Ref
- AWW Yew, SK Ong, and AYC Nee. 2017. Immersive augmented reality environment for the teleoperation of maintenance robots. Procedia CIRP , Vol. 61 (2017), 305--310.Google ScholarCross Ref
Index Terms
- Augmented Reality with Multi-view Merging for Robot Teleoperation
Recommendations
Improving Collocated Robot Teleoperation with Augmented Reality
HRI '18: Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot InteractionRobot teleoperation can be a challenging task, often requiring a great deal of user training and expertise, especially for platforms with high degrees-of-freedom (e.g., industrial manipulators and aerial robots). Users often struggle to synthesize ...
Haptics in Augmented Reality
ICMCS '99: Proceedings of the IEEE International Conference on Multimedia Computing and Systems - Volume 2An augmented reality system merges synthetic sensory information into a user's perception of a three-dimensional environment. An important performance goal for an augmented reality system is that the user perceives a single seamless environment. In most ...
Virtual and Augmented Reality with Head-Tracking for Efficient Teleoperation of Groups of Robots
CW '08: Proceedings of the 2008 International Conference on CyberworldsThis paper deals with the usage of Virtual and Augmented Reality techniques in the field of teleoperation. The global context is a project for collaborative teleoperation of real robots groups. The aim is to enable to a group of teleoprators to control ...
Comments