Abstract
In the field of human-robot interaction, robot collision avoidance with the human in a shared workspace remains a challenge. Many researchers use visual methods to detect the collision between robots and obstacles on the assumption that the robot pose is known because the information about the robot is obtained from the controller and hand-eye calibration is conducted. Therefore, they focus on the motion prediction of obstacles. In this paper, a real-time method based on deep learning is proposed to directly estimate the 3D pose of the robot arm using a color image. The method aims to remove the hand-eye calibration when the system needs to be reconfigured and increase the flexibility of the system by eliminating the requirement that the camera fixed relative to the robot. Our approach has two main contributions. One is that the method estimates the 3D position of the robot base and the relative 3D positions of the predefined key points of the robot to the robot base separately different from other deep learning methods considering the limitations of the dataset. The other is that some datasets are collected through another trained network to avoid tedious calibration process, and the trained network will be reused in the pose estimation task. Finally, the experiments are conducted. The results show that a fully trained system provides an accurate 3D pose estimation for the robot arm in the camera coordinate system. The average errors of the 3D positions of the robot base and the predefined key points are 2.35 cm and 1.99 cm respectively.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Fitzgerald, C.: Developing baxter. In: 2013 IEEE Conference on Technologies for Practical Robot Applications (TePRA), pp. 1–6. IEEE. Greater Boston Area (2013)
Halme, R.J., Lanz, M., Kämäräinen, J., Pieters, R., Latokartano, J., Hietanen, A.: Review of vision-based safety systems for human-robot collaboration. Procedia CIRP 72(1), 111–116 (2018)
Miseikis, J., Glette, K., Elle, O.J., Torresen, J.: Automatic calibration of a robot manipulator and multi 3D camera system. In: 2016 IEEE/SICE International Symposium on System Integration (SII), pp. 735–741. IEEE, Sapporo (2016)
Stasse, O., Escande, A., Mansard, N., Miossec, S., Evrard, P., Kheddar, A.: Real-time (self)-collision avoidance task on a HRP-2 humanoid robot. In: 2008 IEEE International Conference on Robotics and Automation (ICRA), pp. 3200–3205. IEEE, Pasadena (2008)
Besl, P.J., McKay, N.D.: Method for registration of 3-D shapes. In: Sensor Fusion IV: Control Paradigms and Data Structures, vol. 1611, pp. 586–607 (1992)
Zeng, A., et al.: Multi-view self-supervised deep learning for 6D pose estimation in the amazon picking challenge. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 1386–1383. IEEE, Singapore (2017)
Tremblay, J., To, T., Sundaralingam, B., Xiang, Y., Fox, D., Birchfield, S.: Deep object pose estimation for semantic robotic grasping of household objects. arXiv preprint arXiv:1809.10790 (2018)
Gratal, X., Bohg, J., Björkman, M., Kragic, D.: Scene representation and object grasping using active vision. In: IROS 2010 Workshop on Defining and Solving Realistic Perception Problems in Personal Robotics (2010)
Hebert, P., et al.: Combined shape, appearance and silhouette for simultaneous manipulator and object tracking. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp. 2405–2412. IEEE, St. Paul (2012)
Krainin, M., Henry, P., Ren, X., Fox, D.: Manipulator and object tracking for in-hand 3D object modeling. Int. J. Robot. Res. 30(11), 1311–1327 (2011)
Bohg, J., Romero, J., Herzog, A., Schaal, S.: Robot arm pose estimation through pixel-wise part classification. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 3143–3150. IEEE, Hong Kong (2014)
Widmaier, F., Kappler, D., Schaal, S., Bohg, J.: Robot arm pose estimation by pixel-wise regression of joint angles. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 616–623. IEEE, Stockholm (2016)
Miseikis, J., Brijacak, I., Yahyanejad, S., Glette, K., Elle, O.J., Torresen, J.: Multi-objective convolutional neural networks for robot localisation and 3D position estimation in 2D camera images. In: 2018 15th International Conference on Ubiquitous Robots (UR), pp. 597–603. IEEE, Honolulu (2018)
Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 483–499. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_29
Mehta, D., et al.: VNect: real-time 3D human pose estimation with a single RGB camera. ACM Trans. Graph. (TOG) 36(4), 44 (2017)
Sun, X., Xiao, B., Wei, F., Liang, S., Wei, Y.: Integral human pose regression. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11210, pp. 536–553. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01231-1_33
Acknowledgment
This research work is partially supported by National Natural Science Foundation of China under Grant No. 51775344.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhou, F., Chi, Z., Zhuang, C., Ding, H. (2019). 3D Pose Estimation of Robot Arm with RGB Images Based on Deep Learning. In: Yu, H., Liu, J., Liu, L., Ju, Z., Liu, Y., Zhou, D. (eds) Intelligent Robotics and Applications. ICIRA 2019. Lecture Notes in Computer Science(), vol 11743. Springer, Cham. https://doi.org/10.1007/978-3-030-27538-9_46
Download citation
DOI: https://doi.org/10.1007/978-3-030-27538-9_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-27537-2
Online ISBN: 978-3-030-27538-9
eBook Packages: Computer ScienceComputer Science (R0)