Abstract
This paper proposes a novel approach for the autonomous 3D model generation of unknown objects. A humanoid robot (or any setup with two manipulators) holds the object to model in one hand, views it from different perspectives and registers the depth information using a RGB-D sensor. The occlusions due to limited movement of the manipulator and the gripper itself covering the object are avoided by switching the object from one hand to the other. This allows for additional viewpoints leading to the registration of more depth information of the object. The contributions of this paper are as follows: 1. A humanoid robot that manipulates objects and obtains depth information 2. Tracing the hand movements with the robots head to be able to see the object at every moment 3. Filtering the point clouds to remove parts of the robot from them 4. Utilizing the Normal Iterative Closest Point algorithm (depth points, surface normals and curvature information) to register point clouds over time. This method will be applied to those pointclouds that include the robots gripper for optimal convergence; the resultant transform is then applied to those point clouds that describe only the segmented object 5. Changing the object from one hand to another 6. Merging the resulting object’s partial point clouds from both the left and right hands 7. Generating a mesh of the object based on the triangulation of final points of the object’s surface. No prior knowledge of the objects is necessary. No human intervention nor external help (i.e visual markers, turntables ...) is required either.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aldoma, A., Tombari, F., Stefano, L.D., Vincze, M.: A global hypotheses verification method for 3D object recognition. In: 2012, European Conference on Computer Vision (ECCV). Lecture Notes in Computer Science, vol. 7574. Springer
Zhu, M., Derpanis, K., Yang, Y., Brahmbhatt, S., Zhang, M., Phillips, C., Lecce, M., Daniilidis, K.: Single image 3D object detection and pose estimation for grasping. In: Proceedings International Conference on Robotics and Automation (ICRA), Hong Kong, China, 31 May–5 June 2014
Sarkar, K., Varanasi1, K., Stricker, D.: Trained 3D models for CNN based object recognition. In: Proceedings International Conference on Computer Vision (ICCV), Santiago, Chile, 13–16 Dec 2015
Peng, X., Sun, B., Ali, K., Saenko, K.: Learning deep object detectors from 3D models. In: Proceedings International Conference on Computer Vision (ICCV), Santiago, Chile, 13–16 Dec 2015
Gupta, S., Arbelaez, P., Girshick, R., Malik, J.: Aligning 3D models to RGB-D images of cluttered scenes. In: Proceedings International Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015
Pas, A., Platt, R.: Using geometry to detect grasp poses in 3D point clouds. In: Proceedings International Symposium on Robotics Research (ISRR), Genova, Italy, Sept 2015
Denkowski, M.: GPU accelerated 3D object reconstruction. Procedia Comput. Sci. 18, 290–298 (2013)
Mihalyi, R.-G., Pathak, K., Vaskevicius, N., Fromm, T., Birk, A.: Robust 3d object modeling with a low-cost RGBD-sensor and AR-markers for applications with untrained end-users. Robot. Auto. Syst. 66, 1–17 (2015)
Xie, J., Hsu, Y.-F., Feris, R., Sun, M.-T.: Fine registration of 3d point clouds fusing structural and photometric information using an RGB-D camera. In: J. Visual Commun. Image Represent. 32, 194–204 (2015)
Foissotte, T., Stasse, O., Escande, A., Wieber, P.-B., Kheddar, A.: A two-steps next-best-view algorithm for autonomous 3D object modeling by a humanoid robot. In: Proceedings International Conference on Robotics and Automation (ICRA), Kobe, Japan, 12–17 May 2009
Jaiswal, M., Xie, J., Sun, M.-T.: 3D object modeling with a Kinect camera. In: Proceedings Signal and Information Processing Association Annual Summit and Conference (APSIPA), Chiang Mai, Thailand, 6–9 June 2014
Krainin, M., Henry, P., Ren, X.: Manipulator and object tracking for in-hand 3d object modeling. Int. J. Robot. Res.
Llopart, A., Ravn, O., Andersen, N., Kim, J.-H.: Generalized framework for the parallel semantic segmentation of multiple objects and posterior manipulation. In: Proceedings International Conference on Robotics and Biomimetics (ROBIO), Macau, China, 5–8 Dec 2017
Rusu, R., Marton, Z.C., Blodow, N., Dolha, M., Beetz, M.: Towards 3d point cloud based object maps for household environments. Robot. Auto. Syst. 56(11), 927–941 (2008)
Marton, C., Radu, R., Beetz, M.: On fast surface reconstruction methods for large and noisy datasets. In: Proceedings International Conference on Robotics and Automation (ICRA), Kobe, Japan, 12–17 May 2009
Weise, T., Wismer, T., Leibe, B., Gool, L.V.: In-hand scanning with online loop closure. In: Proceedings International Conference on Computer Vision Workshops (ICCV Workshops), Kyoto, Japan, 27 Sept–4 Oct 2009
Krainin, M., Curless, B., Fox, D.: Autonomous generation of complete 3D object models using next best view manipulation planning. In: Proceedings International Conference on Robotics and Automation (ICRA), Shanghai, China, 9–13 May 2011
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Llopart, A., Ravn, O., Andersen, N.A., Kim, JH. (2019). Autonomous 3D Model Generation of Unknown Objects for Dual-Manipulator Humanoid Robots. In: Kim, JH., et al. Robot Intelligence Technology and Applications 5. RiTA 2017. Advances in Intelligent Systems and Computing, vol 751. Springer, Cham. https://doi.org/10.1007/978-3-319-78452-6_41
Download citation
DOI: https://doi.org/10.1007/978-3-319-78452-6_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-78451-9
Online ISBN: 978-3-319-78452-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)