skip to main content
10.1145/3574131.3574453acmconferencesArticle/Chapter ViewAbstractPublication PagessiggraphConference Proceedingsconference-collections
research-article

3D Grasp Pose Generation from 2D Anchors and Local Surface

Published: 13 January 2023 Publication History

Abstract

This work proposes a three-dimensional (3D) robot grasp pose generation method for robot manipulators from the predicted two-dimensional (2D) anchors and the depth information of the local surface. Compared to the traditional image-based grasp area detection methods in which the grasp pose is only presented by two contact points, the proposed method can generate a more accurate 3D grasp pose. Furthermore, different from the 6-DoF object pose regression methods in which the point cloud of the whole objects is considered, the proposed method is very lightweight, since the 3D computation is only processed on the depth information of the local grasp surface. The method consists of three steps: (1) detecting the 2D grasp anchor and extracting the local grasp surface from the image; (2) obtaining the average vector of the objects’ local grasp surface from the objects’ local point cloud; (3) generating the 3D grasp pose from 2D grasp anchor based on the average vector of local grasp surface. The experiments are carried out on the Cornell and Jacquard grasp datasets. It is found that the proposed method yields improvement in the grasp accuracy compared to state-of-the-art 2D anchor methods. And the proposed method is also validated on the practical grasp tasks deployed on a UR5 arm with Robotiq Grippers F85. It outperforms state-of-the-art 2D anchor methods on the grasp success rate for dozens of practical grasp tasks.

References

[1]
Anonymous. [n. d.]. Euler angles between two 3D vectors. https://stackoverflow.com/questions/15101103/euler-angles-between-two-3d-vectors. accessed 29 May 2022.
[2]
Xiang Bai, Song Bai, Zhuotun Zhu, and Longin Jan Latecki. 2015. 3D Shape Matching via Two Layer Coding. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 12(2015), 2361–2373.
[3]
Martin John Baker. [n. d.]. Maths - Conversion Axis - Angle to Euler. http://www.euclideanspace.com/maths/geometry/rotations/conversions/angleToEuler/index.htm. accessed 29 May 2022.
[4]
Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. 2020. YOLOv4: Optimal Speed and Accuracy of Object Detection. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[5]
Jeannette Bohg and Danica Kragic. 2010. Learning grasping points with shape context. Robotics and Autonomous Systems 58, 4 (2010), 362–377.
[6]
Daniel Bolya, Chong Zhou, Fanyi Xiao, and Yong Jae Lee. 2019. YOLACT: Real-time Instance Segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision.
[7]
Alexandros Bouganis and Murray Shanahan. 2008. Flexible Object Recognition in Cluttered Scenes Using Relative Point Distribution Models. In 19th International Conference on Pattern Recognition (ICPR).
[8]
Eric Brachmann, Frank Michel, Alexander Krull, Michael Ying Yang, Stefan Gumhold, and Carsten Rother. 2016. Uncertainty-driven 6D pose estimation of objects and scenes from a single RGB image. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 3364–3372.
[9]
Fu-Jen Chu, Ruinian Xu, and Patricio A. Vela. 2019. Detecting Robotic Affordances on Novel Objects with Regional Attention and Attributes. IEEE Robotics and Automation Letters(2019).
[10]
Navneet Dalal and Bill Triggs. 2005. Histograms of Oriented Gradients for Human Detection. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). 886–893.
[11]
Amaury Depierre, Emmanuel Dellandrea, and Liming Chen. 2018. Jacquard: A Large Scale Dataset for Robotic Grasp Detection. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
[12]
Ge Gao, Mikko Lauri, Yulong Wang, Xiaolin Hu, Jianwei Zhang, and Simone Frintrop. 2020. 6D Object Pose Regression via Supervised Learning on Point Clouds. In IEEE International Conference on Robotics and Automation (ICRA).
[13]
Ning Guo, Baohua Zhang, Jun Zhou, Ketian Zhan, and Shuang Lai. 2020. Pose estimation and adaptable grasp configuration with point cloud registration and geometry understanding for fruit grasp planning. Computers and Electronics in Agriculture 179 (2020).
[14]
Yun Jiang, Stephen Moseson, and Ashutosh Saxena. 2011. Efficient grasping from RGBD images: Learning using a new rectangle representation. In IEEE International Conference on Robotics and Automation (ICRA).
[15]
Wadim Kehl, Fabian Manhardt, Federico Tombari, Slobodan Ilic, and Nassir Navab. 2017. SSD-6D: Making RGB based 3D detection and 6D pose estimation great again. In IEEE International Conference on Computer Vision (ICCV). 1521–1529.
[16]
Klaas Klasing, Daniel Althoff, Dirk Wollherr, and Martin Buss. 2009. Comparison of surface normal estimation methods for range sensing applications. In IEEE International Conference on Robotics and Automation.
[17]
Ian Lenz, Honglak Lee, and Ashutosh Saxena. 2013. Deep learning for detecting robotic grasps. The International Journal of Robotics Research 34, 4–5(2013).
[18]
Chunfang Liu, Bin Fang, Fuchun Sun, Xiaoli Li, and Wenbing Huang. 2019. Learning to Grasp Familiar Objects Based on Experience and Objects’ Shape Affordance. IEEE Transactions on Systems, Man, and Cybernetics: Systems 49, 12(2019), 2710–2723.
[19]
David Liu and Tsuhan Chen. 2004. Soft shape context for iterative closest point registration. In International Conference on Image Processing (ICIP).
[20]
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C. Berg. 2016. SSD: Single Shot MultiBox Detector. In 14th European Conference on Computer Vision (ECCV).
[21]
MindSpore. [n. d.]. MindSpore. https://www.mindspore.cn/doc/note/en/r1.1/networklistms.html accessed 30 May 2022.
[22]
Niloy J. Mitra, An Nguyen, and Leonidas Guibas. 2004. Estimating surface normals in noisy point cloud data. International Journal of Computational Geometry and Applications 14, 5(2004), 261–276.
[23]
Arsalan Mousavian, Clemens Eppner, and Dieter Fox. 2019. 6-DOF GraspNet: Variational Grasp Generation for Object Manipulation. In Proceedings of the IEEE/CVF International Conference on Computer Vision.
[24]
Adithyavairavan Murali, Arsalan Mousavian, Clemens Eppner, Chris Paxton, and Dieter Fox. 2020. 6-DOF Grasping for Target-driven Object Manipulation in Clutter. In International Conference on Robotics and Automation (ICRA).
[25]
Chavdar Papazov, Sami Haddadin, Sven Parusel, Kai Krieger, and Darius Burschka. 2012. Rigid 3D geometry matching for grasping of known objects in cluttered scenes. The International Journal of Robotics Research 31, 4 (2012), 538–553.
[26]
Georgios Pavlakos, Xiaowei Zhou, Aaron Chan, Konstantinos G. Derpanis, and Kostas Daniilidis. 2017. 6-DOF object pose from semantic keypoints. In IEEE International Conference on Robotics and Automation (ICRA).
[27]
J. Redmon and A. Angelova. 2015. Real-time grasp detection using convolutional neural networks. In IEEE International Conference on Robotics and Automation (ICRA). IEEE.
[28]
Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2015. You Only Look Once: Unified, Real-Time Object Detection. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[29]
Joseph Redmon and Ali Farhadi. 2018. YOLOv3: An Incremental Improvement. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[30]
Hamid Rezatofighi, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese. 2019. Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
[31]
Eduardo Godinho Ribeiro, Raul de Queiroz Mendes, and Valdir Grassi Jr.2021. Real-Time Deep Learning Approach to Visual Servo Control and Grasp Detection for Autonomous Robotic Manipulation. Robotics and Autonomous Systems 139 (2021), 1037–1057.
[32]
Patrick Snape and Stefanos Zafeiriou. 2014. Kernel-PCA Analysis of Surface Normals for Shape-from-Shading. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[33]
Joviša unić, Kaoru Hirota, and Paul L. Rosin. 2010. A Hu Moment Invariant as a Shape Circularity Measure. Pattern Recognition 43, 1 (jan 2010), 47–57. https://doi.org/10.1016/j.patcog.2009.06.017
[34]
Chen Wang, Danfei Xu, Yuke Zhu, Roberto Martín-Martín, Cewu Lu, Li Fei-Fei, and Silvio Savarese. 2020. DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion. In IEEE International Conference on Robotics and Automation (ICRA).
[35]
Yu Xiang, Tanner Schmidt, Venkatraman Narayanan, and Dieter Fox. 2018. PoseCNN: a convolutional neural network for 6D object pose estimation in cluttered scenes. In Science and Systems Conference (RSS).
[36]
Hanbo Zhang, Xuguang Lan, Site Bai, Xinwen Zhou, Zhiqiang Tian, and Nanning Zheng. 2019. ROI-based Robotic Grasp Detection for Object Overlapping Scenes. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 4768–4775.
[37]
Xinwen Zhou, Xuguang Lan, Hanbo Zhang, Zhiqiang Tian, Yang Zhang, and Nanning Zheng. 2018. Fully Convolutional Grasp Detection Network with Oriented Anchor Box. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

Cited By

View all
  • (2024)6D Assembly Pose Estimation by Point Cloud Registration for Robotic Manipulation2024 IEEE 20th International Conference on Automation Science and Engineering (CASE)10.1109/CASE59546.2024.10711374(846-853)Online publication date: 28-Aug-2024
  • (2024)Efficient event-based robotic grasping perception using hyperdimensional computingInternet of Things10.1016/j.iot.2024.10120726(101207)Online publication date: Jul-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
VRCAI '22: Proceedings of the 18th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry
December 2022
284 pages
ISBN:9798400700316
DOI:10.1145/3574131
  • Editors:
  • Enhua Wu,
  • Lionel Ming-Shuan Ni,
  • Zhigeng Pan,
  • Daniel Thalmann,
  • Ping Li,
  • Charlie C.L. Wang,
  • Lei Zhu,
  • Minghao Yang
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 January 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Convolutional Neural Networks (CNN)
  2. Pose Generation
  3. Robotic Grasp

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • Science and Technology on Aerospace Flight Dynamics Laboratory
  • the National Natural Science Foundation of China
  • This work is supported by the National Key Research & Development Program of China
  • the Guangxi Key Research and Development Program

Conference

VRCAI '22
Sponsor:

Acceptance Rates

Overall Acceptance Rate 51 of 107 submissions, 48%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)18
  • Downloads (Last 6 weeks)4
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)6D Assembly Pose Estimation by Point Cloud Registration for Robotic Manipulation2024 IEEE 20th International Conference on Automation Science and Engineering (CASE)10.1109/CASE59546.2024.10711374(846-853)Online publication date: 28-Aug-2024
  • (2024)Efficient event-based robotic grasping perception using hyperdimensional computingInternet of Things10.1016/j.iot.2024.10120726(101207)Online publication date: Jul-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media