Skip to main content
SpringerLink
Log in

Efficiently finding poses for multiple grasp types with partial point clouds by uncoupling grasp shape and scale

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

We present an algorithm that discovers grasp pose solutions for multiple grasp types for a multi-fingered mechanical gripper using partially-sensed point clouds of unknown objects. The algorithm introduces two key ideas: (1) a histogram of finger contact normals is used to represent a grasp “shape” to guide a gripper orientation search in a histogram of object(s) surface normals, and (2) voxel grid representations of gripper contacts and object(s) are cross-correlated to match finger contact points, i.e. grasp “scale`”, to discover a grasp pose. Collision constraints are incorporated in the cross-correlation computation. We show via simulations and preliminary experiments that (1) grasp poses for three grasp types (i.e. lateral, power, and tripodal) are found quickly without interrupting the robot’s motion, (2) the quality of grasp pose solutions is consistent with respect to voxel resolution changes for both partial and complete point cloud scans, (3) grasp type definitions are scalable for n-contacts and can incorporate constraints for collision checks in one integrated step, and (4) planned grasp poses are successfully executed with a mechanical gripper demonstrating the robustness of grasp pose solutions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Similar content being viewed by others

References

  • Balasubramanian, R., Xu, L., Brook, P. D., Smith, J. R., & Matsuoka, Y. (2012). Physical human interactive guidance: Identifying grasping principles from human-planned grasps. IEEE Transactions on Robotics, 28(4), 899–910.

    Article  Google Scholar 

  • Bekiroglu, Y., Marturi, N., Roa, M. A., Adjigble, K. J. M., Pardi, T., Grimm, C., & Stolkin, R. (2019). Benchmarking protocol for grasp planning algorithms. IEEE Robotics and Automation Letters, 5(2), 315–322.

    Article  Google Scholar 

  • Berenson, D., & Srinivasa, S. S. (2008). Grasp synthesis in cluttered environments for dexterous hands. Paper presented at the 8th IEEE-RAS international conference on humanoid robots.

  • Berscheid, L., Rühr, T., & Kröger, T. (2019). Improving data efficiency of self-supervised learning for robotic grasping. Paper presented at the 2019 international conference on robotics and automation (ICRA).

  • Bhatia Nitin, V. (2010). Survey of nearest neighbor techniques. International Journal of Computer Science and Information Security, 2(2), 302–305.

    Google Scholar 

  • Bohg, J., Morales, A., Asfour, T., & Kragic, D. (2014). Data-driven grasp synthesis—A survey. IEEE Transactions on Robotics, 30(2), 289–309.

    Article  Google Scholar 

  • Bone, G. M., Lambert, A., & Edwards, M. (2008). Automated modeling and robotic grasping of unknown three-dimensional objects. Paper presented at the 2008 IEEE international conference on robotics and automation (ICRA).

  • Bounab, B., Sidobre, D., & Zaatri, A. (2008). Central axis approach for computing n-finger force-closure grasps. Paper presented at the 2008 IEEE international conference on robotics and automation (ICRA).

  • Cai, J., Cheng, H., Zhang, Z., & Su, J. (2019). MetaGrasp: Data efficient grasping by affordance interpreter network. Paper presented at the international conference on robotics and automation (ICRA).

  • Calli, B., Singh, A., Walsman, A., Srinivasa, S., Abbeel, P., & Dollar, A. M. (2015). The ycb object and model set: Towards common benchmarks for manipulation research. Paper presented at the 2015 international conference on advanced robotics (ICAR).

  • Ciocarlie, M., Lackner, C., & Allen, P. (2007). Soft finger model with adaptive contact geometry for grasping and manipulation tasks. Paper presented at the second joint eurohaptics conference and symposium on haptic interfaces for virtual environment and teleoperator systems (WHC'07).

  • Dang, H., & Allen, P. K. (2014). Stable grasping under pose uncertainty using tactile feedback. Autonomous Robots, 36(4), 309–330.

    Article  Google Scholar 

  • Dessalene, E., Ong, Y. H., Morrow, J., Balasubramanian, R., & Grimm, C. (2019). Using geometric features to represent near-contact behavior in robotic grasping. Paper presented at the IEEE international conference on robotics and automation (ICRA).

  • Detry, R., Ek, C. H., Madry, M., Piater, J., & Kragic, D. (2012). Generalizing grasps across partly similar objects. Paper presented at the 2012 IEEE international conference on robotics and automation (ICRA).

  • El-Khoury, S., & Sahbani, A. (2009). On computing robust n-finger force-closure grasps of 3D objects. Paper presented at the 2009 IEEE international conference on robotics and automation (ICRA).

  • Feix, T., Romero, J., Schmiedmayer, H.-B., Dollar, A. M., & Kragic, D. (2015). The grasp taxonomy of human grasp types. IEEE Transactions on Human-Machine Systems, 46(1), 66–77.

    Article  Google Scholar 

  • Fischinger, D., Weiss, A., & Vincze, M. (2015). Learning grasps with topographic features. The International Journal of Robotics Research (IJRR), 34(9), 1167–1194.

    Article  Google Scholar 

  • Frigo, M., & Johnson, S. G. (2005). The design and implementation of FFTW3. Proceedings of the IEEE, 93(2), 216–231.

    Article  Google Scholar 

  • Goldfeder, C., Allen, P. K., Lackner, C., & Pelossof, R. (2007). Grasp planning via decomposition trees. Paper presented at the IEEE international conference on robotics and automation (ICRA).

  • Goldfeder, C., Ciocarlie, M., Dang, H., & Allen, P. K. (2009). The columbia grasp database. Paper presented at the 2009 IEEE international conference on robotics and automation (ICRA).

  • Gualtieri, M., Ten Pas, A., Saenko, K., & Platt, R. (2016). High precision grasp pose detection in dense clutter. Paper presented at the IEEE/RSJ international conference on intelligent robots and systems (IROS).

  • Hasegawa, S., Wada, K., Kitagawa, S., Uchimi, Y., Okada, K., & Inaba, M. (2019). GraspFusion: Realizing complex motion by learning and fusing grasp modalities with instance segmentation. Paper presented at the IEEE international conference on robotics and automation (ICRA).

  • Hegedus, M. J., Gupta, K., & Mehrandezh, M. (2019). Towards an integrated autonomous data-driven grasping system with a mobile manipulator. In IEEE international conference on robotics and automation (ICRA) (pp. 1596–1600).

  • Heinemann, F., Puhlmann, S., Eppner, C., Élvarez-Ruiz, J., Maertens, M., & Brock, O. (2015). A taxonomy of human grasping behavior suitable for transfer to robotic hands. Paper presented at the IEEE international conference on robotics and automation (ICRA).

  • Herzog, A., Pastor, P., Kalakrishnan, M., Righetti, L., Bohg, J., Asfour, T., & Schaal, S. (2014). Learning of grasp selection based on shape-templates. Autonomous Robots, 36(1–2), 51–65.

    Article  Google Scholar 

  • Huebner, K., & Kragic, D. (2008). Selection of robot pre-grasps using box-based shape approximation. Paper presented at the 2008 IEEE/RSJ international conference on intelligent robots and systems (IROS).

  • Jain, S., & Argall, B. (2016). Grasp detection for assistive robotic manipulation. Paper presented at the 2016 IEEE international conference on robotics and automation (ICRA).

  • Karaoguz, H., & Jensfelt, P. (2019). Object detection approach for robot grasp detection. Paper presented at the IEEE international conference on robotics and automation (ICRA).

  • Kavraki, L. E. (1995). Computation of configuration-space obstacles using the fast fourier transform. IEEE Transactions on Robotics and Automation, 11(3), 408–413.

    Article  MathSciNet  Google Scholar 

  • Kleeberger, K., Bormann, R., Kraus, W., & Huber, M. F. (2020). A survey on learning-based robotic grasping. Current Robotics Reports, 1, 1–11.

    Article  Google Scholar 

  • Kumar, R. M., & Sreekumar, K. (2014). A survey on image feature descriptors. The International Journal of Computing Science and Information Technology, 5, 7668–7673.

    Google Scholar 

  • Le, Q. V., Kamm, D., Kara, A. F., & Ng, A. Y. (2010). Learning to grasp objects with multiple contact points. Paper presented at the 2010 IEEE international conference on robotics and automation (ICRA).

  • Lei, Q., Meijer, J., & Wisse, M. (2017). A survey of unknown object grasping and our fast grasping algorithm-C shape grasping. Paper presented at the 2017 3rd International Conference on Control Automation and Robotics.

  • Li, J.-W., Liu, H., & Cai, H.-G. (2003). On computing three-finger force-closure grasps of 2-D and 3-D objects. IEEE Transactions on Robotics and Automation, 19(1), 155–161.

    Article  Google Scholar 

  • Liang, H., Ma, X., Li, S., Görner, M., Tang, S., Fang, B., Zhang, J. (2019). Pointnetgpd: Detecting grasp configurations from point sets. Paper presented at the IEEE international conference on robotics and automation (ICRA).

  • Lu, Q., & Hermans, T. (2019). Modeling grasp type improves learning-based grasp planning. IEEE Robotics and Automation Letters, 4(2), 784–791.

    Article  Google Scholar 

  • Miller, A. T., & Allen, P. K. (1999). Examples of 3D grasp quality computations. Paper presented at the Proceedings 1999 IEEE international conference on robotics and automation (Cat. No. 99CH36288C).

  • Miller, A. T., & Allen, P. K. (2004). Graspit! a versatile simulator for robotic grasping. IEEE Robotics & Automation Magazine, 11(4), 110–122.

    Article  Google Scholar 

  • Miller, A. T., Knoop, S., Christensen, H. I., & Allen, P. K. (2003). Automatic grasp planning using shape primitives. Paper presented at the IEEE international conference on robotics and automation (ICRA).

  • Nguyen, V.-D. (1988). Constructing force-closure grasps. The International Journal of Robotics Research (IJRR), 7(3), 3–16.

    Article  Google Scholar 

  • Nieuwenhuisen, M., Stückler, J., Berner, A., Klein, R., & Behnke, S. (2012). Shape-primitive based object recognition and grasping. Paper presented at the 7th German Conference on Robotics (ROBOTIK).

  • Przybylski, M., Asfour, T., & Dillmann, R. (2011). Planning grasps for robotic hands using a novel object representation based on the medial axis transform. Paper presented at the 2011 IEEE/RSJ international conference on intelligent robots and systems (IROS).

  • Rimon, E., & Blake, A. (1996). Caging 2D bodies by 1-parameter two-fingered gripping systems. Paper presented at the IEEE international conference on robotics and automation (ICRA).

  • Rojas, N., & Dollar, A. M. (2016). Classification and kinematic equivalents of contact types for fingertip-based robot hand manipulation. Journal of Mechanisms and Robotics, 8(4), 041041.

    Article  Google Scholar 

  • Segal, A., Haehnel, D., & Thrun, S. (2009). Generalized-icp. Paper presented at the Robotics: Science and Systems.

  • Seita, D., Pokorny, F. T., Mahler, J., Kragic, D., Franklin, M., Canny, J., & Goldberg, K. (2016). Large-scale supervised learning of the grasp robustness of surface patch pairs. Paper presented at the 2016 IEEE Int. Conf. on Simul., Model., and Program. for Auton. Robots (SIMPAR).

  • Shao, L., Ferreira, F., Jorda, M., Nambiar, V., Luo, J., Solowjow, E., & Bohg, J. (2020). Unigrasp: Learning a unified model to grasp with multifingered robotic hands. IEEE Robotics and Automation Letters, 5(2), 2286–2293.

    Article  Google Scholar 

  • Stival, F., Michieletto, S., Cognolato, M., Pagello, E., Müller, H., & Atzori, M. (2019). A quantitative taxonomy of human hand grasps. Journal of NeuroEngineering and Rehabilitation, 16(1), 1–17.

    Article  Google Scholar 

  • Ten Pas, A. (2017). Grasp pose generation. Retrieved from https://github.com/atenpas/gpg

  • Ten Pas, A., Gualtieri, M., Saenko, K., & Platt, R. (2017). Grasp pose detection in point clouds. The International Journal of Robotics Research (IJRR), 36(13–14), 1455–1473.

    Google Scholar 

  • Wang, B., Jiang, L., Li, J., & Cai, H. (2005). Grasping unknown objects based on 3d model reconstruction. Paper presented at the 2005 IEEE/ASME international conference on advanced intelligent mechatronics (AIM).

  • Weisz, J., & Allen, P. K. (2012). Pose error robust grasping from contact wrench space metrics. Paper presented at the IEEE international conference on robotics and automation (ICRA).

  • Zhu, X., & Ding, H. (2004). Planning force-closure grasps on 3-D objects. Paper presented at the IEEE Int. Conf. Robot. Autom. (ICRA).

Download references

Funding

The funding was provided by Natural Sciences and Engineering Research Council of Canada.

Author information

Authors and Affiliations

  1. RAMP Lab, School of Engineering Science, Simon Fraser University, Burnaby, BC, V5A 1S6, Canada

    Michael Hegedus & Kamal Gupta

  2. Faculty of Engineering and Applied Science, University of Regina, Regina, SK, S4S 0A2, Canada

    Mehran Mehrandezh

Authors
  1. Michael Hegedus
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kamal Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mehran Mehrandezh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Hegedus.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Kamal Gupta and Mehran Mehrandezh: Research supported by NSERC Discovery and RTI Grants.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (MP4 5854 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hegedus, M., Gupta, K. & Mehrandezh, M. Efficiently finding poses for multiple grasp types with partial point clouds by uncoupling grasp shape and scale. Auton Robot 46, 749–767 (2022). https://doi.org/10.1007/s10514-022-10049-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-022-10049-6

Keywords

Search

Navigation