Skip to main content
Log in

Data-driven grasping

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

This paper propose a novel framework for a data driven grasp planner that indexes partial sensor data into a database of 3D models with known grasps and transfers grasps from those models to novel objects. We show how to construct such a database and also demonstrate multiple methods for matching into it, aligning the matched models with the known sensor data of the object to be grasped, and selecting an appropriate grasp to use. Our approach is experimentally validated in both simulated trials and trials with robots.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aleotti, J., & Caselli, S. (2007). Robot grasp synthesis from virtual demonstration and topology-preserving environment reconstruction. In IEEE international conference on intelligent robots and systems.

    Google Scholar 

  • Aydin, Y., & Nakajima, M. (1999). Database guided computer animation of human grasping using forward and inverse kinematics. Computers & Graphics, 23(1).

  • Balasubramanian, R., Xu, L., Brook, P. D., Smith, J. R., & Matsuoka, Y. (2010). Human-guided grasp measures improve grasp robustness on physical robot. In IEEE international conference on robotics and automation.

    Google Scholar 

  • Belongie, S., & Malik, J. (2000). Matching with shape contexts. In Workshop on content-based access of image and video libraries.

    Google Scholar 

  • Biederman, I. (1995). Visual object recognition. In S. F. Kosslyn & D. N. Osherson (Eds.), An invitation to cognitive science: Vol. 2 (2nd ed.). Chap. 4.

    Google Scholar 

  • Bohg, J., & Kragic, D. (2010). Learning grasping points with shape context. Robotics and Autonomous Systems, 58(4).

  • Bowers, D. L., & Lumia, R. (2003). Manipulation of unmodeled objects using intelligent grasping schemes. IEEE Transactions on Fuzzy Systems, 11(3).

  • Chen, D.-Y., Ouhyoung, M., Tian, X. P., & Shen, Y. T. (2003). On visual similarity based 3d model retrieval. In Eurographics.

    Google Scholar 

  • Ciocarlie, M., & Allen, P. K. (2009). Hand posture subspaces for dexterous robotic grasping. The International Journal of Robotics Research, 28, 851–867.

    Article  Google Scholar 

  • Ciocarlie, M., Goldfeder, C., & Allen, P. K. (2007a). Dimensionality reduction for hand-independent dexterous robotic grasping. In IEEE international conference on intelligent robots and systems.

    Google Scholar 

  • Ciocarlie, M., Lackner, C., & Allen, P. K. (2007b). Soft finger model with adaptive contact geometry for grasping and manipulation tasks. In IEEE world haptics.

    Google Scholar 

  • Dueck, G., & Scheuer, T. (1990). Threshold accepting: a general purpose optimization algorithm appearing superior to simulated annealing. Journal of Computational Physics, 90(1).

  • Ferrari, C., & Canny, J. (2002). Planning optimal grasps. In IEEE international conference on robotics and automation.

    Google Scholar 

  • Gelfand, N., Mitra, N. J., Guibas, L. J., & Pottmann, H. (2005). Robust global registration. In Symposium on geometry processing.

    Google Scholar 

  • Glover, J., Rus, D., & Roy, N. (2008). Probabilistic models of object geometry for grasp planning. In Robotics: science and systems.

    Google Scholar 

  • Goldfeder, C., & Allen, P. K. (2008). Autotagging to improve text search for 3d models. In Joint conference on digital libraries.

    Google Scholar 

  • Goldfeder, C., Ciocarlie, M., Dang, H., & Allen, P. K. (2009a). The Columbia grasp database. In IEEE international conference on robotics and automation.

    Google Scholar 

  • Goldfeder, C., Ciocarlie, M., Peretzman, J., Dang, H., & Allen, P. K. (2009b). Data-driven grasping with partial sensor data. In IEEE international conference on intelligent robots and systems.

    Google Scholar 

  • Huber, D. F., & Hebert, M. (2003). Fully automatic registration of multiple 3d data sets. Image and Vision Computing, 21(7).

  • Jaccard, P. (1901). Étude comparative de la distribution florale dans une portion des alpes et des jura. Bulletin del la Société Vaudoise des Sciences Naturelles, 37.

  • Johnson, A. E., & Hebert, M. (1997). Surface registration by matching oriented points. In 3-D digital imaging and modeling.

    Google Scholar 

  • Kullback, S., & Leibler, R. A. (1951). On information and sufficiency. Annals of Mathematical Statistics, 22.

  • Landy, M. S., & Graham, N. (2004). Visual perception of texture. In L. M. Chalupa & J. S. Werner (Eds.), The visual neurosciences. Chap. 2.

    Google Scholar 

  • Leung, T., & Malik, J. (2001). Representing and recognizing the visual appearance of materials using three-dimensional textons. International Journal of Computer Vision, 43(1).

  • Li, Y., & Pollard, N. S. (2005). A shape matching algorithm for synthesizing humanlike enveloping grasps. In Humanoid robots.

    Google Scholar 

  • Liu, T., Moore, A., Gray, A., & Yang, K. (2004). An investigation of practical approximate nearest neighbor algorithms. In Conference on neural information processing systems.

    Google Scholar 

  • Lowe, D. G. (1999). Object recognition from local scale-invariant features. In IEEE international conference on computer vision.

    Google Scholar 

  • Makadia, A., Patterson, A., & Daniilidis, K. (2006). Fully automatic registration of 3D point clouds. In IEEE conference on computer vision and pattern recognition.

    Google Scholar 

  • Miller, A., & Allen, P. K. (2004). Graspit!: a versatile simulator for robotic grasping. IEEE Robotics and Automation Magazine, 11(4).

  • Morales, A., Asfour, T., Azad, P., Knoop, S., & Dillmann, R. (2006). Integrated grasp planning and visual object localization for a humanoid robot with five-fingered hands. In IEEE international conference on intelligent robots and systems.

    Google Scholar 

  • Novotni, M., & Klein, R. (2003). 3D Zernike descriptors for content based shape retrieval. In ACM symposium on solid modeling and applications.

    Google Scholar 

  • Nowak, E., Jurie, F., & Triggs, B. (2006). Sampling strategies for bag-of-features image classification. In European conference on computer vision.

    Google Scholar 

  • Ohbuchi, R., Osada, K., Furuya, T., & Banno, T. (2008). Salient local visual featuers for shape-based 3D model retrieval. In IEEE international conference on shape modeling and applications.

    Google Scholar 

  • Papadakis, P., Pratikakis, I., Theoharis, T., & Perantonis, S. (2010). PANORAMA: a 3D shape descriptor based on panoramic views for unsupervised 3d object retrieval. International Journal of Computer Vision, Special Issue on: 3D Object Retrieval, 89(2).

  • Pelossof, R., Miller, A., Allen, P. K., & Jebara, T. (2004). An svm learning approach to robotic grasping. In IEEE international conference on robotics and automation.

    Google Scholar 

  • Platt, R., Fagg, A. H., & Grupen, R. (2002). Nullspace composition of control laws for grasping. In IEEE international conference on robotics and automation.

    Google Scholar 

  • Romea, A. C., & Srinivasa, S. (2010). Efficient multi-view object recognition and full pose estimation. In IEEE international conference on robotics and automation.

    Google Scholar 

  • Rother, C., Kolmogorov, V., & Blake, A. (2004). Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics, 23.

  • Rusinkiewicz, S., & Levoy, M. (2001). Efficient variants of the icp algorithm. In IEEE international conference on 3-D digital imaging and modeling.

    Google Scholar 

  • Salvi, J., Matabosch, C., Fofi, D., & Forest, J. (2007). A review of recent range image registration methods with accuracy evaluation. Image and Vision Computing, 25(5).

  • Saxena, A., Driemeyer, J., & Ng, A. Y. (2008). Robotic grasping of novel objects using vision. International Journal of Robotics Research.

  • Shilane, P., Min, P., Kazhdan, M., & Funkhouser, T. (2004). The Princeton shape benchmark. In IEEE international conference on shape modeling and applications.

    Google Scholar 

  • Sivamani, R. K., Goodman, J., Gitis, N. V., & Maibach, H. I. (2003). Coefficient of friction: tribological studies in man—an overview. Skin Research and Technology, 9(3).

  • Srinivasa, S., Ferguson, D., Helfrich, C., Berenson, D., Romea, A. C., Diankov, R., Gallagher, G., Hollinger, G., Kuffner, J., & Vandeweghe, J. M. (2010). HERB: a home exploring robotic butler. Autonomous Robots, 28(1).

  • Torres, M. M., Romea, A. C., & Srinivasa, S. (2010). Moped: a scalable and low latency object recognition and pose estimation system. In IEEE international conference on robotics and automation.

    Google Scholar 

  • Triggs, B., Mclauchlan, P. F., Hartley, R. I., & Fitzgibbon, A. W. (2000). Lecture notes in computer science: Vol. 1883. Bundle adjustment—a modern synthesis.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Corey Goldfeder.

Additional information

This work was funded in part by NIH BRP grant 1RO1 NS 050256-01A2 and a Google research grant. We would like to thank Siddhartha Srinivasan, Dmitry Berenson and Mehmet Dogar from Intel Pittsburgh Lab for their help in providing access to the HERB system.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

(MP4 35636 KB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Goldfeder, C., Allen, P.K. Data-driven grasping. Auton Robot 31, 1–20 (2011). https://doi.org/10.1007/s10514-011-9228-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-011-9228-1

Keywords

Navigation