Skip to main content
SpringerLink
Log in

A kernel-based approach to learning contact distributions for robot manipulation tasks

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

Manipulation tasks often require robots to recognize interactions between objects. For example, a robot may need to determine if it has grasped an object properly or if one object is resting on another in a stable manner. These interactions usually depend on the contacts between the objects, with different distributions of contacts affording different interactions. In this paper, we address the problem of learning to recognize interactions between objects based on contact distributions. We present a kernel-based approach for representing the estimated contact distributions. The kernel can be used for various interactions, and it allows the robot to employ a variety of kernel methods from machine learning. The approach was evaluated on blind grasping, lifting, and stacking tasks. Using 30 training samples and the proposed kernel, the robot already achieved classification accuracies of 71.9, 85.93, and 97.5% for the blind grasping, lifting and stacking tasks respectively. The kernel was also used to cluster interactions using spectral clustering. The clustering method successfully differentiated between different types of interactions, including placing, inserting, and pushing. The contact points were extracted using tactile sensors or 3D point cloud models of the objects. The robot could construct small towers of assorted blocks using the classifier for the stacking task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Abdo, N., Kretzschmar, H., Spinello, L., & Stachniss, C. (2013). Learning manipulation actions from a few demonstrations. In: International conference on robotics and automation (ICRA)

  • Amores, J. (2013). Multiple instance classification: Review, taxonomy and comparative study. Artificial Intelligence, 201, 81–105.

    Article  MathSciNet  MATH  Google Scholar 

  • Bekiroglu, Y., Detry, R., & Kragic, D. (2011) Learning tactile characterizations of object- and pose-specific grasps. In: International conference on intelligent robots and systems (IROS)

  • Ben Amor, H., Kroemer, O., Hillenbrand, U., Neumann, G., & Peters, J. (2012). Generalization of human grasping for multi-fingered robot hands. In: International conference on intelligent robots and systems (IROS)

  • Bicchi, A., & Kumar, V. (2000). Robotic grasping and contact: A review. In: International conference on robotics and automation (ICRA)

  • Bohg, J., Johnson-Roberson, M., León, B., Felip, J., Gratal, X., Bergström, N., Kragic, D., & Morales, A. (2011). Mind the gap–robotic grasping under incomplete observation. In: International conference on robotics and automation (ICRA)

  • Bohg, J., Morales, A., Asfour, T., & Kragic, D. (2014). Data-driven grasp synthesis-a survey. IEEE Transactions on Robotics, 30, 289–309.

    Article  Google Scholar 

  • Boularias, A., Kroemer, O., & Peters, J. (2011). Learning robot grasping from 3d images with markov random fields. In: International conference on intelligent robot systems (IROS)

  • Chebotar, Y., Hausman, K., Kroemer, O., Sukhatme, G., & Schaal, S. (2016). Generalizing regrasping with supervised policy learning. In: International symposium on experimental robotics (ISER)

  • Chen, Z., Lii, N. Y., Wimboeck, T., Fan, S., Jin, M., Borst, C., & Liu, H. (2010). Experimental study on impedance control for the five-finger dexterous robot hand dlr-hit ii. In: International conference on intelligent robots and systems (IROS)

  • Dang, H., & Allen, P. K. (2012). Learning grasp stability. In: International conference on robotics and automation (ICRA)

  • Detry, R., Ek, C. H., Madry, M., Piater, J., & Kragic, D. (2012). Generalizing grasps across partly similar objects. In: International conference on robotics and automation (ICRA)

  • Eppner, C., & Brock, O. (2013). Grasping unknown objects by exploiting shape adaptability and environmental constraints. In: International conference on intelligent robots and systems (IROS)

  • Ferrari, C., & Canny, J. (1992). Planning optimal grasps. In: International conference on robotics and automation (ICRA) pp 2290–2295

  • Gibson, J. J. (1986). The ecological approach to visual perception. Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Griffith, S., Sinapov, J., Sukhoy, V., & Stoytchev, A. (2012). A behavior-grounded approach to forming object categories: Separating containers from noncontainers. IEEE Transactions on Autonomous Mental Development, 4(1), 54–69.

    Article  Google Scholar 

  • Goldfeder, C., Ciocarlie, M., Dang, H., & Allen, P. K. (2009). The Columbia grasp database. In: International conference on robotics and automation (ICRA)

  • Hermans, T., Li, F, Rehg, J. M., & Bobick, A. F. (2013). Learning contact locations for pushing and orienting unknown objects. In: International conference on humanoid robots

  • Herzog, A., Pastor, P., Kalakrishnan, M., Righetti, L., Bohg, J., Asfour, T., et al. (2013). Learning of grasp selection based on shape-templates. Autonomous Robots, 36, 51–65.

    Article  Google Scholar 

  • Hofmann, T., Schölkopf, B., & Smola, A. J. (2008). Kernel methods in machine learning. Annals of Statistics

  • Jebara, T., Kondor, R., & Howard, A. (2004). Probability product kernels. Journal of Machine Learning Research (JMLR), 5, 819–844.

  • Jebara, T., Kondor, R., & Howard, A. (2004). Probability product kernels. Journal of Machine Learning Research (JMLR), 5, 819–844.

    MathSciNet  MATH  Google Scholar 

  • Jenssen, R., Principe, J. C., Erdogmus, D., & Eltoft, T. (2006). The cauchy-schwarz divergence and parzen windowing: Connections to graph theory and mercer kernels. Journal of the Franklin Institute, 343(6), 614–629.

    Article  MathSciNet  MATH  Google Scholar 

  • Jentoft, L. P., Tenzer, Y., Vogt, D., Liu, J., Wood, R. J., Howe, R. D. (2013). Flexible, stretchable tactile arrays from mems barometers. In: International conference on advanced robotics (ICAR)

  • Jiang, Y., Lim, M., Zheng, C., & Saxena, A. (2012). Learning to place new objects in a scene. International Journal of Robotic Research (IJRR), 31(9), 1021–1043.

    Article  Google Scholar 

  • Kappler, D., Bohg, B., & Schaal, S. (2015). Leveraging big data for grasp planning. In: IEEE international conference on robotics and automation (ICRA)

  • Karayiannidis, Y., Smith, C., Vina, F. E., & Kragic, D. (2014). Online contact point estimation for uncalibrated tool use. In: International conference on robotics and automation (ICRA)

  • Kopicki, M., Detry, R., Adjigble, M., Stolkin, R., Leonardis, A., & Wyatt, J. L. (2015). One shot learning and generation of dexterous grasps for novel objects. The International Journal of Robotics Research (IJRR), 35, 959–976.

    Article  Google Scholar 

  • Kopicki, M. S., Zurek, S., Stolkin, R., Morwald, T., & Wyatt, J. L. (2011). Learning to predict how rigid objects behave under simple manipulation. In: International conference on robotics and automation (ICRA)

  • Kroemer, O., Ben Amor, H., Ewerton, M., & Peters, J. (2012a). Point cloud completion using extrusions. In: The international conference on humanoid robots

  • Kroemer, O., Daniel, C., Neumann, G., van Hoof, H., & Peters, J. (2015). Towards learning hierarchical skills for multi-phase manipulation tasks. In: International conference on robotics and automation (ICRA)

  • Kroemer, O., & Peters, J. (2014). Predicting object interactions from contact distributions. In: International conference on intelligent robots and systems

  • Kroemer, O., Ugur, E., Oztop, E., & Peters, J. (2012b). A kernel-based approach to direct action perception. In: International conference on robotics and automation (ICRA)

  • Kulick, J., Lang, T., Toussaint, M., & Lopes, M. (2013). Active Learning for Teaching a Robot Grounded Relational Symbols. In: International joint conference on artificial intelligence (IJCAI)

  • Laaksonen, J., Nikandrova, E., & Kyrki, V. (2012). Probabilistic sensor-based grasping. In: International conference on intelligent robots and systems (IROS)

  • Leischnig, S., Luettgen, S., Kroemer, O., & Peters, J. (2015). A comparison of contact distribution representations for learning to predict object interactions. In: International conference on humanoid robots

  • Lenz, I., Lee, H., & Saxena, A. (2013). Deep learning for detecting robotic grasps. In: Robotics: Science and systems (RSS)

  • Li, Q., Schürmann, C., Haschke, R., & Ritter, H. J. (2013) A control framework for tactile servoing. In: Robotics: Science and systems (R:SS)

  • Li, Z., & Sastry, S. S. (1988). Task-oriented optimal grasping by multifingered robot hands. Journal of Robotics and Automation, 4(1), 32–44.

  • Luxburg, U. (2007). A tutorial on spectral clustering. Statistics and Computing, 17(4), 395–416.

    Article  MathSciNet  Google Scholar 

  • Madry, M., Bo, L., Kragic, D., & Fox, D. (2014). ST-HMP: Unsupervised spatio-temporal feature learning for tactile data. In: International conference on robotics and automation (ICRA)

  • Miller, A., & Allen, P. (1999). Examples of 3d grasp quality computations. In: International conference on robotics and automation (ICRA)

  • Miller, A., & Allen, P. (2004). Graspit!: A versatile simulator for robotic grasping. IEEE Robotics and Automation Magazine, 11, 110–122.

    Article  Google Scholar 

  • Molchanov, A., Kroemer, O., Su, Z., & Sukhatme, G. S. (2016). Contact localization on grasped objects using tactile sensing. In: International conference on intelligent robots and systems (IROS)

  • Montesano, L., Lopes, M., Bernardino, A., & Santos-Victor, J. (2007). Modeling affordances using bayesian networks. In: International conference on intelligent robot systems (IROS)

  • Ning, X., & Karypis, G. (2008). The set classification problem and solution methods. In: International conference on data mining workshops

  • Odhner, L. U., Jentoft, L. P., Claffee, M. R., Corson, N., Tenzer, Y., Ma, R. R., et al. (2014). A compliant, underactuated hand for robust manipulation. The International Journal of Robotics Research (IJRR), 33(5), 736–752.

    Article  Google Scholar 

  • Roa, M. A., & Suàrez, R. (2015). Grasp quality measures: Review and performance. Autonomous Robots, 38, 65–88.

    Article  Google Scholar 

  • Rosman, B., & Ramamoorthy, S. (2011). Learning spatial relationships between objects. The International Journal of Robotics Research, 30, 1328–1342.

    Article  Google Scholar 

  • Rusu, R. B., & Cousins, S. (2011). 3D is here: Point Cloud Library (PCL). In: International conference on robotics and automation (ICRA)

  • Sahin, E., Cakmak, M., Dogar, M. R., Ugur, E., & Ucoluk, G. (2007). To Afford or Not to Afford: A New Formalization of Affordances Toward Affordance-Based Robot Control. Adaptive Behavior, 4, 447–472.

    Article  Google Scholar 

  • Schölkopf, B., & Smola, A. J. (2001). Learning with kernels: Support vector machines, regularization, optimization, and beyond (1st ed.). Cambridge: The MIT Press.

    Google Scholar 

  • Shi, J., & Malik, J. (2000). Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), 888–905.

  • Sjoo, K., & Jensfelt, P. (2011). Learning spatial relations from functional simulation. In: International conference on intelligent robots and systems (IROS)

  • ten Pa, A., & Platt, R. (2015). Using geometry to detect grasp poses in 3d point clouds. In: International symposium on robotics research (ISRR)

  • Trinkle, J., & Paul, R. P. (1990). Planning for dextrous manipulation with sliding contacts. International Journal of Robotics Research, 9(3), 24–48.

    Article  Google Scholar 

  • Ugur, E., & Piater, J. (2015). Bottom-up learning of object categories, action effects and logical rules: From continuous manipulative exploration to symbolic planning. In: International conference on robotics and automation (ICRA), pp 2627–2633

  • Vedaldi, A., Gulshan, V., Varma, M., & Zisserman, A. (2009). Multiple kernels for object detection. In: International conference on computer vision (ICCV)

  • Veiga, F., van Hoof, H., Peters, J., & Hermans, T. (2015). Stabilizing novel objects by learning to predict tactile slip. In: International conference on intelligent robots and systems (IROS)

  • Will, P. M., & Grossman, D. D. (1975). An experimental system for computer controlled mechanical assembly. IEEE Transactions on Computers, 24(9), 879–888.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Southern California, Los Angeles, CA, USA

    Oliver Kroemer

  2. Technische Universitaet Darmstadt, Darmstadt, Germany

    Simon Leischnig, Stefan Luettgen & Jan Peters

  3. MPI for Intelligent Systems, Tuebingen, Germany

    Jan Peters

Authors
  1. Oliver Kroemer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simon Leischnig
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefan Luettgen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jan Peters
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oliver Kroemer.

Additional information

The research leading to these results has received funding from the European Community’s Seventh Framework Programme under Grant Agreements 610878 (3rdHand), 600716 (CoDyCo), and 610967 (TACMAN).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kroemer, O., Leischnig, S., Luettgen, S. et al. A kernel-based approach to learning contact distributions for robot manipulation tasks. Auton Robot 42, 581–600 (2018). https://doi.org/10.1007/s10514-017-9651-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-017-9651-z

Keywords

Search

Navigation