Skip to main content
SpringerLink
Log in

Def-Grasp: A Robot Grasping Detection Method for Deformable Objects Without Force Sensor

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Learning-based robot grasping of deformable objects without force sensors is a challenging task. This paper proposes a method without force sensors to predict deformation properties and the grasp poses of objects to solve the problem of deformable objects damaged by the gripper. Firstly, the deformation representation of the object is defined according to the deformation ratio of the object grasped. Then, the lightweight dataset with object deformation properties is produced based on the truncated signed distance function technique. The dataset can render images from any perspective for the training network. Subsequently, we investigate the synergies between grasp pose and gripper closure and use the above lightweight dataset to train deep network. The resulting network, Def-Grasp, can predict object grasp poses and deformation properties, the gripper closure is calculated using the network predictions. Finally, the expected experimental results show that the deformable objects in the cluttered scene are not damaged by the gripper during the 6DoF grasping process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Du G, Wang K, Lian S, Zhao K (2021) Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review. Artif Intell Rev 54(3):1677–1734

    Article  Google Scholar 

  2. Kleeberger K, Bormann R, Kraus W, Huber MF (2020) A survey on learning-based robotic grasping. Current Robotics Reports pp. 1–11

  3. Deng X, Xiang Y, Mousavian A, Eppner C, Bretl T, Fox D (2020) In: 2020 IEEE International Conference on Robotics and Automation (ICRA) (IEEE), pp. 3665–3671

  4. Liang H, Ma X, Li S, Görner M, Tang S, Fang B, Sun F, Zhang J (2019) In: 2019 International Conference on Robotics and Automation (ICRA) (IEEE), pp. 3629–3635

  5. Mousavian A, Eppner C, Fox D (2019) In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2901–2910

  6. Hagelskjær F, Buch AG (2020) In: 2020 IEEE International Conference on Image Processing (ICIP) (IEEE), pp. 2641–2645

  7. Yang D, Tosun T, Eisner B, Isler V, Lee D (2021) In: 2021 IEEE International Conference on Robotics and Automation (ICRA) (IEEE), pp. 6350–6356

  8. Song S, Zeng A, Lee J, Funkhouser T (2020) Grasping in the wild: learning 6dof closed-loop grasping from low-cost demonstrations. IEEE Robot Autom Lett 5(3):4978–4985

    Article  Google Scholar 

  9. Xu J, Danielczuk M, Ichnowski J, Mahler J, Steinbach E, Goldberg K (2020) In: 2020 IEEE International Conference on Robotics and Automation (ICRA) (IEEE), pp. 1546–1552

  10. Sanchez J, Corrales JA, Bouzgarrou BC, Mezouar Y (2018) Robotic manipulation and sensing of deformable objects in domestic and industrial applications: a survey. Int J Robot Res 37(7):688–716

    Article  Google Scholar 

  11. Le TN, Lundell J, Kyrki V (2020) In: 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC) (IEEE), pp. 342–349

  12. Le TN, Lundell J, Abu-Dakka FJ, Kyrki (2021) Deformation-aware data-driven grasp synthesis. arXiv preprint arXiv:2109.05320

  13. Li T, Chi C, Wang C, Xue N, Liu C, Liu C (2018) In: 2018 2nd IEEE advanced information management, communicates, electronic and automation control conference (IMCEC) (IEEE), pp. 732–736

  14. James JW, Lepora NF (2020) Slip detection for grasp stabilization with a multifingered tactile robot hand. IEEE Trans Robot 37(2):506–519

    Article  Google Scholar 

  15. Garcia-Garcia A, Zapata-Impata BS, Orts-Escolano S, Gil P, Garcia-Rodriguez J (2019) In: 2019 International Joint Conference on Neural Networks (IJCNN) (IEEE), pp. 1–8

  16. Redmon J, Angelova A (2015) In: 2015 IEEE International Conference on Robotics and Automation (ICRA) (IEEE), pp. 1316–1322

  17. Chu FJ, Xu R, Vela PA (2018) Real-world multiobject, multigrasp detection. IEEE Robot Autom Lett 3(4):3355–3362

    Article  Google Scholar 

  18. Yan X, Hsu J, Khansari M, Bai Y, Pathak A, Gupta A, Davidson J, Lee H (2018) In: 2018 IEEE International Conference on Robotics and Automation (ICRA) (IEEE), pp. 3766–3773

  19. Fang HS, Wang C, Gou M, Lu C (2020) In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11,444–11,453

  20. Huang I, Narang Y, Eppner C, Sundaralingam B, Macklin M, Hermans T, Fox D (2021) Defgraspsim: Simulation-based grasping of 3d deformable objects. arXiv preprint arXiv:2107.05778

  21. Le TN, Lundell J, Abu-Dakka FJ, Kyrki V (2021) Towards synthesizing grasps for 3d deformable objects with physics-based simulation. arXiv preprint arXiv:2107.08898

  22. Zeng A, Song S, Welker S, Lee J, Rodriguez A, Funkhouser T (2018) In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE), pp. 4238–4245

  23. Zhang H, Lan X, Bai S, Zhou X, Tian Z, Zheng N (2019) In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE), pp. 4768–4775

  24. Newcombe RA (2011) Real-time dense surface mapping and tracking. IEEE ISMAR, IEEE, October 2011

  25. Depierre A, Dellandréa E, Chen L (2018) In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE), pp. 3511–3516

  26. Dharbaneshwer S, Subramanian SJ, Kohlhoff K (2019) Robotic grasp analysis using deformable solid mechanics. Meccanica 54(11):1767–1784

    Article  MathSciNet  Google Scholar 

  27. Dharbaneshwer S, Thondiyath A, Subramanian SJ, Chen IM (2021) A finite element based simulation framework for robotic grasp analysis. Proc Instit Mech Eng Part C J Mech Eng Sci 235(13):2482–2495

    Article  Google Scholar 

  28. Dharbaneshwer S, Thondiyath A, Subramanian SJ, Chen I et al (2021) Finite element-based grasp analysis using contact pressure maps of a robotic gripper. J Brazilian Soc Mech Sci Eng 43(4):1–11

    Article  Google Scholar 

  29. Dharbaneshwer S, Thondiyath A (2021) Contact area-based modeling of robotic grasps using deformable solid mechanics. Int J Appl Mech 13(03):2150038

    Article  Google Scholar 

  30. Calli B, Singh A, Bruce J, Walsman A, Konolige K, Srinivasa S, Abbeel P, Dollar AM (2017) Yale-cmu-Berkeley dataset for robotic manipulation research. Int J Robot Res 36(3):261–268

    Article  Google Scholar 

  31. Qi CR, Su H, Mo K, Guibas LJ (2017) In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652–660

  32. Qi CR, Yi L, Su H, Guibas LJ (2017) Pointnet++: Deep hierarchical feature learning on point sets in a metric space. arXiv preprint arXiv:1706.02413

  33. Guerrero P, Kleiman Y, Ovsjanikov M, Mitra NJ (2018) in Computer Graphics Forum, vol. 37 (Wiley Online Library), pp. 75–85

  34. Landrieu L, Simonovsky M (2018) In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 4558–4567

  35. Murali A, Mousavian A, Eppner C, Paxton C, Fox D (2020) In: 2020 IEEE International Conference on Robotics and Automation (ICRA) (IEEE), pp. 6232–6238

  36. Azpiri AI, Ortega EL, Cobo AA (2021) Affordance-based grasping point detection using graph convolutional networks for industrial bin-picking applications. Sensors 21(3):816

    Article  Google Scholar 

  37. Dong H, Chen CY, Qiu C, Yeow CH, Yu H (2021) Gsg: A granary soft gripper with mechanical force sensing via 3-dimensional snap-through structure. arXiv preprint arXiv:2111.04046

  38. D’Avella S, Fontana M, Vertechy R, Tripicchio P (2022) In: 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE) (IEEE), pp. 1309–1314

  39. Judd E, Aksoy B, Digumarti KM, Shea H, Floreano D (2022) In: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE), pp. 10,003–10,008

  40. Cui S, Wang R, Wei J, Li F, Wang S (2020) In: 2020 IEEE International Conference on Robotics and Automation (ICRA) (IEEE), pp. 538–544

  41. Ruan W, Zhu W, Wang K, Lu Q, Yeh W, Luo L, Su C, Wang Q (2022) In: International Conference on Cognitive Systems and Signal Processing (Springer), pp. 593–604

  42. Jiang Y, Moseson S, Saxena A (2011) In: 2011 IEEE International conference on robotics and automation (IEEE), pp. 3304–3311

  43. Mahler J, Liang J, Niyaz S, Laskey M, Doan R, Liu X, Ojea JA, Goldberg K (2017) Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. arXiv preprint arXiv:1703.09312

  44. Kehl W, Manhardt F, Tombari F, Ilic S, Navab N (2107) In: Proceedings of the IEEE international conference on computer vision, pp. 1521–1529

  45. Qi CR, Litany O, He K, Guibas LJ (2019) In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9277–9286

  46. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

  47. Qin Y, Chen R, Zhu H, Song M, Xu J, Su H (2020) In: Conference on robot learning (PMLR), pp. 53–65

  48. Zhao B, Zhang H, Lan X, Wang H, Tian Z, Zheng N (2021) In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 13,474–13,480. https://doi.org/10.1109/ICRA48506.2021.9561920

Download references

Author information

Authors and Affiliations

  1. College of Information and Engineering, Zhejiang University of Technology, Hangzhou, Zhejiang, China

    Xinyi Yu, Rui Huang, Chongliang Zhao, Libo Zhou & Linlin Ou

Authors
  1. Xinyi Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rui Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chongliang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Libo Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Linlin Ou
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Prof. Yu provided the core idea of the article. Mr. Huang and Mr. Zhao wrote the code and article writing. Dr. Zhou and Prof. Ou reviewed the manuscript.

Corresponding author

Correspondence to Linlin Ou.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, X., Huang, R., Zhao, C. et al. Def-Grasp: A Robot Grasping Detection Method for Deformable Objects Without Force Sensor. Neural Process Lett 55, 11739–11756 (2023). https://doi.org/10.1007/s11063-023-11398-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-023-11398-8

Keywords

Search

Navigation