Skip to main content
SpringerLink
Log in

Real-World Semantic Grasp Detection Using Ontology Features: Learning to Concentrate on Object Features

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Recognizing the category of the object and using the features of the object itself to predict grasp configuration is of great significance to improve the accuracy of the grasp detection model and expand its application. Unfortunately, most current research methods predict the potential grasping position in the scene through global or local features, which makes the model’s performance easily affected by the shifts in background features. Therefore, in this paper, we propose an attention-based end-to-end grasp detection model, which uses semantic segmentation to distinguish object features and background features in the input image and guides the model to focus on the features of the target object ontology during the training process. This method effectively reduces the background features that are weakly correlated to the target object, thus making the features more unique and guaranteeing the accuracy and efficiency of grasp detection. Experimental results show that the proposed method can achieve 98.36% accuracy in Cornell Grasp Dataset. Furthermore, our results on complex multi-object scenarios or more rigorous evaluation metrics show the domain adaptability of our method over the state-of-the-art.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Redmon J, Angelova A (2015) Real-time grasp detection using convolutional neural networks. In: 2015 IEEE international conference on robotics and automation (ICRA). IEEE, pp 1316–1322

  2. Shao Z, Qu Y, Ren G, Wang G, Guan Y, Shi Z, Tan J (2020) Batch normalization masked sparse autoencoder for robotic grasping detection. In: 2020 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 9614–9619

  3. Cheng H, Ho D, Meng MQ-H (2020) High accuracy and efficiency grasp pose detection scheme with dense predictions. In: 2020 IEEE international conference on robotics and automation (ICRA). IEEE, pp 3604–3610

  4. Kumra S, Kanan C (2017) Robotic grasp detection using deep convolutional neural networks. In: 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 769–776

  5. Guo D, Sun F, Liu H, Kong T, Fang B, Xi N (2017) A hybrid deep architecture for robotic grasp detection. In: 2017 IEEE international conference on robotics and automation (ICRA). IEEE, pp 1609–1614

  6. Chu F-J, Xu R, Vela PA (2018) Real-world multiobject, multigrasp detection. IEEE Robot Autom Lett 3(4):3355–3362

    Article  Google Scholar 

  7. Zhou X, Lan X, Zhang H, Tian Z, Zhang Y, Zheng N (2018) Fully convolutional grasp detection network with oriented anchor box. In: 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 7223–7230

  8. Park D, Seo Y, Chun SY (2020) Real-time, highly accurate robotic grasp detection using fully convolutional neural network with rotation ensemble module. In: 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, pp 9397–9403

  9. Ren S, He K, Girshick R, Sun J (2015) Faster rcnn: towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst 28 :91–99

  10. Jia Q, Cao Z, Zhao X, Pang L, Yu Y, Yu U (2018) Object recognition, localization and grasp detection using a unified deep convolutional neural network with multi-task loss. In: 2018 IEEE international conference on robotics and biomimetics (ROBIO). IEEE, pp. 1557–1562

  11. Zhang H, Lan X, Bai S, Wan L, Yang C, Zheng N (2019) A multi-task convolutional neural network for autonomous robotic grasping in object stacking scenes. In: 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS. IEEE), pp. 6435–6442

  12. Park D, Seo Y, Shin D, Choi J, Chun SY (2020) A single multi-task deep neural network with post-processing for object detection with reasoning and robotic grasp detection. In: 2020 IEEE international conference on robotics and automation (ICRA). IEEE, pp. 7300–7306

  13. Zhang H, Lan X, Bai S, Zhou X, Tian Z, Zheng N (2019) Roi-based robotic grasp detection for object overlapping scenes. In: 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp. 4768–4775

  14. Dong M, Wei S, Yu X, Yin J (2021) Mask-gd segmentation based robotic grasp detection. Comput Commun 178:124–130

    Article  Google Scholar 

  15. Morrison D, Corke P, Leitner J (2018) Closing the loop for robotic grasping: a real-time, generative grasp synthesis approach. arXiv preprint arXiv:1804.05172

  16. Caldera S, Rassau A, Chai D (2018) Review of deep learning methods in robotic grasp detection. Multimodal Technol Interact 2(3):57

    Article  Google Scholar 

  17. Du G, Wang K, Lian S, Zhao K (2021) Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review. Artif Intell Rev 54(3):1677–1734

    Article  Google Scholar 

  18. Bohg J, Morales A, Asfour T, Kragic D (2013) Data-driven grasp synthesis-a survey. IEEE Trans Rob 30(2):289–309

    Article  Google Scholar 

  19. Sahbani A, El-Khoury S, Bidaud P (2012) An overview of 3d object grasp synthesis algorithms. Robot Auton Syst 60(3):326–336

    Article  Google Scholar 

  20. Bicchi A, Kumar V (2000) Robotic grasping and contact: A review. In: Proceedings 2000 ICRA. Millennium Conference. In: IEEE international conference on robotics and automation. Symposia proceedings (Cat. No. 00CH37065), vol. 1. IEEE, pp 348–353

  21. Jiang Y, Moseson S, Saxena A (2011) Efficient grasping from RGBD images: learning using a new rectangle representation. In: 2011 IEEE international conference on robotics and automation. IEEE, pp 3304–3311

  22. Song Y, Gao L, Li X, Shen W (2020) A novel robotic grasp detection method based on region proposal networks. Robot Comput Integr Manuf 65:101963

    Article  Google Scholar 

  23. Zhang H, Zhou X, Lan X, Li J, Tian Z, Zheng N (2019) A real-time robotic grasping approach with oriented anchor box. IEEE Trans Syst Man Cybern Syst 51(5):3014–3025

    Article  Google Scholar 

  24. Jang E, Vijayanarasimhan S, Pastor P, Ibarz J, Levine S (2017) End-to-end learning of semantic grasping. arXiv preprint arXiv:1707.01932

  25. Cai J, Tao X, Cheng H, Zhang Z (2020) CCAN: constraint co-attention network for instance grasping. In: 2020 IEEE international conference on robotics and automation (ICRA). IEEE, pp. 8353–8359

  26. Yun J, Moseson S, Saxena A (2011) Efficient grasping from RGBD images: learning using a new rectangle representation. In: IEEE

  27. Kumra S, Joshi S, Sahin F (2020) Antipodal robotic grasping using generative residual convolutional neural network. In: 2020 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 9626–9633

  28. Wang Y, Zheng Y, Gao B, Huang D (2021) Double-dot network for antipodal grasp detection. In: 2021 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 4654–4661

  29. Yu S, Zhai D-H, Xia Y, Wu H, Liao J (2022) SE-ResUNet: a novel robotic grasp detection method. IEEE Robot Autom Lett 7(2):5238–5245

    Article  Google Scholar 

  30. Dong M, Bai Y, Wei S, Yu X (2022) Robotic grasp detection based on transformer. In: Intelligent robotics and applications: 15th international conference, ICIRA 2022, Harbin, China, August 1–3, 2022, proceedings, part IV. Springer, pp 437–448

Download references

Author information

Authors and Affiliations

  1. School of Modern Post (School of Automation), Beijing University of Posts and Telecommunications, Beijing, 100876, China

    Mingshuai Dong, Yuxuan Bai, Shimin Wei & Xiuli Yu

Authors
  1. Mingshuai Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuxuan Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shimin Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiuli Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiuli Yu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dong, M., Bai, Y., Wei, S. et al. Real-World Semantic Grasp Detection Using Ontology Features: Learning to Concentrate on Object Features. Neural Process Lett 55, 8419–8439 (2023). https://doi.org/10.1007/s11063-023-11318-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-023-11318-w

Keywords

Search

Navigation