Skip to main content
SpringerLink
Log in

6DoF Pose Estimation for Industrial Manipulation Based on Synthetic Data

  • Conference paper
  • First Online:
Proceedings of the 2018 International Symposium on Experimental Robotics (ISER 2018)

Part of the book series: Springer Proceedings in Advanced Robotics ((SPAR,volume 11))

Included in the following conference series:

Abstract

We present a perception system for mobile manipulation tasks. The primary design goal of the proposed system is to minimize human interaction during system setup which is achieved by several means, such as automatic training data generation, the use of simulated training data, and 3D model based geometric matching. We employ a state-of-the art deep-learning based bounding box detector for rough localization of objects and a Point Pair Feature based matching algorithm for 6DoF pose estimation. The proposed approach shows promising results on our recently published dataset for industrial object detection and pose estimation. Furthermore, the system’s performance during four days of live operation at the Automatica 2018 trade fair is analyzed and failure cases are presented and discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Balint-Benczedi, F., Beetz, M.: Variations on a theme: ‘it is a strange kind of memory that only works backwards’. In: International Conference on Intelligent Robots (IROS). IEEE (2018, under review)

    Google Scholar 

  2. Balint-Benczedi, F., Marton, Z.-C., Durner, M., Beetz, M.: Storing and retrieving perceptual episodic memories for long-term manipulation tasks. In: 18th International Conference on Advanced Robotics (ICAR). IEEE (2017). Best Paper Finalist

    Google Scholar 

  3. Beetz, M., Balint-Benczedi, F., Blodow, N., Nyga, D., Wiedemeyer, T., Marton, Z.-C.: RoboSherlock: unstructured information processing for robot perception. In: IEEE International Conference on Robotics and Automation (ICRA), Seattle, Washington, USA (2015). Best Service Robotics Paper Award

    Google Scholar 

  4. Brachmann, E., Michel, F., Krull, A., Ying Yang, M., Gumhold, S., Rother, C.: Uncertainty-driven 6D pose estimation of objects and scenes from a single RGB image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3364–3372 (2016)

    Google Scholar 

  5. Csurka, G.: Domain adaptation for visual applications: a comprehensive survey. arXiv preprint arXiv:1702.05374 (2017)

  6. Dai, J., Li, Y., He, K., Sun, J.: R-FCN: object detection via region-based fully convolutional networks. In: Advances in Neural Information Processing Systems, pp. 379–387 (2016)

    Google Scholar 

  7. Dömel, A., Kriegel, S., Brucker, M., Suppa, M.: Autonomous pick and place operations in industrial production. In: 12th International Conference on Ubiquitous Robots and Ambient Intelligence, p. 356. IEEE (2015). Best Video Paper Award

    Google Scholar 

  8. Dömel, A., Kriegel, S., Kaßecker, M., Brucker, M., Bodenmüller, T., Suppa, M.: Towards fully autonomous mobile manipulation for industrial environments. Int. J. Adv. Robot. Syst. (2017)

    Google Scholar 

  9. Drost, B., Ulrich, M., Navab, N., Ilic, S.: Model globally, match locally: efficient and robust 3D object recognition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 998–1005, June 2010

    Google Scholar 

  10. Durner, M., Kriegel, S., Riedel, S., Brucker, M., Márton, Z.-C., Bálint-Benczédi, F., Triebel, R.: Experience-based optimization of robotic perception. In: 18th International Conference on Advanced Robotics (ICAR), pp. 32–39. IEEE (2017). Best Paper Finalist

    Google Scholar 

  11. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL visual object classes challenge 2012 (VOC2012) Results (2012). http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html

  12. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  13. Hinterstoisser, S., Lepetit, V., Rajkumar, N., Konolige, K.: Going further with point pair features. In: European Conference on Computer Vision (2016)

    Google Scholar 

  14. Hinterstoisser, S., Lepetit, V., Wohlhart, P., Konolige, K.: On pre-trained image features and synthetic images for deep learning. arXiv:1710.10710 (2017)

  15. Hodaň, T., Matas, J., Obdržálek, Š.: On evaluation of 6D object pose estimation. In: Hua, G., Jégou, H. (eds.) European Conference on Computer Vision, vol. 9915, pp. 606–619. Springer, Cham (2016)

    Google Scholar 

  16. Kehl, W., Manhardt, F., Tombari, F., Ilic, S., Navab, N.: SSD-6D: making RGB-based 3D detection and 6D pose estimation great again. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1521–1529 (2017)

    Google Scholar 

  17. Kriegel, S., Brucker, M., Marton, Z.C., Bodenmüller, T., Suppa, M.: Combining object modeling and recognition for active scene exploration. In: International Conference on Intelligent Robots and Systems, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2384–2391, November 2013

    Google Scholar 

  18. Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. arXiv preprint arXiv:1708.02002 (2017)

  19. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., Berg, A.C.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) European Conference on Computer Vision, vol. 9905, pp. 21–37. Springer, Cham (2016)

    Google Scholar 

  20. Mitash, C., Bekris, K.E., Boularias, A.: A self-supervised learning system for object detection using physics simulation and multi-view pose estimation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 545–551. IEEE (2017)

    Google Scholar 

  21. Movshovitz-Attias, Y., Kanade, T., Sheikh, Y.: How useful is photo-realistic rendering for visual learning? In: Hua, G., Jégou, H. (eds.) European Conference on Computer Vision, pp. 202–217. Springer, Cham (2016)

    Google Scholar 

  22. Rad, M., Lepetit, V.: BB8: a scalable, accurate, robust to partial occlusion method for predicting the 3D poses of challenging objects without using depth. arXiv preprint arXiv:1703.10896 (2017)

  23. Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. arXiv preprint arXiv:1612.08242 (2016)

  24. Redmon, J., Farhadi, A.: YOLOv3: An incremental improvement. arXiv (2018)

    Google Scholar 

  25. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)

    Google Scholar 

  26. Schmidt, T., Hertkorn, K., Newcombe, R., Marton, Z., Fox, D.: Depth-based tracking with physical constraints for robot manipulation. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 119–126, May 2015. Best Paper Award finalist

    Google Scholar 

  27. Su, H., Qi, C.R., Li, Y., Guibas, L.J.: Render for CNN: viewpoint estimation in images using CNNS trained with rendered 3D model views. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2686–2694 (2015)

    Google Scholar 

  28. Sundermeyer, M., Marton, Z.-C., Durner, M., Brucker, M., Triebel, R.: Implicit 3D orientation learning for 6D object detection from RGB images. In: The European Conference on Computer Vision (ECCV), September 2018

    Chapter  Google Scholar 

  29. Tekin, B., Sinha, S.N., Fua, P.: Real-time seamless single shot 6D object pose prediction. arXiv preprint arXiv:1711.08848 (2017)

  30. Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., Abbeel, P.: Domain randomization for transferring deep neural networks from simulation to the real world. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2017)

    Google Scholar 

  31. Vidal, J., Lin, C.-Y., Martí, R.: 6D pose estimation using an improved method based on point pair features. arXiv preprint arXiv:1802.08516 (2018)

  32. Wahl, E., Hillenbrand, U., Hirzinger, G.: Surflet-pair-relation histograms: a statistical 3D-shape representation for rapid classification. In: Fourth International Conference on 3-D Digital Imaging and Modeling (3DIM), pp. 474–481. IEEE (2003)

    Google Scholar 

  33. Weichao, Q., Fangwei, Z., Yi, Z., Siyuan, Q., Xiao, Z., Kim, T.S., Wang, Y., Yuille, A.: UnrealCV: virtual worlds for computer vision. In: ACM Multimedia Open Source Software Competition (2017)

    Google Scholar 

Download references

Acknowledgements

We thank the entire team of the mobile manipulation Automatica demo at DLR, especially Andreas Dömel for coordination, Peter Lehner for the robotic manipulation, and Sebastian Riedel for the logging component. We also thank Amrutha Saseendran for her help with the detector, as well as the teams of the other three Factory of the Future demos.

Author information

Authors and Affiliations

  1. German Aerospace Center (DLR), Institute of Robotics and Mechatronics, Münchner Str. 20, Oberpfaffenhofen, 82234, Weßling, Germany

    Manuel Brucker, Maximilian Durner, Zoltán-Csaba Márton, Martin Sundermeyer & Rudolph Triebel

  2. Institute for Artificial Intelligence, University of Bremen, Bremen, Germany

    Ferenc Bálint-Benczédi

Authors
  1. Manuel Brucker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maximilian Durner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zoltán-Csaba Márton
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ferenc Bálint-Benczédi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Martin Sundermeyer
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Rudolph Triebel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zoltán-Csaba Márton .

Editor information

Editors and Affiliations

  1. Robotics Engineering, Worcester Polytechnic Institute, Worcester, MA, USA

    Jing Xiao

  2. Karlsruhe Institute of Technology, Karlsruhe, Baden-Württemberg, Germany

    Torsten Kröger

  3. Department of Computer Science, Stanford University, Stanford, CA, USA

    Oussama Khatib

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Brucker, M., Durner, M., Márton, ZC., Bálint-Benczédi, F., Sundermeyer, M., Triebel, R. (2020). 6DoF Pose Estimation for Industrial Manipulation Based on Synthetic Data. In: Xiao, J., Kröger, T., Khatib, O. (eds) Proceedings of the 2018 International Symposium on Experimental Robotics. ISER 2018. Springer Proceedings in Advanced Robotics, vol 11. Springer, Cham. https://doi.org/10.1007/978-3-030-33950-0_58

Download citation

Publish with us

Policies and ethics

Search

Navigation