Skip to main content
Log in

Recognition and Localization of Sacks for Autonomous Container Unloading by Fitting Superquadrics in Noisy, Partial Views from a Low-cost RGBD Sensor

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

A significant amount of cargo worldwide is transported in sacks and bags e.g. wheat, rice, coffee and cacao beans, etc. Despite being very strenuous and the health risks involved, the handling of sacks in logistics is predominantly done through manual labor. Hence, the automation of tasks such as cargo unloading from shipping containers is of high importance. However, it faces many challenges due to the unstructured nature of packaging. One of the prerequisites for creating autonomous systems for handling bags or sacks is a robust perception component. In this work, we present a perception pipeline to recognize and localize sacks with a low-cost sensor in unstructured settings with partial views. The backbone of our perception strategy is based on two main contributions presented in this work. First, we introduce a fast convexity test between neighboring patches, which is a part of a two-level segmentation leading to a robust detection of object candidates. Second, we formulate a numerically stable form of superquadric fitting, which allows for an extension of the feasible region of the corresponding optimization problem. Both of the contributions are of interest for applications using superquadrics for representing curved object parts and hence extend beyond the specific scenario of sack/bag recognition and localization presented here. The perception modules introduced in this work are embedded into a newly designed robotic platform capable of manipulating 70 kg sacks - a standard weight when transporting coffee and cocao beans. Moreover, the robot is fully integrated in a coffee storage warehouse. Therefore we substantiate our approach with experiments in a real-world scenario of autonomous unloading of coffee cargo delivered in a shipping container.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Akbiyik, H., Kirchheim, A., Echelmeyer, W.: Latest trends in the container market : analyse of qualitative and quantitative features of the containerised import in european ports. Bremer Value-Reports fr Produktion und Logistik 3(1), 15 (2009)

    Google Scholar 

  2. Aldoma, A., Tombari, F., Vincze, M.: Supervised Learning of Hidden and Non-Hidden 0-Order Affordances and Detection in Real Scenes Robotics and Automation (ICRA), 2012 IEEE International Conference On, pp. 1732–1739 (2012)

  3. Aldoma, A., Vincze, M., Blodow, N., Gossow, D., Gedikli, S., Rusu, R.B., Bradski, G.: Cad-Model Recognition and 6Dof Pose Estimation Using 3D Cues Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference On, pp. 585–592 (2011)

  4. Anand, A., Koppula, H.S., Joachims, T., Saxena, A.: Contextually guided semantic labeling and search for three-dimensional point clouds. Int. J. Robot. Res. 32(1), 19–34 (2013)

    Article  Google Scholar 

  5. Barr, A.H., transformations, angle-preserving: Superquadrics IEEE Computer graphics and Applications 2(January) (1981)

  6. Barr, A.H.: Global and local deformations of solid primitives. ACM Siggraph Comput. Graph. 18 (3), 21–30 (1984)

    Article  Google Scholar 

  7. Bo, L., Lai, K., Ren, X., Fox, D.: Object Recognition with Hierarchical Kernel Descriptors Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference On, pp. 1729–1736 (2011)

  8. Boult, T.E., Gross, A.D.: Recovery of Superquadrics from Depth Information Proc. Workshop on Spatial Reasoning and … (1987)

  9. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)

    Article  Google Scholar 

  10. Echelmeyer, W., Kirchheim, A., Robotics-logistics, E. Wellbrock.: Challenges for Automation of Logistic Processes Automation and Logistics, 2008. ICAL 2008. IEEE International Conference On, pp. 2099–2103 (2008)

  11. Jiang, Y., Lim, M., Zheng, C., Saxena, A.: Learning to place new objects in a scene. Int. J. Robot. Res. 31(9), 1021–1043 (2012)

    Article  Google Scholar 

  12. Katsoulas, D.: Reliable Recovery of Piled Box-Like Objects via Parabolically Deformable Superquadrics Computer Vision, 2003. Proceedings. Ninth IEEE …, vol 2, pp. 931–938 (2003)

  13. Katsoulas, D., Bergen, L., Tassakos, L.: A Versatile Depalletizer of Boxes Based on Range Imagery Robotics and Automation, 2002. Proceedings. ICRA ’02. IEEE International Conference On, vol. 4, pp. 4313–4319 Vol.4 (2002)

  14. Katsoulas, D.: Robust recovery of piled box-like objects in range images. PhD thesis (2004)

  15. Kavoussanos, M., Pouliezos, A.: Visionary automation of sack handling and emptying. IEEE Robot. Autom. Mag. 7(4), 44–49 (2000)

    Article  Google Scholar 

  16. Kazerooni, H., Foley, C.: A robotic mechanism for grasping sacks. IEEE Trans. Autom. Sci. Eng. 2(2), 111–120 (2005)

    Article  Google Scholar 

  17. Kirchheim, A., Burwinkel, M., Echelmeyer, W.: Automatic Unloading of Heavy Sacks from Containers Automation and Logistics, 2008. ICAL 2008. IEEE International Conference On, pp. 946–951 (2008)

  18. Kragic, D., Hager, G.D.: Special issue on robotic vision. Int. J. Robot. Res. 31(4), 379–380 (2012)

    Article  Google Scholar 

  19. Krainin, M., Henry, P., Ren, X., Fox, D.: Manipulator and object tracking for in-hand 3d object modeling. Int. J. Robot. Res. 30(11), 1311–1327 (2011)

    Article  Google Scholar 

  20. Lai, K., Bo, L., Ren, X., Fox, D.: A Large-Scale Hierarchical Multi-View Rgb-D Object Dataset Robotics and Automation (ICRA), 2011 IEEE International Conference On, pp. 1817–1824 (2011)

  21. Mihalyi, R.-G., Pathak, K., Vaskevicius, N., Fromm, T., Birk, A.: Robust 3d object modeling with a low-cost rgbd-sensor and ar-markers for applications with untrained end-users. Robot. Auton. Syst. (RAS) 66, 1–17 (2015)

    Article  Google Scholar 

  22. Papazov, C., Haddadin, S., Parusel, S., Krieger, K., Burschka, D.: Rigid 3d geometry matching for grasping of known objects in cluttered scenes. Int. J. Robot. Res. 31(4), 538–553 (2012)

    Article  Google Scholar 

  23. Scholz-Reiter, B., Echelmeyer, W., Wellbrock, E.: Development of a Robot-Based System for Automated Unloading of Variable Packages out of Transport Units and Containers Automation and Logistics, 2008. ICAL 2008. IEEE International Conference On, pp. 2766–2770 (2008)

  24. Sivia, D., Skilling, J.: Data Analysis: A Bayesian Tutorial. Oxford science publications. OUP Oxford (2006)

  25. Solina, F., Bajcsy, R.: Recovery of parametric models from range images: the case for superquadrics with global deformations. IEEE Trans. Pattern Anal. Mach. Intell. 12(2), 131–147 (1990)

    Article  Google Scholar 

  26. Vaskevicius, N., Birk, A., Pathak, K., Schwertfeger, S.: Efficient representation in 3D environment modeling for planetary robotic exploration. Adv. Robot. 24(8-9), 1169–1197 (2010)

    Article  Google Scholar 

  27. Vaskevicius, N., Pathak, K., Birk, A.: Fitting Superquadrics in Noisy, Partial Views from a Low-Cost Rgbd Sensor for Recognition and Localization of Sacks in Autonomous Unloading of Shipping Containers Automation Science and Engineering (CASE), 2014 IEEE International Conference On, pp. 255–262. 2 (2014)

  28. Vaskevicius, N., Pathak, K., Ichim, A.-E., Birk, A.: The Jacobs Robotics Approach to Object Recognition and Localization in the Context of the ICRA’11 Solutions in Perception Challenge IEEE Conf. on Robotics and Automation, St. Paul, MN, USA (2012)

  29. Zhang, Y.: Experimental comparison of superquadric fitting objective functions. Pattern Recogn. Lett. 24(14), 2185–2193 (2003)

    Article  MATH  Google Scholar 

Download references

Acknowledgments

The research leading to the results presented here has received funding from the European Community’s Seventh Framework Programme (EU FP7 ICT-2) within the project “Cognitive Robot for Automation Logistics Processes (RobLog)”.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Narunas Vaskevicius.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(MP4 83.7 MB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vaskevicius, N., Pathak, K. & Birk, A. Recognition and Localization of Sacks for Autonomous Container Unloading by Fitting Superquadrics in Noisy, Partial Views from a Low-cost RGBD Sensor. J Intell Robot Syst 88, 57–71 (2017). https://doi.org/10.1007/s10846-017-0540-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-017-0540-7

Keywords

Mathematics Subject Classification (2010)

Navigation