Abstract
A significant amount of cargo worldwide is transported in sacks and bags e.g. wheat, rice, coffee and cacao beans, etc. Despite being very strenuous and the health risks involved, the handling of sacks in logistics is predominantly done through manual labor. Hence, the automation of tasks such as cargo unloading from shipping containers is of high importance. However, it faces many challenges due to the unstructured nature of packaging. One of the prerequisites for creating autonomous systems for handling bags or sacks is a robust perception component. In this work, we present a perception pipeline to recognize and localize sacks with a low-cost sensor in unstructured settings with partial views. The backbone of our perception strategy is based on two main contributions presented in this work. First, we introduce a fast convexity test between neighboring patches, which is a part of a two-level segmentation leading to a robust detection of object candidates. Second, we formulate a numerically stable form of superquadric fitting, which allows for an extension of the feasible region of the corresponding optimization problem. Both of the contributions are of interest for applications using superquadrics for representing curved object parts and hence extend beyond the specific scenario of sack/bag recognition and localization presented here. The perception modules introduced in this work are embedded into a newly designed robotic platform capable of manipulating 70 kg sacks - a standard weight when transporting coffee and cocao beans. Moreover, the robot is fully integrated in a coffee storage warehouse. Therefore we substantiate our approach with experiments in a real-world scenario of autonomous unloading of coffee cargo delivered in a shipping container.
Similar content being viewed by others
References
Akbiyik, H., Kirchheim, A., Echelmeyer, W.: Latest trends in the container market : analyse of qualitative and quantitative features of the containerised import in european ports. Bremer Value-Reports fr Produktion und Logistik 3(1), 15 (2009)
Aldoma, A., Tombari, F., Vincze, M.: Supervised Learning of Hidden and Non-Hidden 0-Order Affordances and Detection in Real Scenes Robotics and Automation (ICRA), 2012 IEEE International Conference On, pp. 1732–1739 (2012)
Aldoma, A., Vincze, M., Blodow, N., Gossow, D., Gedikli, S., Rusu, R.B., Bradski, G.: Cad-Model Recognition and 6Dof Pose Estimation Using 3D Cues Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference On, pp. 585–592 (2011)
Anand, A., Koppula, H.S., Joachims, T., Saxena, A.: Contextually guided semantic labeling and search for three-dimensional point clouds. Int. J. Robot. Res. 32(1), 19–34 (2013)
Barr, A.H., transformations, angle-preserving: Superquadrics IEEE Computer graphics and Applications 2(January) (1981)
Barr, A.H.: Global and local deformations of solid primitives. ACM Siggraph Comput. Graph. 18 (3), 21–30 (1984)
Bo, L., Lai, K., Ren, X., Fox, D.: Object Recognition with Hierarchical Kernel Descriptors Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference On, pp. 1729–1736 (2011)
Boult, T.E., Gross, A.D.: Recovery of Superquadrics from Depth Information Proc. Workshop on Spatial Reasoning and … (1987)
Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)
Echelmeyer, W., Kirchheim, A., Robotics-logistics, E. Wellbrock.: Challenges for Automation of Logistic Processes Automation and Logistics, 2008. ICAL 2008. IEEE International Conference On, pp. 2099–2103 (2008)
Jiang, Y., Lim, M., Zheng, C., Saxena, A.: Learning to place new objects in a scene. Int. J. Robot. Res. 31(9), 1021–1043 (2012)
Katsoulas, D.: Reliable Recovery of Piled Box-Like Objects via Parabolically Deformable Superquadrics Computer Vision, 2003. Proceedings. Ninth IEEE …, vol 2, pp. 931–938 (2003)
Katsoulas, D., Bergen, L., Tassakos, L.: A Versatile Depalletizer of Boxes Based on Range Imagery Robotics and Automation, 2002. Proceedings. ICRA ’02. IEEE International Conference On, vol. 4, pp. 4313–4319 Vol.4 (2002)
Katsoulas, D.: Robust recovery of piled box-like objects in range images. PhD thesis (2004)
Kavoussanos, M., Pouliezos, A.: Visionary automation of sack handling and emptying. IEEE Robot. Autom. Mag. 7(4), 44–49 (2000)
Kazerooni, H., Foley, C.: A robotic mechanism for grasping sacks. IEEE Trans. Autom. Sci. Eng. 2(2), 111–120 (2005)
Kirchheim, A., Burwinkel, M., Echelmeyer, W.: Automatic Unloading of Heavy Sacks from Containers Automation and Logistics, 2008. ICAL 2008. IEEE International Conference On, pp. 946–951 (2008)
Kragic, D., Hager, G.D.: Special issue on robotic vision. Int. J. Robot. Res. 31(4), 379–380 (2012)
Krainin, M., Henry, P., Ren, X., Fox, D.: Manipulator and object tracking for in-hand 3d object modeling. Int. J. Robot. Res. 30(11), 1311–1327 (2011)
Lai, K., Bo, L., Ren, X., Fox, D.: A Large-Scale Hierarchical Multi-View Rgb-D Object Dataset Robotics and Automation (ICRA), 2011 IEEE International Conference On, pp. 1817–1824 (2011)
Mihalyi, R.-G., Pathak, K., Vaskevicius, N., Fromm, T., Birk, A.: Robust 3d object modeling with a low-cost rgbd-sensor and ar-markers for applications with untrained end-users. Robot. Auton. Syst. (RAS) 66, 1–17 (2015)
Papazov, C., Haddadin, S., Parusel, S., Krieger, K., Burschka, D.: Rigid 3d geometry matching for grasping of known objects in cluttered scenes. Int. J. Robot. Res. 31(4), 538–553 (2012)
Scholz-Reiter, B., Echelmeyer, W., Wellbrock, E.: Development of a Robot-Based System for Automated Unloading of Variable Packages out of Transport Units and Containers Automation and Logistics, 2008. ICAL 2008. IEEE International Conference On, pp. 2766–2770 (2008)
Sivia, D., Skilling, J.: Data Analysis: A Bayesian Tutorial. Oxford science publications. OUP Oxford (2006)
Solina, F., Bajcsy, R.: Recovery of parametric models from range images: the case for superquadrics with global deformations. IEEE Trans. Pattern Anal. Mach. Intell. 12(2), 131–147 (1990)
Vaskevicius, N., Birk, A., Pathak, K., Schwertfeger, S.: Efficient representation in 3D environment modeling for planetary robotic exploration. Adv. Robot. 24(8-9), 1169–1197 (2010)
Vaskevicius, N., Pathak, K., Birk, A.: Fitting Superquadrics in Noisy, Partial Views from a Low-Cost Rgbd Sensor for Recognition and Localization of Sacks in Autonomous Unloading of Shipping Containers Automation Science and Engineering (CASE), 2014 IEEE International Conference On, pp. 255–262. 2 (2014)
Vaskevicius, N., Pathak, K., Ichim, A.-E., Birk, A.: The Jacobs Robotics Approach to Object Recognition and Localization in the Context of the ICRA’11 Solutions in Perception Challenge IEEE Conf. on Robotics and Automation, St. Paul, MN, USA (2012)
Zhang, Y.: Experimental comparison of superquadric fitting objective functions. Pattern Recogn. Lett. 24(14), 2185–2193 (2003)
Acknowledgments
The research leading to the results presented here has received funding from the European Community’s Seventh Framework Programme (EU FP7 ICT-2) within the project “Cognitive Robot for Automation Logistics Processes (RobLog)”.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Vaskevicius, N., Pathak, K. & Birk, A. Recognition and Localization of Sacks for Autonomous Container Unloading by Fitting Superquadrics in Noisy, Partial Views from a Low-cost RGBD Sensor. J Intell Robot Syst 88, 57–71 (2017). https://doi.org/10.1007/s10846-017-0540-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-017-0540-7
Keywords
- Object recognition
- Superquadric fitting
- Convexity test
- RGB-D segmentation
- Autonomous container unloading