Skip to main content
SpringerLink
Log in

Part-Based Geometric Categorization and Object Reconstruction in Cluttered Table-Top Scenes

Paper Type: Categories (7) and (5)

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

This paper presents an approach for 3D geometry-based object categorization in cluttered table-top scenes. In our method, objects are decomposed into different geometric parts whose spatial arrangement is represented by a graph. The matching and searching of graphs representing the objects is sped up by using a hash table which contains possible spatial configurations of the different parts that constitute the objects. Additive feature descriptors are used to label partially or completely visible object parts. In this work we categorize objects into five geometric shapes: sphere, box, flat, cylindrical, and disk/plate, as these shapes represent the majority of objects found on tables in typical households. Moreover, we reconstruct complete 3D models that include the invisible back-sides of objects as well, in order to facilitate manipulation by domestic service robots. Finally, we present an extensive set of experiments on point clouds of objects using an RGBD camera, and our results highlight the improvements over previous methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aldoma, A., Blodow, N., Gossow, D., Gedikli, S., Rusu, R., Vincze, M., Bradski, G.: CAD-model recognition and 6 DOF pose estimation using 3D cues. In: ICCV Workshop on 3D Representation and Recognition (3dRR11). Barcelona (2011)

  2. Aldoma, A., Marton, Z.C., Tombari, F., Wohlkinger, W., Potthast, C., Zeisl, B., Rusu, R.B., Gedikli, S., Vincze, M.: Tutorial: point cloud library–three-dimensional object recognition and 6 DoF pose estimation. Robot. Autom. Mag. 19(3), 80–91 (2012)

    Article  Google Scholar 

  3. Balint-Benczedi, F.,Marton, Z.C., Beetz, M.: Efficient part-graph hashes for object categorization. In: 5th International Conference on Cognitive Systems (CogSys) (2012)

  4. Bergström, N., Björkman, M., Kragic, D.: Generating object hypotheses in natural scenes through human-robot interaction. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 827–833 (2011)

  5. Bersch, C., Pangercic, D., Osentoski, S., Hausman, K., Marton, Z.C., Ueda, R., Okada, K., Beetz, M.: Segmentation of textured and textureless objects through interactive perception. In: RSS Workshop on Robots in Clutter: Manipulation, Perception and Navigation in Human Environments. Sydney (2012)

  6. Biederman, I.: Recognition-by-components. A theory of human image understanding. Psychol. Rev. (1987)

  7. Blodow, N., Goron, L.C., Marton, Z.C., Pangercic, D., Rühr, T., Tenorth, M., Beetz, M.: Autonomous semantic mapping for robots performing everyday manipulation tasks in kitchen environments. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). San Francisco (2011)

  8. Comaniciu, D., Meer, P., Member, S.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24, 603–619 (2002)

    Article  Google Scholar 

  9. Dickinson, S.: The evolution of object categorization and the challenge of image abstraction. In: Dickinson, S. Leonardis, A., Schiele, B., Tarr, M. (eds) Object Categorization: Computer and Human Vision Perspectives (2009)

  10. Felzenszwalb, P.F., Girshick, R.B., McAllester, D.A., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9) (2010)

  11. Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 59(2), 167–181. doi:10.1023/B:VISI.000002228819776.77

  12. Fergus, R., Perona, P., Zisserman, A.: Object class recognition by unsupervised scale-invariant learning. In: In CVPR, pp. 264–271 (2003)

  13. Fowlkes, C.C., Martin, D.R., Malik, J.: Local figure-ground cues are valid for natural images. J. Vis. 7(8) (2007)

  14. Gomes, R., Krause, A., Perona, P.: Discriminative clustering by regularized information maximization. Adv. Neural Inf. Process. Syst. 23, 1–9 (2010)

    Google Scholar 

  15. Goron, L.C., Marton, Z.C., Lazea, G., Beetz, M.: Segmenting cylindrical and box-like objects in cluttered 3D scenes. In: 7th German Conference on Robotics (ROBOTIK 2012). Munich (2012)

  16. Hausman, K., Balint-Benczedi, F., Pangercic, D., Marton, Z.C., Ueda, R., Okada, K., Beetz, M.: Tracking-based interactive segmentation of textureless objects. In: IEEE International Conference on Robotics and Automation (ICRA). Best Service Robotics Paper Award Finalist. Karlsruhe (2013)

  17. Huber, D., Kapuria, A., Donamukkala, R.R., Hebert, M.: Parts-based 3d object classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 04) (2004)

  18. Jacobs, D.W.: Perceptual organization as generic object recognition. In: From Fragments to Objects - Segmentation and Grouping in Vision, chap. IV. Models Of Segmentation And Grouping, pp. 295–329 (2001)

  19. Kanezaki, A., Marton, Z.C., Pangercic, D., Harada, T., Kuniyoshi, Y., Beetz, M.: Voxelized shape and color histograms for RGB-D. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Workshop on Active Semantic Perception and Object Search in the Real World. San Francisco (2011)

  20. Kanezaki, A., Nakayama, H., Harada, T., Kuniyoshi, Y.: High-speed 3d object recognition using additive features in a linear subspace. In: Proceedings of International Conference on Robotics and Automation (ICRA), pp. 3128–3134 (2010)

  21. Kanezaki, A., Suzuki, T., Harada, T., Kuniyoshi, Y.: Fast object detection for robots in a cluttered indoor environment using integral 3D feature table. In: Proceedings of the IEEE ICRA (2011)

  22. Lai, K., Bo, L., Ren, X., Fox, D.: A large-scale hierarchical multi-view rgb-d object dataset. In: Proceedings of International Conference on Robotics and Automation (ICRA) (2011)

  23. Lai, K., Bo, L., Ren, X., Fox, D.: Sparse distance learning for object recognition combining rgb and depth information. In: Proceedings of International Conference on Robotics and Automation (ICRA) (2011)

  24. Lai, K., Fox, D.: Object recognition in 3d point clouds using web data and domain adaptation. Int. J. Robot. Res. 29(8), 1019–1037 (2010). doi:10.1177/0278364910369190

    Article  Google Scholar 

  25. Lam, L., Suen, C.Y.: Optimal combinations of pattern classifiers. Pattern Recognit. Lett 16(9), 945–954 (1995)

    Article  Google Scholar 

  26. Leibe, B., Leonardis, A., Schiele, B.: Robust object detection with interleaved categorization and segmentation. Int. J. Comput. Vis. 77(1–3), 259–289 (2008)

    Article  Google Scholar 

  27. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)

    Article  Google Scholar 

  28. Malisiewicz, T., Efros, A.A.: Improving spatial support for objects via multiple segmentations. In: Proceedings of the British Machine Vision Conference (2007)

  29. Marton, Z.C., Balint-Benczedi, F., Blodow, N., Goron, L.C., Beetz, M.: Object categorization in clutter using additive features and hashing of part-graph descriptors. In: Proceedings of Spatial Cognition 2012. Abbey Kloster Seeon (2012)

  30. Marton, Z.C., Balint-Benczedi, F., Mozos, O.M., Pangercic, D., Beetz, M.: Cumulative object categorization in clutter. In: 2nd Workshop on Robotics in Clutter, at Robotics: Science and Systems (RSS) (2013)

  31. Marton, Z.C., Pangercic, D., Blodow, N., Beetz, M.: Combined 2D-3D categorization and classification for multimodal perception systems. Int. J. Robot. Res. (2011)

  32. Marton, Z.C., Pangercic, D., Rusu, R.B., Holzbach, A., Beetz, M.: Hierarchical object geometric categorization and appearance classification for mobile manipulation. In: Proceedings of 2010, IEEE-RAS International Conference on Humanoid Robots. Nashville (2010)

  33. Marton, Z.C., Rusu, R.B., Jain, D., Klank, U., Beetz, M.: Probabilistic categorization of kitchen objects in table settings with a composite sensor. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. USA (2009)

  34. Marton, Z.C., Seidel, F., Balint-Benczedi, F., Beetz, M.: Ensembles of strong learners for multi-cue classification. Pattern Recognit. Lett. Special Issue on Scene Understandings and Behaviours Analysis (2012)

  35. Mishra, A.K., Aloimonos, Y.: Visual segmentation of “Simple” objects for robots. In: Robotics: Science and Systems (RSS) (2011)

  36. Mozos, O.M., Marton, Z.C., Beetz, M.: Furniture models learned from the www—using web catalogs to locate and categorize unknown furniture pieces in 3D laser scans. Robot. Autom. Mag. 18(2), 22–32 (2011)

    Article  Google Scholar 

  37. Mueller, C.A., Pathak, K., Birk, A.: Object recognition in rgbd images of cluttered environments using graph-based categorization with unsupervised learning of shape parts. In: International Conference on Intelligent Robots and Systems (IROS) (2013)

  38. Pangercic, D., Haltakov, V., Beetz, M.: Fast and robust object detection in household environments using vocabulary trees with sift descriptors. Workshop on Active Semantic Perception and Object Search in the Real World. San Francisco (2011)

  39. Richtsfeld, A., Morwald, T., Prankl, J., Zillich, M., Vincze, M.: Segmentation of unknown objects in indoor environments. In: Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference, pp. 4791–4796. doi:10.1109/IROS.2012.6385661 (2012)

  40. Ruiz-Correa, S., Shapiro, L.G., Meila, M.: A new paradigm for recognizing 3-D object shapes from range data. In: International Conference on Computer Vision (2003)

  41. Rusu, R.B., Blodow, N., Marton, Z.C., Beetz, M.: Close-range scene segmentation and reconstruction of 3D point cloud maps for mobile manipulation in human environments. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). St. Louis (2009)

  42. Rusu, R.B., Cousins, S.: 3D is here: point cloud library (PCL). In: IEEE International Conference on Robotics and Automation (ICRA). Shanghai (2011)

  43. Rusu, R.B., Marton, Z.C., Blodow, N., Dolha, M., Beetz, M.: Towards 3D Point Cloud Based Object Maps for Household Environments. Robot. Auton. Syst. J. (Special Issue on Semantic Knowledge in Robotics) 56(11), 927–941 (2008)

    Article  Google Scholar 

  44. Scavino, E., Wahab, D.A., Basri, H., Mustafa, M.M., Hussain, A.: A Genetic Algorithm for the Segmentation of Known Touching Objects 5, 711–716 (2009)

    Google Scholar 

  45. Schnabel, R., Wahl, R., Klein, R.: Efficient ransac for point-cloud shape detection. In: Computer Graphics Forum, vol. 26, pp. 214–226. Wiley Online Library (2007)

  46. Schuster, M., Okerman, J., Nguyen, H., Rehg, J., Kemp, C.: Perceiving clutter and surfaces for object placement in indoor environments. In: Proceedings of the 10th IEEE-RAS International Conference on Humanoid Robots (Humanoids), pp. 152–159 (2010) doi:10.1109/ICHR.2010.5686328

  47. Shotton, J., Blake, A., Cipolla, R.: Multiscale categorical object recognition using contour fragments. IEEE Trans. Pattern Anal. Mach. Intell. 30(7) (2008)

  48. Shotton, J., Winn, J., Rother, C., Criminisi, A.: Textonboost for image understanding: multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Int. J. Comput. Vis. (2007)

  49. Singh, M., Hoffman, D.D.: Part-Based representations of visual shape and implications for visual cognition. In: From Fragments to Objects–Segmentation and Grouping in Vision, chap. IV. Models Of Segmentation And Grouping, pp. 401–459 (2001)

  50. Socher, R., Lin, C.C.Y., Ng, A.Y., Manning, C.D.: Parsing natural scenes and natural language with recursive neural networks. In: 28th International Conference on Machine Learning, pp. 129–136 (2011)

  51. Somanath, G., Rohith, M., Metaxas, D., Kambhamettu, C.: D-clutter: building object model library from unsupervised segmentation of cluttered scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2783–2789 (2009) doi:10.1109/CVPR.2009.5206579

  52. Sun, M., Bradski, G., Xu, B.X., Savarese, S.: Depth-encoded hough voting for joint object detection and shape recovery. In: Proceedings of the 11th European conference on Computer vision: part V, ECCV’10 pp. 658–671. Springer, Berlin, Heidelberg (2010). http://dl.acm.org/citation.cfm?id=1888150.1888201

  53. Taylor, G., Kleeman, L.: Chapter 4: 3D object modelling and classification. In: Visual Perception and Robotic Manipulation–3D Object Recognition, Tracking and Hand-Eye Coordination, Springer Tracts in Advanced Robotics, vol. 26, pp.57–83. Springer, Berlin, Heidelberg (2006)

  54. Tombari, F., Di Stefano, L.: Object recognition in 3d scenes with occlusions and clutter by hough voting. In: Proceedings of the Fourth Pacific-Rim Symposium on Image and Video Technology (PSIVT), pp. 349–355 (2010) doi:10.1109/PSIVT.2010.65

  55. Torres, M.M., Romea, A.C., Srinivasa, S.: MOPED: a scalable and low latency object recognition and pose estimation system. In: Proceedings of International Conference on Robotics and Automation (ICRA) (2010)

  56. Triebel, R., Shin, J., Siegwart, R.: Segmentation and unsupervised part-based discovery of repetitive objects. In: Proceedings of Robotics: Science and Systems. Zaragoza (2010)

  57. Vernon, D.: Cognitive vision: the case for embodied perception. In: Image and Vision Computing. Elsevier (2005)

  58. Watanabe, S., Pakvasa, N.: Subspace method in pattern recognition. In: Proceedings of 1st International Joint Conference on Pattern Recognition (1973)

  59. Yildirim, I., Jacobs, R.A.: Transfer of object category knowledge across visual and haptic modalities: experimental and computational studies. Cognition 126(2), 135–148 (2013). doi:10.1016/j.cognition.2012.08.005

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Robotics and Mechatronics, German Aerospace Center (DLR), 51147, Oberpfaffenhofen, Germany

    Zoltan-Csaba Marton

  2. Institute of Artificial Intelligence, Universität Bremen, Center for Computing Technologies (TZI), Bremen, Germany

    Ferenc Balint-Benczedi & Michael Beetz

  3. School of Computer Science, University of Lincoln, Lincoln, UK

    Oscar Martinez Mozos

  4. Intelligent Autonomous Systems, Technische Universität München, München, Germany

    Nico Blodow & Lucian Cosmin Goron

  5. Machine Intelligence Lab, Deptartment of Mechano-Informatics, Graduate School of Information Science & Technology, The University of Tokyo, Tokyo, Japan

    Asako Kanezaki

  6. Autonomous Technologies Group Robert Bosch LLC, Palo Alto, CA, 94304, USA

    Dejan Pangercic

Authors
  1. Zoltan-Csaba Marton
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ferenc Balint-Benczedi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Oscar Martinez Mozos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nico Blodow
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Asako Kanezaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lucian Cosmin Goron
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Dejan Pangercic
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Michael Beetz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zoltan-Csaba Marton.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Marton, ZC., Balint-Benczedi, F., Mozos, O.M. et al. Part-Based Geometric Categorization and Object Reconstruction in Cluttered Table-Top Scenes. J Intell Robot Syst 76, 35–56 (2014). https://doi.org/10.1007/s10846-013-0011-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-013-0011-8

Keywords

Mathematics Subject Classifications (2010)

Search

Navigation