Skip to main content
SpringerLink
Log in

3D object recognition from cluttered and occluded scenes with a compact local feature

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

This paper presents a novel local surface descriptor for 3D object recognition in the presence of clutter and occlusion. For a keypoint and its local neighbor points, a unique and repeatable local reference frame is first constructed. Then the depth information of those local points is encoded into a vector using multi-view method. Finally we obtain a compact feature with length of 27. Besides its compactness in length, the generated feature is tested to be not only descriptive on various datasets but also robust against Gaussian noise, short noise and varying mesh resolutions. Based on this descriptor, we propose a hierarchical algorithm for automatic object recognition. By feature matching, scene-point-to-model-point correspondences are established after potential planar clutters are removed from the scene. These correspondences are then utilized for object identification and pose estimation. Experimental results on three popular datasets demonstrate the effectiveness of our proposed technique which achieves the best results compared to the state of the art with all targets correctly recognized and their poses estimated.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Aldoma, A., Tombari, F., Di Stefano, L., Vincze, M.: A global hypotheses verification method for 3d object recognition. In: Computer Vision—ECCV 2012: 12th European Conference on Computer Vision. Florence, Italy, 7–13 October 2012, Proceedings, Part III, pp. 511–524. Springer, Berlin (2012)

  2. Bariya, P., Novatnack, J., Schwartz, G., Nishino, K.: 3D geometric scale variability in range images: features and descriptors. Int. J. Comput. Vis. 99(2), 232–255 (2012)

    Article  MathSciNet  Google Scholar 

  3. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput. Vis. Image Underst. 110(3), 346–359 (2008)

    Article  Google Scholar 

  4. Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18(9), 509–517 (1975). https://doi.org/10.1145/361002.361007

    Article  MathSciNet  MATH  Google Scholar 

  5. Besl, P.J., Mckay, H.D.: A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)

    Article  Google Scholar 

  6. Buch, A.G., Petersen, H.G., Krüger, N.: Local shape feature fusion for improved matching, pose estimation and 3d object recognition. SpringerPlus 5(1), 1–33 (2016)

    Article  Google Scholar 

  7. Chen, H.: 3D free-form object recognition in range images using local surface patches. Pattern Recognit. Lett. 3, 136 (2007)

    Google Scholar 

  8. Chen, X., Li, J., Shi, Z., Yang, W.: Distinctive local surface descriptor for three-dimensional objects based on bispectrum of spherical harmonics. J. Electron. Imaging 25(1), 013,021–013,021 (2016)

    Article  Google Scholar 

  9. Curless, B., Levoy, M.: A volumetric method for building complex models from range images. In: International Conference on Computer Graphics and Interactive Techniques, pp. 303–312 (1996)

  10. Flint, A., Dick, A., van den Hengel, A.: Thrift: local 3d structure recognition. In: 9th Biennial Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications (DICTA 2007), pp. 182–188 (2007)

  11. Frome, A., Huber, D., Kolluri, R., Bülow, T., Malik, J.: Recognizing objects in range data using regional point descriptors. In: Computer Vision—ECCV 2004: 8th European Conference on Computer Vision, Prague, Czech Republic, 11–14 May 2004. Proceedings, Part III, pp. 224–237. Springer, Berlin (2004)

  12. Gao, Y., Zhang, N.: 3D model comparison using spatial structure circular descriptor. Pattern Recognit. 43(3), 1142–1151 (2010)

    Article  MATH  Google Scholar 

  13. Guennebaud, G., Germann, M., Gross, M.: Dynamic sampling and rendering of algebraic point set surfaces. Comput. Graph. Forum 27(2), 653–662 (2010)

    Article  Google Scholar 

  14. Guennebaud, G., Gross, M.: Algebraic point set surfaces. ACM Trans. Graph. 26(3), 1–9 (2007)

    Article  Google Scholar 

  15. Guo, Y., Bennamoun, M., Sohel, F., Lu, M., Wan, J.: 3D object recognition in cluttered scenes with local surface features: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 36(11), 2270–2287 (2014)

    Article  Google Scholar 

  16. Guo, Y., Bennamoun, M., Sohel, F., Lu, M., Wan, J.: An integrated framework for 3-d modeling, object detection, and pose estimation from point-clouds. IEEE Trans. Instrum. Meas. 64(3), 683–693 (2015)

    Article  Google Scholar 

  17. Guo, Y., Bennamoun, M., Sohel, F., Wan, J., Kwok, N.M.: A comprehensive performance evaluation of 3d local feature descriptors. Int. J. Comput. Vis. 116(1), 66–89 (2015)

    Article  MathSciNet  Google Scholar 

  18. Guo, Y., Sohel, F., Bennamoun, M., Lu, M., Wan, J.: Rotational projection statistics for 3d local surface description and object recognition. Int. J. Comput. Vis. 105(1), 63–86 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  19. Guo, Y., Sohel, F., Bennamoun, M., Wan, J.: A novel local surface feature for 3d object recognition under clutter and occlusion. Inf. Sci. 293, 196–213 (2015)

    Article  Google Scholar 

  20. Huang, C.H., Tombari, F., Navab, N.: Repeatable local coordinate frames for 3d human motion tracking: from rigid to non-rigid. In: 2015 International Conference on 3D Vision, pp. 371–379 (2015)

  21. Iyer, N., Jayanti, S., Lou, K., Kalyanaraman, Y., Ramani, K.: Three-dimensional shape searching: state-of-the-art review and future trends. Comput. Aided Des. 37(5), 509–530 (2005)

    Article  Google Scholar 

  22. Johnson, A.E., Hebert, M.: Using spin images for efficient object recognition in cluttered 3d scenes. IEEE Trans. Pattern Anal. Mach. Intell. 21(5), 433–449 (1999)

    Article  Google Scholar 

  23. Kasaei, S.H., Tomé, A.M., Lopes, L.S., Oliveira, M.: Good: a global orthographic object descriptor for 3d object recognition and manipulation. Pattern Recognit. Lett. 83, 312–320 (2016)

    Article  Google Scholar 

  24. Lei, Y., Guo, Y., Hayat, M., Bennamoun, M., Zhou, X.: A two-phase weighted collaborative representation for 3d partial face recognition with single sample. Pattern Recognit. 52, 218–237 (2016)

    Article  Google Scholar 

  25. Liu, C., Hu, W.: Relative pose estimation for cylinder-shaped spacecrafts using single image. IEEE Trans. Aerosp. Electron. Syst. 50(4), 3036–3056 (2014)

    Article  Google Scholar 

  26. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  27. Lu, T., Hu, W., Liu, C., Yang, D.: Relative pose estimation of a lander using crater detection and matching. Optic. Eng. 55(2), 23102–23102 (2016)

    Article  Google Scholar 

  28. Malassiotis, S., Strintzis, M.G.: Snapshots: a novel local surface descriptor and matching algorithm for robust 3d surface alignment. IEEE Trans. Pattern Anal. Mach. Intell. 29(7), 1285–1290 (2007)

    Article  Google Scholar 

  29. Mian, A., Bennamoun, M.: Keypoint detection and local feature matching for textured 3d face recognition. Int. J. Comput. Vis. 79(1), 1–12 (2008)

    Article  Google Scholar 

  30. Mian, A., Bennamoun, M., Owens, R.: On the repeatability and quality of keypoints for local feature-based 3d object retrieval from cluttered scenes. Int. J. Comput. Vis. 89(2), 348–361 (2010)

    Article  Google Scholar 

  31. Mian, A.S., Bennamoun, M., Owens, R.: Three-dimensional model-based object recognition and segmentation in cluttered scenes. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1584–1601 (2006)

    Article  Google Scholar 

  32. Mian, A.S., Bennamoun, M., Owens, R.A.: A novel representation and feature matching algorithm for automatic pairwise registration of range images. Int. J. Comput. Vis. 66(1), 19–40 (2006)

    Article  Google Scholar 

  33. Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005)

    Article  Google Scholar 

  34. Pang, G., Neumann, U.: Fast and robust multi-view 3d object recognition in point clouds. In: 2015 International Conference on 3D Vision (3DV), pp. 171–179 (2015)

  35. Petrelli, A., Stefano, L.D.: On the repeatability of the local reference frame for partial shape matching. In: International Conference on Computer Vision, pp. 2244–2251 (2011)

  36. Petrelli, A., Stefano, L.D.: A repeatable and efficient canonical reference for surface matching. In: 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization Transmission, pp. 403–410 (2012)

  37. Quan, S., Ma, J., Ma, T., Hu, F., Fang, B.: Representing local shape geometry from multi-view silhouette perspective: a distinctive and robust binary 3d feature. Signal Process. Image Commun. 65, 67–80 (2018). https://doi.org/10.1016/j.image.2018.03.015

    Article  Google Scholar 

  38. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (2016)

  39. Rusu, R.B., Blodow, N., Beetz, M.: Fast point feature histograms (fpfh) for 3d registration. In: International Conference on Robotics and Automation, pp. 3212–3217 (2009)

  40. Rusu, R.B., Blodow, N., Marton, Z.C., Beetz, M.: Aligning point cloud views using persistent feature histograms. In: Intelligent RObots and Systems, pp. 3384–3391 (2008)

  41. Salti, S., Tombari, F., Di Stefano, L.: Shot: unique signatures of histograms for surface and texture description. Comput. Vis. Image Underst. 125, 251–264 (2014)

    Article  Google Scholar 

  42. Shah, S.A.A., Bennamoun, M., Boussaid, F.: A novel 3d vorticity based approach for automatic registration of low resolution range images. Pattern Recognit. 48(9), 2859–2871 (2015)

    Article  Google Scholar 

  43. Shah, S.A.A., Bennamoun, M., Boussaid, F.: A novel feature representation for automatic 3d object recognition in cluttered scenes. Neurocomputing 205, 1–15 (2015)

    Article  Google Scholar 

  44. Shang, L., Greenspan, M.: Real-time object recognition in sparse range images using error surface embedding. Int. J. Comput. Vis. 89(2), 211–228 (2010)

    Article  Google Scholar 

  45. Sun, T., Liu, S., Liu, G., Zhu, S., Zhu, Z.: A 3d descriptor based on local height image. In: 2018 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–5 (2018). https://doi.org/10.1109/ISCAS.2018.8351336

  46. Taati, B., Greenspan, M.: Local shape descriptor selection for object recognition in range data. Comput. Vis. Image Underst. 115(5), 681–694 (2011). Special issue on 3D Imaging and Modelling

    Article  Google Scholar 

  47. Tombari, F., Salti, S., Di Stefano, L.: Unique shape context for 3d data description. In: Proceedings of the ACM Workshop on 3D Object Retrieval, 3DOR ’10, pp. 57–62. ACM, New York, NY, USA (2010)

  48. Tombari, F., Salti, S., Di Stefano, L.: Unique signatures of histograms for local surface description. In: Computer Vision - ECCV 2010: 11th European Conference on Computer Vision. Heraklion, Crete, Greece, 5–11 September 2010, Proceedings, Part III, pp. 356–369. Springer, Berlin (2010)

  49. Tombari, F., Salti, S., Di Stefano, L.: Performance evaluation of 3d keypoint detectors. Int. J. Comput. Vis. 102(1), 198–220 (2013)

    Article  Google Scholar 

  50. Tombari, F., Salti, S., Stefano, L.D.: A combined texture-shape descriptor for enhanced 3d feature matching. In: 2011 18th IEEE International Conference on Image Processing, pp. 809–812 (2011)

  51. Yang, J., Cao, Z., Zhang, Q.: A fast and robust local descriptor for 3d point cloud registration. Inf. Sci. 346–347, 163–179 (2016). https://doi.org/10.1016/j.ins.2016.01.095

    Article  Google Scholar 

  52. Yang, J., Zhang, Q., Xian, K., Xiao, Y., Cao, Z.: Rotational contour signatures for both real-valued and binary feature representations of 3d local shape. Comput. Vis. Image Underst. 160, 133–147 (2017). https://doi.org/10.1016/j.cviu.2017.02.004

    Article  Google Scholar 

  53. Yang, J., Zhang, Q., Xiao, Y., Cao, Z.: Toldi: An effective and robust approach for 3d local shape description. Pattern Recognit. 65, 175–187 (2017). https://doi.org/10.1016/j.patcog.2016.11.019

    Article  Google Scholar 

  54. Zaharescu, A., Boyer, E., Varanasi, K., Horaud, R.: Surface feature detection and description with applications to mesh matching. In: Computer Vision and Pattern Recognition, pp. 373–380 (2009)

  55. Zhong, Y.: Intrinsic shape signatures: a shape descriptor for 3d object recognition. In: Computer Vision Workshops (ICCV Workshops), 2009 IEEE 12th International Conference on, pp. 689–696 (2009)

  56. Zou, Y., Wang, X., Zhang, T., Liang, B., Song, J., Liu, H.: Broph: an efficient and compact binary descriptor for 3d point clouds. Pattern Recognit. 76, 522–536 (2018). https://doi.org/10.1016/j.patcog.2017.11.029

    Article  Google Scholar 

Download references

Acknowledgements

The authors wish to thank owners of datasets used in this work for making those datasets publicly available. This work is supported by the National Natural Science Foundation of China (Grant No. 61703017).

Author information

Authors and Affiliations

  1. School of Astronautics, Beihang University, Xueyuan Road, Haidian District, Beijing, 100191, China

    Wulong Guo & Weiduo Hu

  2. Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China

    Chang Liu

  3. Research and Development Department, China Academy of Launch Vehicle Technology, Beijing, China

    Tingting Lu

Authors
  1. Wulong Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weiduo Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tingting Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wulong Guo.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, W., Hu, W., Liu, C. et al. 3D object recognition from cluttered and occluded scenes with a compact local feature. Machine Vision and Applications 30, 763–783 (2019). https://doi.org/10.1007/s00138-019-01027-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-019-01027-7

Keywords

Search

Navigation