Skip to main content
SpringerLink
Log in

A 3D Scene Registration Method via Covariance Descriptors and an Evolutionary Stable Strategy Game Theory Solver

Fusing Photometric and Shape-Based Features

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In this paper we provide an integrated approach for matching patterns in scenes combining 3D and visual information. For local definition of points we propose a descriptor based on the notion of covariance of features for fusion of shape and color information of 3D surfaces, so-called multi-scale covariance descriptor (MCOV). The intrinsic properties of this descriptor are many: it is invariant to spatial rigid transformations, and robust to noise and resolution changes; it can also be used for characteristic point detection; and lies on top of a manifold topology which allows the use of analytical metric properties. This descriptor is complemented with a game theoretic approach for solving the matching correspondences under global geometric constraints. This layer offers a comprehensive understanding of the scene and avoids possible mismatches due to repeated areas or symmetries—which would be impossibly identified by the detector solely at a local level. Our solution is able to accurately match different views of a scene even under spatial transformations, high noise levels and with small overlap between views, outperforming state-of-the-art approaches. Results are validated by comparing MCOV against other state-of-the-art 3D point descriptor methods, and matching complex 3D and color scenes under several challenging conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. http://www.123dapp.com/catch

References

  • Albarelli, A., Rodola, E., & Torsello, A. (2010). A game-theoretic approach to fine surface registration without initial motionestimation. In International conference on computer vision and pattern recognition (CVPR) (pp. 430–437).

  • Albarelli, A., Rota Bulò, S., Torsello, A., & Pelillo, M. (2009). Matching as a non-cooperative game. In International conference on computer vision (ICCV) (pp. 1319–1326).

  • Arsigny, V., Fillard, P., Pennec, X., & Ayache, N. (2006). Log-Euclidean metrics for fast and simple calculus on diffusion tensors. Magnetic Resonance in Medicine, 56(2), 411–421.

    Article  Google Scholar 

  • Besl, P., & McKay, N. (1992). A method for registration of 3-d shapes. Transactions on Pattern Analysis and Machine Intelligence, PAMI, 14, 239–256.

    Article  Google Scholar 

  • Brusco, N., Andreetto, M., Giorgi, A., & Cortelazzo, G. M. (2005). 3d registration by textured spin-images. In International conference on 3D digital imaging and modeling (3DIM) (pp. 262–269).

  • Chen, C. S., Hung, Y. P., & Cheng, J. B. (1999). RANSAC-based darces: A new approach to fast automatic registration of partially overlapping range images. Transactions on Pattern Analysis and Machine Intelligence, PAMI, 21, 1229–1234.

    Article  Google Scholar 

  • Cherian, A., Sra, S., Banerjee, A., & Papanikolopoulos, N. (2011). Efficient similarity search for covariance matrices via the Jensen–Bregman logdet divergence. In International conference on computer vision (ICCV) (pp. 2399–2406). IEEE.

  • Chua, C., & Jarvis, R. (1997). Point signatures: A new representation for 3D object recognition. IJCV, 25, 63–85.

    Article  Google Scholar 

  • Chum, O. & Matas, J. (2005): Matching with PROSAC-progressive sample consensus. In International conference on computer vision and pattern recognition (CVPR) (Vol. 1, pp. 220–226).

  • Chum, O., & Matas, J. (2008). Optimal randomized RANSAC. Transactions on Pattern Analysis and Machine Intelligence, PAMI, 30, 1472–1482.

    Article  Google Scholar 

  • Chum, O., Matas, J., & Obdrzalek, S. (2004). Enhancing RANSAC by generalized model optimization. In Asian conference on computer vision (ACCV) (Vol. 2, pp. 812–817).

  • Fehr, D., Cherian, A., Sivalingam, R., Nickolay, S., Morellas, V., & Papanikolopoulos, N. (2012). Compact covariance descriptors in 3d point clouds for object recognition. In International conference on robotics and automation (ICRA) (pp. 1793–1798).

  • Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24, 381–395.

    Article  MathSciNet  Google Scholar 

  • Flint, A., Dick, A. R., & Van Den Hengel, A. (2007). Thrift: Local 3d structure recognition. Digital Image Computing Techniques and Applications, 7, 182–188.

    Google Scholar 

  • Förstner, W. & Moonen, B. (1999). A metric for covariance matrices. Quo vadis geodesia (pp. 113–128).

  • Frome, A., Huber, D., Kolluri, R., Bülow, T., & Malik, J. (2004). Recognizing objects in range data using regional point descriptors. ECCV, 3023, 224–237.

    Google Scholar 

  • Hartley, R., & Zisserman, A. (2003). Multiple view geometry in computer vision. Cambridge: Cambridge University Press.

    Google Scholar 

  • Horn, B. K. P. (1987). Closed-form solution of absolute orientation using unit quaternions. Optical Society of America, 4, 629–642.

    Article  Google Scholar 

  • Johnson, A. (1997). Spin-images: A representation for 3-d surface matching. Ph.D. thesis. Pittsburgh, PA: Robotics Institute, Carnegie Mellon University.

  • Johnson, A., & Hebert, M. (1999). Using spin images for efficient object recognition in cluttered 3d scenes. Transactions on Pattern Analysis and Machine Intelligence, PAMI, 21, 433–449.

    Article  Google Scholar 

  • Kovnatsky, A., Bronstein, M. M., Bronstein, A. M., & Kimmel, R. (2012). Photometric heat kernel signatures. In Scale space and variational methods in computer vision (pp. 616–627).

  • Lai, K., Bo, L., Ren, X., & Fox, D. (2011). A large-scale hierarchical multi-view RGB-D object dataset. In International conference on robotics and automation (ICRA) (pp. 1817–1824).

  • Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. IJCV, 60, 91–110.

    Article  Google Scholar 

  • Luebke, D. P. (2001). A developer’s survey of polygonal simplification algorithms. Computer Graphics and Applications, 21, 24–35.

    Article  Google Scholar 

  • Rodolà, E., Albarelli, A., Bergamasco, F., & Torsello, A. (2013). A scale independent selection process for 3d object recognition in cluttered scenes. International Journal of Computer Vision, IJCV, 102((1–3)), 129–145.

  • Rusu, R., Blodow, N., & Beetz, M. (2009). Fast point feature histograms (FPFH) for 3d registration. In International conference on robotics and automation (ICRA) (pp. 3212–3217).

  • Tombari, F., Salti, S., & Di Stefano, L. (2011). A combined texture-shape descriptor for enhanced 3d feature matching. In International conference on image processing (ICIP) (pp. 809–812).

  • Tombari, F., Salti, S., & Stefano, L. (2010). Unique signatures of histograms for local surface description. In European conference on computer vision (ECCV) (Vol. 6313, pp. 356–369).

  • Torsello, A., Rodolà, E., & Albarelli, A. (2011). Sampling relevant points for surface registration. In Conference on 3D imaging, modeling, processing, visualization and transmission (3DIMPVT) (pp. 290–295).

  • Tuzel, O., Porikli, F., & Meer, P. (2006). Region covariance: A fast descriptor for detection and classification. In European conference on computer vision (ECCV) (pp. 589–600).

  • Tuzel, O., Porikli, F., & Meer, P. (2007). Human detection via classification on Riemannian manifolds. International conference on computer vision and pattern recognition (CVPR) (pp. 1–8).

  • Tuzel, O., Porikli, F., & Meer, P. (2008). Pedestrian detection via classification on Riemannian manifolds. Transactions on Pattern Analysis and Machine Intelligence, PAMI, 30, 1713–1727.

    Article  Google Scholar 

  • Wilks, S. S. (1932). Certain generalizations in the analysis of variance. Biometrika, 24, 471–494.

    Article  Google Scholar 

  • Yao, J., Odobez, J., et al. (2008). Fast human detection from videos using covariance features. In International workshop on visual surveillance, VS2008.

  • Zaharescu, A., Boyer, E., & Horaud, R. (2012). Keypoints and local descriptors of scalar functions on 2d manifolds. International Journal of Computer Vision, IJCV, 100, 78–98.

    Article  MATH  Google Scholar 

  • Zaharescu, A., Boyer, E., Varanasi, K., & Horaud, R. (2009). Surface feature detection and description with applications to mesh matching. In CVPR (pp. 373–380).

Download references

Acknowledgments

This work has been supported in part by the following research Project Grants: TIN2012-39203 and IPT-2012-0630-020000 awarded by the Spanish Government Ministry of Economy and Competitivity.

Author information

Authors and Affiliations

  1. Department of Information and Communication Technologies, Universitat Pompeu Fabra, Tànger 122-140, 08018, Barcelona, Spain

    Pol Cirujeda, Yashin Dicente Cid, Xavier Mateo & Xavier Binefa

Authors
  1. Pol Cirujeda
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yashin Dicente Cid
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xavier Mateo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xavier Binefa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pol Cirujeda.

Additional information

Communicated by S. Soatto.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cirujeda, P., Dicente Cid, Y., Mateo, X. et al. A 3D Scene Registration Method via Covariance Descriptors and an Evolutionary Stable Strategy Game Theory Solver. Int J Comput Vis 115, 306–329 (2015). https://doi.org/10.1007/s11263-015-0820-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-015-0820-2

Keywords

Search

Navigation