Skip to main content
SpringerLink
Log in

GlcMatch: global and local constraints for reliable feature matching

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

A match is considered as an incorrect match when the matched features in two views do not correspond to the same physical location. It is inevitable that generates mismatches at a local descriptor level. Differentiating true and false matches remains a challenge, especially in the case of ambiguities, wide baselines, and strong illumination variations, which might contain a large number of mismatches (even up to 90%). In this paper, we develop GlcMatch, an outlier rejection method that takes advantage of both global and local constraints to classify putative matches. Specifically, we use vector field consistency to form continuous global smoothness and use triangular mesh constraints to implement the local piecewise smoothness. Evaluation on benchmark datasets demonstrates GlcMatch can obtain large numbers of good quality correspondences and achieve significant performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Aronszajn, N.: Theory of reproducing kernels. Trans. Am. Math. Soc. 68(3), 337–404 (1950)

    Article  MathSciNet  MATH  Google Scholar 

  2. Avidan, S.: Ensemble tracking. IEEE Trans. Pattern Anal. Mach. Intell. 29(2), 261–71 (2007)

    Article  Google Scholar 

  3. Baldassarre, L., Rosasco, L., Barla, A., Verri, A.: Vector field learning via spectral filtering. In: European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 56–71. Springer (2010)

  4. Baldassarre, L., Rosasco, L., Barla, A., Verri, A.: Multi-output learning via spectral filtering. Mach. Learn. 87(3), 259–301 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  5. Balntas, V., Lenc, K., Vedaldi, A., Mikolajczyk, K.: Hpatches: a benchmark and evaluation of handcrafted and learned local descriptors. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3852–3861 (2017)

  6. Barath, D., Matas, J.: Graph-cut ransac. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 6733–6741 (2018)

  7. Barath, D., Matas, J., Noskova, J.: Magsac: marginalizing sample consensus. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 10197–10205 (2019)

  8. Bartoli, A.: Maximizing the predictivity of smooth deformable image warps through cross-validation. J. Math. Imag. Vis. 31(2–3), 133–145 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  9. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Compt. Vis. Image Underst. 110(3), 346–359 (2008)

    Article  Google Scholar 

  10. Bentoutou, Y., Taleb, N., Kpalma, K., Ronsin, J.: An automatic image registration for applications in remote sensing. IEEE Trans. Geosci. Remote Sens. 43(9), 2127–2137 (2005)

    Article  Google Scholar 

  11. Bian, J., Lin, W.Y., Matsushita, Y., Yeung, S.K., Nguyen, T.D., Cheng, M.M.: Gms: grid-based motion statistics for fast, ultra-robust feature correspondence. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4181–4190 (2017)

  12. Cabral, B., Leedom, L.C.: Imaging vector fields using line integral convolution. In: Conference on Computer Graphics and Interactive Techniques, pp. 263-270 (1993)

  13. Caetano, T.S., McAuley, J.J., Cheng, L., Le, Q.V., Smola, A.J.: Learning graph matching. IEEE Trans. Pattern Anal. Mach. Intell. 31(6), 1048–58 (2009)

    Article  Google Scholar 

  14. Carmeli, C., De Vito, E., Toigo, A.: Vector valued reproducing kernel hilbert spaces of integrable functions and mercer theorem. Anal. Appl. 4(04), 377–408 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  15. Chum, O., Matas, J., Kittler, J.: Locally optimized ransac. In: Joint Pattern Recognition Symposium, pp. 236–243. Springer (2003)

  16. Collins, T., Mesejo, P., Bartoli, A.: An analysis of errors in graph-based keypoint matching and proposed solutions. In: European Conference on Computer Vision, vol. 8695, pp. 138–153. Springer (2014)

  17. DeTone, D., Malisiewicz, T., Rabinovich, A.: Superpoint: self-supervised interest point detection and description. In: Computer Vision and Pattern Recognition, pp. 337–33712 (2018)

  18. Evgeniou, T., Pontil, M., Poggio, T.: Regularization networks and support vector machines. Adv. Comput. Math. 13(1), 1 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  19. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  20. Guclu, O., Can, A.B.: Integrating global and local image features for enhanced loop closure detection in rgb-d slam systems. Vis. Comput. 36(6), 1271–1290 (2020)

    Article  Google Scholar 

  21. Han, J., Zhang, D., Hu, X., Guo, L., Ren, J., Wu, F.: Background prior-based salient object detection via deep reconstruction residual. IEEE Trans. Circ. Syst. Video Technol. 25(8), 1309–1321 (2014)

    Google Scholar 

  22. Heinly, J., Dunn, E., Frahm, J.M.: Comparative evaluation of binary features. In: European Conference on Computer Vision, pp. 759–773 (2012)

  23. Kazhdan, M., Bolitho, M., Hoppe, H.: Poisson surface reconstruction. In: Symposium on Geometry Processing, pp. 61–70 (2006)

  24. Leordeanu, M., Hebert, M.: A spectral technique for correspondence problems using pairwise constraints. In: IEEE International Conference on Computer Vision, vol. 2, pp. 1482–1489. IEEE (2005)

  25. Leordeanu, M., Sukthankar, R., Hebert, M.: Unsupervised learning for graph matching. Int. J. Comput. Vis. 96(1), 28–45 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  26. Lin, W.Y., Liu, S., Jiang, N., Do, M.N., Tan, P., Lu, J.: Repmatch: robust feature matching and pose for reconstructing modern cities. In: European Conference on Computer Vision, pp. 562–579. Springer (2016)

  27. Lin, W.Y., Wang, F., Cheng, M.M., Yeung, S.K., Torr, P.H.S., Do, M.N., Lu, J.: Code: coherence based decision boundaries for feature correspondence. IEEE Trans. Pattern Anal. Mach. Intell. 40(1), 34–47 (2018)

    Article  Google Scholar 

  28. Lin, W.Y.D., Cheng, M.M., Lu, J., Yang, H., Do, M.N., Torr, P.: Bilateral functions for global motion modeling. In: European Conference on Computer Vision, pp. 341–356 (2014)

  29. Liu, H., Yan, S.: Common visual pattern discovery via spatially coherent correspondences. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1609–1616 (2010)

  30. Liu, Z., Marlet, R.: Virtual line descriptor and semi-local graph matching method for reliable feature correspondence. In: British Machine Vision Conference, pp. 1–11 (2012)

  31. Liu, Z., Xiang, Q., Tang, J., Wang, Y., Zhao, P.: Robust salient object detection for rgb images. Vis. Comput. 36(9), 1823–1835 (2020)

    Article  Google Scholar 

  32. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  33. Ma, J., Zhao, J., Tian, J., Bai, X., Tu, Z.: Regularized vector field learning with sparse approximation for mismatch removal. Pattern Recognit. 46(12), 3519–3532 (2013)

    Article  MATH  Google Scholar 

  34. Micchelli, C.A., Pontil, M.: On learning vector-valued functions. Neural Comput. 17(1), 177–204 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  35. Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005)

    Article  Google Scholar 

  36. Min, J., Cho, M.: Convolutional hough matching networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2940–2950 (2021)

  37. Morel, J.M., Yu, G.: Asift: a new framework for fully affine invariant image comparison. Siam J. Imag. Sci. 2(2), 438–469 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  38. Mur-Artal, R., Montiel, J.M.M., Tardos, J.D.: Orb-slam: a versatile and accurate monocular slam system. IEEE Trans. Robot. 31(5), 1147–1163 (2015)

    Article  Google Scholar 

  39. Ni, K., Jin, H., Dellaert, F.: Groupsac: efficient consensus in the presence of groupings. In: IEEE International Conference on Computer Vision, pp. 2193–2200. IEEE (2009)

  40. Olsson, C., Enqvist, O.: Stable structure from motion for unordered image collections. In: Scandinavian Conference on Image Analysis, pp. 524–535. Springer (2011)

  41. Pizarro, D., Bartoli, A.: Feature-based deformable surface detection with self-occlusion reasoning. Int. J. Comput. Vis. 97(1), 54–70 (2012)

    Article  MATH  Google Scholar 

  42. Ranftl, R., Koltun, V.: Deep fundamental matrix estimation. In: European Conference on Computer Vision, pp. 292–309 (2018)

  43. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.R.: Orb: An efficient alternative to sift or surf. In: IEEE International Conference on Computer Vision, vol. 11, p. 2. Citeseer (2011)

  44. Sarlin, P.E., DeTone, D., Malisiewicz, T., Rabinovich, A.: Superglue: learning feature matching with graph neural networks. In: Computer Vision and Pattern Recognition, pp. 4938–4947 (2020)

  45. Schonberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: Computer Vision and Pattern Recognition, pp. 4104–4113 (2016). https://doi.org/10.1109/CVPR.2016.445

  46. Schonberger, J.L., Hardmeier, H., Sattler, T., Pollefeys, M.: Comparative evaluation of hand-crafted and learned local features. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 6959–6968 (2017). https://doi.org/10.1109/CVPR.2017.736

  47. Schönberger, J.L., Zheng, E., Frahm, J.M., Pollefeys, M.: Pixelwise view selection for unstructured multi-view stereo. In: European Conference on Computer Vision, pp. 501–518 (2016). https://doi.org/10.1007/978-3-319-46487-9_31

  48. Tian, Y., Yu, X., Fan, B., Wu, F., Heijnen, H., Balntas, V.: Sosnet: second order similarity regularization for local descriptor learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 11016–11025 (2019)

  49. Torresani, L., Kolmogorov, V., Rother, C.: Feature correspondence via graph matching: models and global optimization. In: European Conference on Computer Vision, pp. 596–609. Springer (2008)

  50. Yi, K.M., Trulls, E., Ono, Y., Lepetit, V., Salzmann, M., Fua, P.: Learning to find good correspondences. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2666–2674 (2018)

  51. Zanfir, A., Sminchisescu, C.: Deep learning of graph matching. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2684–2693 (2018)

  52. Zhang, J., Sun, D., Luo, Z., Yao, A., Zhou, L., Shen, T., Chen, Y., Liao, H., Quan, L.: Learning two-view correspondences and geometry using order-aware network. In: IEEE International Conference on Computer Vision, pp. 5845–5854 (2019)

  53. Zhao, J., Ma, J., Tian, J., Ma, J., Zhang, D.: A robust method for vector field learning with application to mismatch removing. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2977–2984. IEEE (2011)

  54. Zhao, X., Wang, G., He, Z., Liang, D., Zhang, S., Tan, J.: Unsupervised inner-point-pairs model for unseen-scene and online moving object detection. Vis. Comput. 1–17 (2022)

  55. Zhou, F., De la Torre, F.: Factorized graph matching. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 127–134. IEEE (2012)

Download references

Acknowledgements

This work was supported in part by the National Key Research and Development Program of China under Grant 2020YFC1523100, in part by the Fundamental Research Funds for the Central Universities under Grant PA2021GDGP0061, and in part by the National Natural Science Foundation of China under Grant 61877016.

Author information

Authors and Affiliations

  1. School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, 230009, China

    Youcheng Cai, Lin Li, Dong Wang, Xintao Huang & Xiaoping Liu

  2. Engineering Research Center of Safety Critical Industrial Measurement and Control Technology, Ministry of Education, Hefei, 230009, China

    Xiaoping Liu

Authors
  1. Youcheng Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xintao Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaoping Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoping Liu.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cai, Y., Li, L., Wang, D. et al. GlcMatch: global and local constraints for reliable feature matching. Vis Comput 39, 2555–2570 (2023). https://doi.org/10.1007/s00371-022-02478-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02478-2

Keywords

Search

Navigation