Skip to main content
SpringerLink
Log in

Dynamic-scale grid structure with weighted-scoring strategy for fast feature matching

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Incorporating correspondences geometric consistency constraints into feature mismatch removal is known to enable robust matching. Most existing methods often take a long computational time or have high requirements for equipment, limiting their use in real-time applications. This paper attempts to remove the mismatches from the putative correspondence set at high speed. We propose a dynamic-scale grid structure into the mismatch removal stage to reduce the time complexity of neighborhood construction. We use descriptor size to simulate the relative scale between two images, then make the grid structure scale dynamic. Furthermore, we put forward a Gaussian-based weighted scoring strategy to combine the descriptor matching stability with geometric consistency. This strategy uses a Gaussian function to give each putative correspondence a different stability score, making the model more robust in complex scenarios. Finally, compared with other real-time or sophisticated models, our model achieves better performance and intensively reduces time cost.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Sturm J, Engelhard N, Endres F, Burgard W, Cremers D (2012) A benchmark for the evaluation of RGB-D SLAM systems. In: 2012 IEEE/RSJ international conference on intelligent robots and systems. https://doi.org/10.1109/iros.2012.6385773

  2. Geiger A, Lenz P, Urtasun R (2012) Are we ready for autonomous driving? the kitti vision benchmark suite. In: 2012 IEEE conference on computer vision and pattern recognition. https://doi.org/10.1109/cvpr.2012.6248074

  3. Knapitsch A, Park J, Zhou Q -Y, Koltun V (2017) Tanks and temples: benchmarking Large-Scale scene reconstruction. ACM Transactions on Graphics 36:78. https://doi.org/10.1145/3072959.3073599

    Article  Google Scholar 

  4. Wilson K, Snavely N (2014) Robust global translations with 1dsfm. In: European Conference on Computer Vision. https://doi.org/10.1007/978-3-319-10578-9_5

  5. Pauline C N, Steven H (2003) SIFT: Predicting amino acid changes that affect protein function. Nucleic Acids Research 31:3812–3814. https://doi.org/10.1093/nar/gkg509

    Article  Google Scholar 

  6. Bian J, Lin W, Liu Y, Zhang L, Yeung S, Cheng M, Reid L (2020) GMS: Grid-based motion statistics for fast, ultra-robust feature correspondence. Int J Comput Vis 128(6):1580–1593. https://doi.org/10.1007/s11263-019-01280-3

    Article  MathSciNet  Google Scholar 

  7. Ma J, Jiang X, Jiang J, Zhao J, Guo X (2019) LMR: Learning a Two-Class classifier for mismatch removal. IEEE Trans Image Process 2:4045–4059. https://doi.org/10.1109/TIP.2019.2906490

    Article  MathSciNet  Google Scholar 

  8. Ma J, Zhao J, Jiang J, Zhou H, Guo X (2019) Locality preserving matching. Int J Comput Vis 127:512–531. https://doi.org/10.24963/ijcai.2017/627

    Article  MathSciNet  MATH  Google Scholar 

  9. Liu Y, Li Y, Dai L, Yang C, Wei L, Lai T, Chen R (2021) Robust feature matching via advanced neighborhood topology consensus. Neurocomputing 421:273–284. https://doi.org/10.1016/j.neucom.2020.09.047

    Article  Google Scholar 

  10. Sarlin P-E, DeTone D, Malisiewicz T, Rabinovich A (2020) Superglue: Learning feature matching with graph neural networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. https://doi.org/10.1109/cvpr42600.2020.00499

  11. Morel J-M, Yu GJSjois (2009) . ASIFT: A new framework for fully affine invariant image comparison 2:438–469. https://doi.org/10.1137/080732730

    Article  Google Scholar 

  12. Mainali P, Lafruit G, Yang Q, Geelen B, Van Gool L, Lauwereins R (2013) SIFER: Scale-invariant feature detector with error resilience. Int J Comput Vis 104:172–197. https://doi.org/10.1007/s11263-013-0622-3

    Article  MATH  Google Scholar 

  13. Rublee E, Rabaud V, Konolige K, Bradski G (2012) ORB: An efficient alternative to SIFT or SURF. In: International Conference on Computer Vision. https://doi.org/10.1109/iccv.2011.6126544

  14. Aldana-Iuit J, Mishkin D, Chum O, Matas J (2016) In the saddle: chasing fast and repeatable features. In: 2016 23rd International Conference on Pattern Recognition (ICPR)

  15. DeTone D, Malisiewicz T, Rabinovich A (2018) SuperPoint: Self-supervised interest point detection and description. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops. https://doi.org/10.1109/cvprw.2018.00060

  16. Bhowmik A, Gumhold S, Rother C, Brachmann E (2020) Reinforced feature points: Optimizing feature detection and description for a high-level task. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. https://doi.org/10.1109/cvpr42600.2020.00500

  17. Dai J, Jin S, Zhang J, Nguyen T Q (2020) Boosting feature matching accuracy with pairwise affine estimation. IEEE Trans Image Process 29:8278–8291. https://doi.org/10.1109/tip.2020.3013384

    Article  Google Scholar 

  18. Wang C, Wang L, Liu L (2014) Progressive mode-seeking on graphs for sparse feature matching. In: European conference on computer vision. https://doi.org/10.1007/978-3-319-10605-2_51

  19. FM A, Bolles R C (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 26(6):381–395. https://doi.org/10.1016/b978-0-08-051581-6.50070-2

    Article  MathSciNet  Google Scholar 

  20. Fragoso V, Sen P, Rodriguez S, Turk M (2013) EVSAC: accelerating hypotheses generation by modeling matching scores with extreme value theory. In: Proceedings of the IEEE international conference on computer vision. https://doi.org/10.1109/iccv.2013.307

  21. Barath D, Ivashechkin M, Japa MJ (2019) Progressive NAPSAC: sampling from gradually growing neighborhoods. arXiv:1906.022951906.02295

  22. Ma J, Jiang X, Fan A, Jiang J, Yan JJIJoCV (2021) . Image matching from handcrafted to deep features: A survey 129:23–79. https://doi.org/10.1007/s11263-020-01359-2

    Article  Google Scholar 

  23. Leordeanu M, Hebert M (2005) A spectral technique for correspondence problems using pairwise constraints. In: Tenth IEEE international conference on computer vision. https://doi.org/10.1109/ICCV.2005.20

  24. Liu H, Yan S (2010) Common visual pattern discovery via spatially coherent correspondences. In: 2010 IEEE computer society conference on computer vision and pattern recognition. https://doi.org/10.1109/cvpr.2010.5539780

  25. Lin W-YD, Cheng M-M, Lu J, Yang H, Do MN, Torr P (2014) Bilateral functions for global motion modeling. European conference on computer vision. https://doi.org/10.1007/978-3-319-10593-2_23

  26. Lin W, Wang F, Cheng M, Yeung S, Torr P H S, Do M N, Lu J (2018) CODE: coherence based decision boundaries for feature correspondence. IEEE Trans Pattern Anal Mach Intell 40:34–47. https://doi.org/10.1109/tpami.2017.2652468

    Article  Google Scholar 

  27. Bian J, Lin W, Matsushita Y, Yeung S-K, Nguyen T-D, Cheng M (2017) GMS: Grid-based motion statistics for fast, ultra-robust feature correspondence. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). https://doi.org/10.1109/cvpr.2017.302

  28. Wang G, Chen Y (2021) Robust feature matching using guided local outlier factor. Pattern Recogn 117:107986. https://doi.org/10.1016/j.patcog.2021.107986

    Article  Google Scholar 

  29. Sun J, Shen Z, Wang Y, Bao H, Zhou X (2021) LoFTR: Detector-free local feature matching with transformers. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

  30. Zhou Q, Sattler T, Leal-Taixe L (2021) Patch2pix: Epipolar-guided pixel-level correspondences. In: Proceedings of the IEEE/ CVF conference on computer vision and pattern recognition

  31. Moore A (1991) An introductory tutorial on kd-trees. IEEE colloquium on quantum computing: theory, applications & implications

  32. Beis JS, Lowe DG (1997) Shape indexing using approximate nearest-neighbour search in high-dimensional spaces. Proceedings of IEEE computer society conference on computer vision and pattern recognition. https://doi.org/10.1109/cvpr.1997.609451

  33. Muja M, Lowe D (2009) Flann: Fast library for approximate nearest neighbors user manual. Computer Science Department 5 https://doi.org/10.1109/icscn.2017.8085676

  34. Li Y, Huang Q, Liu Y, Huang Y, Sun X (2019) Robust feature matching via multi-scale grid structure. In: 2019 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/ SocialCom/SustainCom). https://doi.org/10.1109/ispa-bdcloud-sustaincom-socialcom48970.2019.00169

  35. Zheng Z, Ma Y, Zheng H, Ju J, Lin MJIA (2018) . UGC: Real-time, ultra-robust feature correspondence via unilateral grid-based clustering 6:55501–55508. https://doi.org/10.1109/access.2018.2871729

    Google Scholar 

  36. Bian J, Wu Y, Zhao J, Liu Y, Zhang L, Cheng M, Reid L (2019) An evaluation of feature matchers for fundamental matrix estimation. British machine vision conference (BMVC)

  37. Jin Y, Mishkin D, Mishchuk A, Matas J, Fua P, Yi K M, Trulls E (2021) Image matching across wide baselines: from paper to practice. Int J Comput Vis 129:517–547. https://doi.org/10.1007/s11263-020-01385-0

    Article  Google Scholar 

  38. Sloane NJA (2001) Information Sciences Research Center, AT&T Shannon Labs http://neilsloane.com/oadir/

Download references

Acknowledgment

This work is partly supported by the National Key Research and Development Program of China under grant no. 2018YFC0407905, and the fundamental research funds of China for central universities under grant no. B200202188.

Author information

Authors and Affiliations

  1. Key Laboratory of Water Big Data Technology of Ministry of Water Resources, Hohai University, Nanjing, China

    Lijie Yang, Qian Huang, Xing Li & Yang Yuan

  2. School of Computer and Information, Hohai University, Nanjing, China

    Lijie Yang, Qian Huang, Xing Li & Yang Yuan

Authors
  1. Lijie Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qian Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yang Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qian Huang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, L., Huang, Q., Li, X. et al. Dynamic-scale grid structure with weighted-scoring strategy for fast feature matching. Appl Intell 52, 10576–10590 (2022). https://doi.org/10.1007/s10489-021-02990-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02990-3

Keywords

Search

Navigation