Skip to main content
SpringerLink
Log in

Feature Matching via Motion-Consistency Driven Probabilistic Graphical Model

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

This paper proposes an effective method, termed as motion-consistency driven matching (MCDM), for mismatch removal from given tentative correspondences between two feature sets. In particular, we regard each correspondence as a hypothetical node, and formulate the matching problem into a probabilistic graphical model to infer the state of each node (e.g., true or false correspondence). By investigating the motion consistency of true correspondences, a general prior is incorporated into our formulation to differentiate false correspondences from the true ones. The final inference is casted into an integer quadratic programming problem, and the solution is obtained by using an efficient optimization technique based on the Frank-Wolfe algorithm. Extensive experiments on general feature matching, as well as fundamental matrix estimation, relative pose estimation and loop-closure detection, demonstrate that our MCDM possesses strong generalization ability as well as high accuracy, which outperforms state-of-the-art methods. Meanwhile, due to the low computational complexity, the proposed method is efficient for practical feature matching tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. https://cs.adelaide.edu.au/~hwong/doku.php?id=data.

  2. http://cmp.felk.cvut.cz/wbs/.

References

  • Angeli, A., Filliat, D., Doncieux, S., & Meyer, J. A. (2008). Fast and incremental method for loop-closure detection using bags of visual words. IEEE Transactions on Robotics, 24(5), 1027–1037.

    Article  Google Scholar 

  • Baeza-Yates, R., Ribeiro-Neto, B., et al. (1999). Modern information retrieval. ACM Press.

  • Barath, D., & Matas, J. (2018). Graph-cut ransac. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 6733–6741).

  • Barath, D., Matas, J., & Noskova, J. (2019). Magsac: Marginalizing sample consensus. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 10197–10205).

  • Barath, D., Noskova, J., Ivashechkin, M., & Matas, J. (2020). Magsac++, a fast, reliable and accurate robust estimator. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 1304–1312).

  • Bay, H., Tuytelaars, T., & Van Gool, L. (2006). Surf: Speeded up robust features. In European conference on computer vision (pp. 404–417).

  • Bentley, J. L. (1975). Multidimensional binary search trees used for associative searching. Commun. ACM, 18(9), 509–517.

    Article  Google Scholar 

  • Bian, J., Lin, W. Y., Liu, Y., Zhang, L., Yeung, S. K., Cheng, M. M., & Reid, I. (2020). GMS: Grid-based motion statistics for fast, ultra-robust feature correspondence. International Journal of Computer Vision, 128(6), 1580–1593.

    Article  MathSciNet  Google Scholar 

  • Bian, J. W., Wu, Y. H., Zhao, J., Liu, Y., Zhang, L., Cheng, M. M., & Reid, I. (2019). An evaluation of feature matchers for fundamental matrix estimation. arXiv preprint arXiv:1908.09474

  • Burri, M., Nikolic, J., Gohl, P., Schneider, T., Rehder, J., Omari, S., Achtelik, M. W., & Siegwart, R. (2016). The euroc micro aerial vehicle datasets. The International Journal of Robotics Research, 35(10), 1157–1163.

    Article  Google Scholar 

  • Bustos, Á. P., & Chin, T. J. (2018). Guaranteed outlier removal for point cloud registration with correspondences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12), 2868–2882.

    Article  Google Scholar 

  • Cho, M., Lee, J., & Lee, K. M. (2010). Reweighted random walks for graph matching. In European conference on computer vision (pp. 492–505).

  • Choy, C., Lee, J., Ranftl, R., Park, J., & Koltun, V. (2020). High-dimensional convolutional networks for geometric pattern recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 11227–11236).

  • Chum, O., & Matas, J. (2005). Matching with prosac-progressive sample consensus. In IEEE computer society conference on computer vision and pattern recognition (Vol. 1, pp. 220–226).

  • Chum, O., Matas, J., & Kittler, J. (2003). Locally optimized ransac. In Proceedings of the joint pattern recognition symposium (pp. 236–243).

  • Chum, O., Werner, T., & Matas, J. (2005). Two-view geometry estimation unaffected by a dominant plane. In IEEE computer society conference on computer vision and pattern recognition (pp. 772–779).

  • Deng, H., Birdal, T., & Ilic, S. (2018). Ppfnet: Global context aware local features for robust 3d point matching. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 195–205).

  • Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications ACM, 24(6), 381–395.

    Article  MathSciNet  Google Scholar 

  • Frank, M., & Wolfe, P. (1956). An algorithm for quadratic programming. Naval Research Logistics Quarterly, 3(1–2), 95–110.

    Article  MathSciNet  Google Scholar 

  • Geiger, A., Lenz, P., & Urtasun, R. (2012). Are we ready for autonomous driving? The kitti vision benchmark suite. In IEEE conference on computer vision and pattern recognition (pp. 3354–3361).

  • Goldstein, A. A. (1965). On steepest descent. Journal of the Society for Industrial and Applied Mathematics, Series A: Control, 3(1), 147–151.

    Article  MathSciNet  Google Scholar 

  • Hartley, R., & Zisserman, A. (2003). Multiple view geometry in computer vision. Cambridge University Press.

  • Heinly, J., Schonberger, J. L., Dunn, E., & Frahm, J. M. (2015). Reconstructing the world* in six days*(as captured by the yahoo 100 million image dataset). In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3287–3295).

  • Ivashechkin, M., Barath, D., & Matas, J. (2021). Vsac: Efficient and accurate estimator for h and f. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 15243–15252).

  • Jaggi, M. (2013). Revisiting frank-wolfe: Projection-free sparse convex optimization. In International conference on machine learning (pp. 427–435).

  • Knapitsch, A., Park, J., Zhou, Q. Y., & Koltun, V. (2017). Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions on Graphics (ToG), 36(4), 1–13.

    Article  Google Scholar 

  • Koller, D., Friedman, N., & Bach, F. (2009). Probabilistic graphical models: Principles and techniques. MIT Press.

  • Lebeda, K., Matas, J., Chum, O. (2012). Fixing the locally optimized ransac–full experimental evaluation. In Proceedings of British machine vision conference (pp. 1–11).

  • Leordeanu, M., & Hebert, M. (2005). A spectral technique for correspondence problems using pairwise constraints. In Proc. IEEE Int. Conf. Comput. Vis. (Vol. 2, pp. 1482–1489).

  • Leordeanu, M., Hebert, M., & Sukthankar, R. (2009). An integer projected fixed point method for graph matching and map inference. In Advances in neural information processing systems (pp. 1114–1122).

  • Li, X., & Hu, Z. (2010). Rejecting mismatches by correspondence function. International Journal of Computer Vision, 89(1), 1–17.

    Article  Google Scholar 

  • Lin, W. Y., Wang, F., Cheng, M. M., Yeung, S. K., Torr, P. H., Do, M. N., & Lu, J. (2018). Code: Coherence based decision boundaries for feature correspondence. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(1), 34–47.

    Article  Google Scholar 

  • Liu, H., & Yan, S. (2010). Common visual pattern discovery via spatially coherent correspondences. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 1609–1616).

  • Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.

    Article  Google Scholar 

  • Ma, J., Jiang, X., Fan, A., Jiang, J., & Yan, J. (2021). Image matching from handcrafted to deep features: A survey. International Journal of Computer Vision, 129(1), 23–79.

    Article  MathSciNet  Google Scholar 

  • Ma, J., Wu, J., Zhao, J., Jiang, J., Zhou, H., & Sheng, Q. Z. (2019). Nonrigid point set registration with robust transformation learning under manifold regularization. IEEE Transactions on Neural Networks and Learning Systems, 30(12), 3584–3597.

    Article  MathSciNet  Google Scholar 

  • Ma, J., Zhao, J., Jiang, J., Zhou, H., & Guo, X. (2019). Locality preserving matching. International Journal of Computer Vision, 127(5), 512–531.

    Article  MathSciNet  Google Scholar 

  • Ma, J., Zhao, J., Tian, J., Yuille, A. L., & Tu, Z. (2014). Robust point matching via vector field consensus. IEEE Transactions on Image Processing, 23(4), 1706–1721.

    Article  MathSciNet  Google Scholar 

  • Ma, J., Zhou, H., Zhao, J., Gao, Y., Jiang, J., & Tian, J. (2015). Robust feature matching for remote sensing image registration via locally linear transforming. IEEE Transactions on Geoscience and Remote Sensing, 53(12), 6469–6481.

    Article  Google Scholar 

  • Mishkin, D., Matas, J., & Perdoch, M. (2015). Mods: Fast and robust method for two-view matching. Computer Vision and Image Understanding, 141, 81–93.

    Article  Google Scholar 

  • Nistér, D. (2004). An efficient solution to the five-point relative pose problem. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(6), 756–770.

    Article  Google Scholar 

  • Raguram, R., Chum, O., Pollefeys, M., Matas, J., & Frahm, J. M. (2013). Usac: A universal framework for random sample consensus. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 2022–2038.

    Article  Google Scholar 

  • Rublee, E., Rabaud, V., Konolige, K., & Bradski, G. (2011). Orb: An efficient alternative to sift or surf. In Proceedings of IEEE international conference on computer vision pp. 2564–2571.

  • Sarlin, P. E., DeTone, D., Malisiewicz, T., & Rabinovich, A. (2020). Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 4938–4947).

  • Schonberger, J. L., Frahm, J. M. (2016). Structure-from-motion revisited. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4104–4113).

  • Speciale, P., Paudel, D. P., Oswald, M. R., Riemenschneider, H., Gool, L. V., & Pollefeys, M. (2018). Consensus maximization for semantic region correspondences. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7317–7326).

  • Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D. (2012). A benchmark for the evaluation of rgb-d slam systems. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 573–580).

  • Sun, W., Jiang, W., Trulls, E., Tagliasacchi, A., & Yi, K. M. (2020). Acne: Attentive context normalization for robust permutation-equivariant learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 11286–11295).

  • Thomee, B., Shamma, D. A., Friedland, G., Elizalde, B., Ni, K., Poland, D., Borth, D., & Li, L. J. (2016). Yfcc100m: The new data in multimedia research. Communications of the ACM, 59(2), 64–73.

  • Torr, P. H., & Zisserman, A. (2000). Mlesac: A new robust estimator with application to estimating image geometry. Computer Vision and Image Understanding, 78(1), 138–156.

    Article  Google Scholar 

  • Vongkulbhisal, J., Ugalde, B. I., De la Torre, F., & Costeira, J. P. (2018). Inverse composition discriminative optimization for point cloud registration. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2993–3001).

  • Wang, C., Wang, L., & Liu, L. (2014). Progressive mode-seeking on graphs for sparse feature matching. In European conference on computer vision (pp. 788–802).

  • Wilson, K., & Snavely, N. (2014). Robust global translations with 1dsfm. In Proceedings of the European conference on computer vision (pp. 61–75).

  • Wong, H. S., Chin, T. J., Yu, J., & Suter, D. (2011). Dynamic and hierarchical multi-structure geometric model fitting. In International conference on computer vision (pp. 1044–1051).

  • Yan, J., Cho, M., Zha, H., Yang, X., & Chu, S. M. (2016). Multi-graph matching via affinity optimization with graduated consistency regularization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(6), 1228–1242.

    Article  Google Scholar 

  • Yan, J., Li, C., Li, Y., & Cao, G. (2018). Adaptive discrete hypergraph matching. IEEE Transactions on Cybernetics, 48(2), 765–779.

    Article  Google Scholar 

  • Yi, K. M., Trulls, E., Ono, Y., Lepetit, V., Salzmann, M., & Fua, P. (2018). Learning to find good correspondences. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2666–2674).

  • Zass, R., & Shashua, A. (2008). Probabilistic graph and hypergraph matching. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1–8).

  • Zhang, J., Sun, D., Luo, Z., Yao, A., Zhou, L., Shen, T., Chen, Y., Quan, L., & Liao, H. (2019). Learning two-view correspondences and geometry using order-aware network. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 5845–5854).

  • Zhang, Z. (1998). Determining the epipolar geometry and its uncertainty: A review. International Journal of Computer Vision, 27(2), 161–195.

    Article  Google Scholar 

  • Zhao, C., Cao, Z., Li, C., Li, X., & Yang, J. (2019). Nm-net: Mining reliable neighbors for robust feature correspondences. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 215–224).

  • Zhou, F., & De la Torre, F. (2016). Factorized graph matching. IEEE Conference on Computer Vision and Pattern Recognition, 38(9), 1774–1789.

    Google Scholar 

  • Zhou, L., Zhu, S., Shen, T., Wang, J., Fang, T., & Quan, L. (2017). Progressive large scale-invariant image matching in scale space. In Proceedings of the IEEE international conference on computer vision (pp. 2381–2390).

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant No. 61773295.

Author information

Authors and Affiliations

  1. Electronic Information School, Wuhan University, Wuhan, 430072, China

    Jiayi Ma, Aoxiang Fan & Xingyu Jiang

  2. College of Computer and Control Engineering, Minjiang University, Fuzhou, 350108, China

    Guobao Xiao

Authors
  1. Jiayi Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aoxiang Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xingyu Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guobao Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiayi Ma.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Communicated by V. Lepetit.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, J., Fan, A., Jiang, X. et al. Feature Matching via Motion-Consistency Driven Probabilistic Graphical Model. Int J Comput Vis 130, 2249–2264 (2022). https://doi.org/10.1007/s11263-022-01644-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-022-01644-2

Keywords

Search

Navigation