Skip to main content
SpringerLink
Log in

Self-adaptive SURF for image-to-video matching

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Speeded up robust feature (SURF) is one of the most popular feature-based algorithms handling image matching. Compared to emerging deep learning neural network-based image matching algorithms, SURF is much faster with comparable accuracy. Currently, it is still one of the dominant algorithms adopted in majority of real-time applications. With the increasing popularity of video-based computer vision applications, image matching between an image and different frames of a video stream is required. Traditional algorithms could fail to deal with live video because spatiotemporal differences between frames could cause significant fluctuation in the results. In this study, we propose a self-adaptive methodology to improve the stability and precision of image–video matching. The proposed methodology dynamically adjusts threshold in feature points extraction to control the number of extracted feature points based on the content of the previous frame. Minimum ratio of distance (MROD) matching is integrated to preclude false matches while keeping abundant sample sizes. Finally, multiple homography matrix (H-Matrix) are estimated using progressive sample consensus (PROSAC) with various reprojection errors. The model with lowest mean square error (MSE) will be selected for image-to-video frame matching. The experimental results show that the self-adaptive SURF offers more accurate and stable results while balancing single frame processing time in image-video matching.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Availability of data and materials

Available upon request.

References

  1. Shapiro, L.S., Michael Brady, J.: Feature-based correspondence: an eigenvector approach. Image Vis. Comput. 10(5), 283–288 (1992). https://doi.org/10.1016/0262-8856(92)90043-3

    Article  Google Scholar 

  2. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004). https://doi.org/10.1023/B:VISI.0000029664.99615.94

    Article  Google Scholar 

  3. Bay, H., Tuytelaars, T., Van Gool, L.: SURF: speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) Computer vision—ECCV 2006, vol. 3951, pp. 404–417. Springer, Berlin, Heidelberg (2006)

    Chapter  Google Scholar 

  4. Li, Z., Dong, Z., Yu, A., He, Z., Zhu, X., et al.: A robust image sequence registration algorithm for videosar combining surf with inter-frame processing. IEEE Conference Publications (2020). https://ieeexplore.ieee.org/document/8899848

  5. Zhang, Y., Yang, L., Wang, Z., et al.: Research on video image stitching technology based on SURF (2020). https://ieeexplore.ieee.org/document/6406008

  6. Sarlin, P.-E., DeTone, D., Malisiewicz T., Rabinovich, A.: SuperGlue: learning feature matching with graph neural networks. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020). https://doi.org/10.1109/CVPR42600.2020.00499. https://ieeexplore.ieee.org/document/9157489. Accessed 01 Jan 2023

  7. Sun, J., Shen, Z., Wang, Y., Bao, H., Zhou, X.: LoFTR: detector-free local feature matching with transformers. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021). https://doi.org/10.1109/CVPR46437.2021.00881. https://ieeexplore.ieee.org/document/9578008. Accessed 01 Jan 2023

  8. Cantzler, H.: Random sample consensus (RANSAC). Institute for Perception, Action and Behaviour, Division of Informatics, Univ. of Edinburgh (1981). http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.106.3035&rep=rep1&type=pdf

  9. Chum, O., Matas, J.: Matching with PROSAC—progressive sample consensus. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA, 2005, vol. 1, pp. 220–226. https://doi.org/10.1109/CVPR.2005.221

Download references

Funding

No funding was received to assist with the preparation of this manuscript.

Author information

Authors and Affiliations

  1. College of Computing and Software Engineering, Kennesaw State University, Marietta, GA, 30060, USA

    Ming Yang, Jiaming Li & Zhigang Li

  2. Blue Vision Systems LLC, Alpharetta, GA, 30005, USA

    Wen Li

  3. School of Art and Science, Rutgers, The State University of New Jersey, New Brunswick, NJ, 08901, USA

    Kairui Zhang

Authors
  1. Ming Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiaming Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhigang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kairui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Authors’ contributions are in the same order as the order of author list. Dr. Ming Yang proposed the idea and directed the research. Mr. Jiaming Li implemented the experiments and collected the results. Dr. Zhigang Li helped with data collection/analysis and paper writing. Mr. Wen Li helped with the implementation and results collection. Mr. Kairui Zhang helped with the implementation and results collection. All authors reviewed the manuscript.

Corresponding author

Correspondence to Ming Yang.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests;

Ethical approval

Not applicable;

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, M., Li, J., Li, Z. et al. Self-adaptive SURF for image-to-video matching. SIViP 18, 751–759 (2024). https://doi.org/10.1007/s11760-023-02802-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02802-w

Keywords

Search

Navigation