skip to main content
10.1145/3638682.3638706acmotherconferencesArticle/Chapter ViewAbstractPublication PagesvsipConference Proceedingsconference-collections
research-article

Leveraging SuperGlue and DKM for Deep Learning-Based Robust Image Matching Towards Efficient 3D Reconstruction

Published: 22 May 2024 Publication History

Abstract

Image matching, which is a cornerstone in 3D reconstruction, poses significant and considerable challenges when applied to diverse, unstructured image collections. This paper presents a robust methodology employing SuperGlue and DKM to address these challenges. Initially, EfficientNet is employed to retrieve matching image pairs based on global feature extraction. Subsequently, SuperGlue identifies correspondences between 2D features across these image pairs, while DKM manages large geometric deformations and appearance variations, enhancing the robustness and precision of the matching process. In the Kaggle competition Image Matching Challenge 2023, our approach demonstrated its effectiveness, securing the 34th position among 494 teams and achieving a mean Average Accuracy (mAA) score of 0.470. This achievement not only underscores the potential of our method in real-world applications, particularly in 3D reconstruction from unstructured image collections, but also contributes to the advancement of efficient 3D reconstruction techniques. The implications of this work extend to various applications, including mapping services, cultural heritage preservation, and numerous online services, paving the way for future research in 3D vision tasks and emphasizing the importance of robust image matching techniques in the broader field of computer vision.

References

[1]
J. Cheng, C. Leng, J. Wu, H. Cui, and H. Lu. Fast and Accurate Image Matching with Cascade Hashing for 3D Reconstruction. In CVPR, 2014.
[2]
M. Cao, H. Gao, and W. Jia. Stable Image Matching for 3D Reconstruction in Outdoor. Int. J. Circuit Theory Appl. 2021, 49, 2274-2289.
[3]
K. He, X. Zhang, S. Ren, and J. Sun. Deep Residual Learning for Image Recognition. In CVPR, 2016.
[4]
M. Tan and Q. V. Le. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In ICML, 2019.
[5]
D. DeTone, T. Malisiewicz, and A. Rabinovich. SuperPoint: Self-Supervised Interest Point Detection and Description. In CVPR, 2018.
[6]
A. Dosovitskiy, An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv: 2010.11929.
[7]
P. -E. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich. SuperGlue: Learning Feature Matching with Graph Neural Networks. In CVPR, 2020.
[8]
J. Edstedt, I. Athanasiadis, M. Wadenbäck, and M. Felsberg. DKM: Dense Kernelized Feature Matching for Geometry Estimation. In CVPR, 2023.
[9]
Image Matching Challenge 2023: https://www.kaggle.com/competitions/image-matching-challenge-2023/.
[10]
Gzwwhhh: https://www.kaggle.com/gzwwhhh
[11]
COLMAP: https://colmap.github.io
[12]
J. Zhou, Graph Neural Networks: A Review of Methods and Applications. AI Open 2020, 1, 57-81.
[13]
A. Vaswani, Attention Is All You Need. In NIPS, 2017.
[14]
Z. Li and N. Snavely. MegaDepth: Learning Single-View Depth Prediction from Internet Photos. In CVPR, 2018.
[15]
V. Ailani, D. Prakash, and K. S. Venkatesh. Self Localization with Edge Detection in 3D Space. J. Image Graph., 2013, 1, 99-103.
[16]
L. Tao, An Adaptive Differential Evolution Algorithm with a Point-Based Approach for 3D Point Cloud Registration. J. Image Graph., 2022, 10, 1-9.
[17]
N. A. Hadi, Centroid Based on Branching Contour Matching for 3D Reconstruction Using Beta-Spline. J. Image Graph., 2013, 1, 138-142.
[18]
N. I. A. Abdulqawi and M. S. A. Mansor, A Computer Method for Generating 3D Point Cloud from 2D Digital Image. J. Image Graph., 2016, 4, 89-92.

Index Terms

  1. Leveraging SuperGlue and DKM for Deep Learning-Based Robust Image Matching Towards Efficient 3D Reconstruction

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    VSIP '23: Proceedings of the 2023 5th International Conference on Video, Signal and Image Processing
    November 2023
    237 pages
    ISBN:9798400709272
    DOI:10.1145/3638682
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 22 May 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. 3D reconstruction
    2. DKM
    3. Deep learning
    4. Image matching
    5. SuperGlue

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    VSIP 2023

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 30
      Total Downloads
    • Downloads (Last 12 months)30
    • Downloads (Last 6 weeks)5
    Reflects downloads up to 05 Mar 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media