skip to main content
10.1145/3652628.3652743acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicaiceConference Proceedingsconference-collections
research-article

MCL-Geo: Multi-branch Contrastive Learning for Cross-view Geo-localizationA multi-branch contrastive learning framework plus uniform cross-view contrastive loss for cross-view geo-localization targets.

Published: 23 May 2024 Publication History

Abstract

Cross-view geo-localization aims to match images of the same location taken from different platforms, such as drones and satellites. However, the altitude differences between the perspectives of these platforms can lead to irregularities in the appearance of the target object. To obtain cross-view invariant features, existing methods pair samples from different perspectives and use contrastive learning between analogous samples. However, these methods are influenced by the backbone, while overlooking more distinctive features. Therefore, in order to effectively learn the consistency information between images across views, we propose a multi-branch framework combine a uniform contrastive loss function which can simultaneously mine the consistency information between multiple samples. In addition, we propose a novelty special designed data processing strategy to process the input of contrastive learning model, which can prompt the model learning fine-grained invariant details about the target between different perspectives. Experiments on widely used public benchmarks show that our proposed method achieves superior performance with fewer parameter models.

References

[1]
T. Shen, Y. Wei, L. Kang, S. Wan and Y. -H. Yang, "MCCG: A ConvNeXt-based Multiple-Classifier Method for Cross-view Geo-localization," in IEEE Transactions on Circuits and Systems for Video Technology. 2023. 3296074.
[2]
Deuser F, Habel K, Oswald N. Sample4Geo: Hard Negative Sampling For Cross-View Geo-Localization[J]. arXiv preprint arXiv:2303.11851, 2023.
[3]
M. Zhai, Z. Bessinger, S. Workman, and N. Jacobs, “Predicting ground-level scene layout from aerial imagery,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 867–875, 2017. 2, 5, 6
[4]
L. Liu and H. Li, “Lending orientation to neural networks for cross-view geo-localization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5624–5633, 2019. 2, 5, 6
[5]
Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A convnet for the 2020s,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11976–11986, 2022. 4, 7
[6]
Z. Zheng, Y. Wei, and Y. Yang, “University-1652: A multi-view multi-source benchmark for drone-based geo-localization,” in Proceedings of the 28th ACM international conference on Multimedia, pp. 1395–1403, 2020. 2, 3, 5, 6
[7]
R. Zhu, L. Yin, M. Yang, F. Wu, Y. Yang and W. Hu, "SUES-200: A Multi-Height Multi-Scene Cross-View Image Benchmark Across Drone and Satellite," in IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 9, pp. 4825-4839, Sept. 2023.
[8]
S. Workman, R. Souvenir, and N. Jacobs, “Wide-area image geolocalization with aerial reference imagery,” in 2015 IEEE International Conference on Computer Vision (ICCV), pp. 3961–3969, 2015. 2, 5
[9]
N. Jacobs, K. Miskell, and R. Pless, “Webcam geolocalization using aggregate light levels,” in 2011 IEEE Workshop on Applications of Computer Vision (WACV), pp. 132–138, 2011. 1
[10]
T. Wang, Z. Zheng, C. Yan, J. Zhang, Y. Sun, B. Zheng, and Y. Yang, “Each part matters: Local patterns facilitate crossview geo-localization,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 2, pp. 867–879, 2021. 2, 6
[11]
T.-Y. Lin, Y. Cui, S. Belongie, and J. Hays, “Learning deep representations for ground-to-aerial geolocalization,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 5007–5015
[12]
S. Hu, M. Feng, R. M. Nguyen, and G. H. Lee, “Cvm-net: Cross-view matching network for image-based ground-to-aerial geo-localization,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7258–7267.
[13]
S. Zhu, M. Shah, and C. Chen, “Transgeo: Transformer is all you need for cross-view image geo-localization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 1162–1171.
[14]
Wang T, Fan S, Liu D, Transformer-guided convolutional neural network for cross-view geolocalization[J]. arXiv preprint arXiv:2204.09967, 2022.
[15]
Loshchilov I, Hutter F. Decoupled weight decay regularization[J]. arXiv preprint arXiv:1711.05101, 2017.
[16]
Ding L, Zhou J, Meng L, A Practical Cross-View Image Matching Method between DRONE and Satellite for DRONE-Based Geo-Localization[J]. Remote Sensing, 2021, 13(1): 47.
[17]
Wang T, Zheng Z, Yan C, Each part matters: Local patterns facilitate cross-view geo-localization[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 32(2): 867-879.
[18]
Tian X, Shao J, Ouyang D, UAV-satellite view synthesis for cross-view geo-localization[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 32(7): 4804-4815.
[19]
Lin J, Zheng Z, Zhong Z, Joint representation learning and keypoint detection for cross-view geo-localization[J]. IEEE Transactions on Image Processing, 2022, 31: 3780-3792.
[20]
Wang T, Zheng Z, Zhu Z, Learning cross-view geo-localization embeddings via dynamic weighted decorrelation regularization[J]. arXiv preprint arXiv:2211.05296, 2022.
[21]
Lu Z, Pu T, Chen T, Content-Aware Hierarchical Representation Selection for Cross-View Geo-Localization[C]//Proceedings of the Asian Conference on Computer Vision. 2022: 4211-4224.
[22]
Hu Q, Li W, Xu X, Learning discriminative representations via variational self-distillation for cross-view geo-localization[J]. Computers and Electrical Engineering, 2022, 103: 108335.
[23]
M. Dai, J. Hu, J. Zhuang and E. Zheng, "A Transformer-Based Feature Segmentation and Region Alignment Method for UAV-View Geo-Localization," in IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 7, pp. 4376-4389, July 2022.
[24]
Bui D V, Kubo M, Sato H. A Part-aware Attention Neural Network for Cross-view Geo-localization between UAV and Satellite[J]. Journal of Robotics, Networking and Artificial Life, 2022, 9(3): 275-284.
[25]
Zhu Y, Yang H, Lu Y, Simple, Effective and General: A New Backbone for Cross-view Image Geo-localization[J]. arXiv preprint arXiv:2302.01572, 2023.
[26]
Zhu R, Yang M, Yin L, Uav's status is worth considering: A fusion representations matching method for geo-localization[J]. Sensors, 2023, 23(2): 720.
[27]
Ramaswamy H G. Ablation-cam: Visual explanations for deep convolutional network via gradient-free localization[C]//proceedings of the IEEE/CVF winter conference on applications of computer vision. 2020: 983-991.

Index Terms

  1. MCL-Geo: Multi-branch Contrastive Learning for Cross-view Geo-localizationA multi-branch contrastive learning framework plus uniform cross-view contrastive loss for cross-view geo-localization targets.

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICAICE '23: Proceedings of the 4th International Conference on Artificial Intelligence and Computer Engineering
    November 2023
    1263 pages
    ISBN:9798400708831
    DOI:10.1145/3652628
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 23 May 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    ICAICE 2023

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 56
      Total Downloads
    • Downloads (Last 12 months)56
    • Downloads (Last 6 weeks)12
    Reflects downloads up to 28 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media