skip to main content
10.1145/3689095.3689098acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
short-paper

WELN: Siamese Network-based Framework for Geo-localization in Extreme Weather

Published: 28 October 2024 Publication History

Abstract

Cross-view geo-localization is a task of matching the same geographic image from differerent views, e.g., drone and satellite. Due to its GPS-free advantage, cross-view geo-localization is gaining increasing research interest, especially in drone-based localization and navigation applications. In order to guarantee system robustness, existing methods mainly focused on image augmentation and denoising, while facing performace degradation when extreme weather considered. In this paper, we propose an end-to-end image retrieval framework, WELN. By integrating the advanced EVA02 netwotk and LPN algorithm, WELN can extract valuable classification features more efficiently even under extreme weather conditions. Additionally, to enhance model robustness, we expand the University-1652 dataset with nine different weather conditions added. Our method achieves state-of-the-art Recall@1 accuracy on University-1652 dataset, with 92.87% for drone-view target localization task and 93.46% for drone navigation task. Besides, we gain the fourth place in the ACMMM24 Multimedia Drone-Satellite Matching Challenge. Our code will be open sourced at https://github.com/koorter/WELN.

References

[1]
Hongji Yang, Xiufan Lu, and Ying J. Zhu. 2021. Cross-view Geo-localization with Layer-to-Layer Transformer. Neural Information Processing Systems.
[2]
Niluthpol Chowdhury Mithun, Kshitij Minhas, Han-Pang Chiu, Taragay Oskiper, Mikhail Sizintsev, Supun Samarasekera, and Rakesh Kumar. 2023. Cross-View Visual Geo-Localization for Outdoor Augmented Reality. 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR) (2023), 493--502.
[3]
Wen Chen, Chenglong Zhang, Yu Peng, Yao Yao, Miaomiao Cai, and Danan Dong. 2024. Enhancing GNSS Positioning in Urban Canyon Areas via a Modified Design Matrix Approach. IEEE Internet of Things Journal, Vol. 11, 6 (2024), 10252--10265.
[4]
Mu Chen, Zhedong Zheng, and Yi Yang. 2023. Transferring to Real-World Layouts: A Depth-aware Framework for Scene Adaptation. arxiv: 2311.12682 [cs.CV]
[5]
Zhedong Zheng, Yujiao Shi, Tingyu Wang, Chen Chen, Pengfei Zhu, and Richard Hartley. 2024. The 2nd Workshop on UAVs in Multimedia: Capturing the World from a New Perspective. In Proceedings of the 32nd ACM International Conference on Multimedia Workshop.
[6]
Depeng Li and Zhigang Zeng. 2023. CRNet: A Fast Continual Learning Framework With Random Theory. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 45, 9 (2023), 10731--10744.
[7]
Yuning Cui and Alois Knoll. 2024. Dual-domain strip attention for image restoration. Neural Networks, Vol. 171 (2024), 429--439.
[8]
Yuxin Fang, Quan Sun, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. 2023. EVA-02: A Visual Representation for Neon Genesis. ArXiv, Vol. abs/2303.11331 (2023).
[9]
Tingyu Wang, Zhedong Zheng, Chenggang Yan, Jiyong Zhang, Yaoqi Sun, Bolun Zheng, and Yi Yang. 2022. Each Part Matters: Local Patterns Facilitate Cross-View Geo-Localization. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 32, 2 (2022), 867--879.
[10]
Zhedong Zheng, Yunchao Wei, and Yi Yang. 2020. University-1652: A Multi-view Multi-source Benchmark for Drone-based Geo-localization. Association for Computing Machinery.
[11]
Tingyu Wang, Zhedong Zheng, Zunjie Zhu, Yaoqi Sun, Yi Yang, and Chenggang Yan. 2022. Learning cross-view geo-localization embeddings via dynamic weighted decorrelation regularization. arXiv preprint arXiv:2211.05296 (2022).
[12]
Sijie Zhu, Mubarak Shah, and Chen Chen. 2022. TransGeo: Transformer Is All You Need for Cross-view Image Geo-localization. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022), 1152--1161.
[13]
Yingying Zhu, Hongji Yang, Yuxin Lu, and Qiang Huang. 2023. Simple, Effective and General: A New Backbone for Cross-view Image Geo-localization. ArXiv, Vol. abs/2302.01572 (2023).
[14]
Yicong Tian, Chen Chen, and Mubarak Shah. 2017. Cross-View Image Matching for Geo-Localization in Urban Environments. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1998--2006.
[15]
Mu Chen, Zhedong Zheng, Yi Yang, and Tat-Seng Chua. 2022. PiPa: Pixel- and Patch-wise Self-supervised Learning for Domain Adaptative Semantic Segmentation. arxiv: 2211.07609 [cs.CV]
[16]
Tingyu Wang, Zhedong Zheng, Yaoqi Sun, Chenggang Yan, Yi Yang, and Tat-Seng Chua. 2024. Multiple-environment Self-adaptive Network for aerial-view geo-localization. Pattern Recognition, Vol. 152 (2024), 110363.
[17]
Wei-Ting Chen, Zhi-Kai Huang, Cheng-Che Tsai, Hao-Hsiang Yang, Jian-Jiun Ding, and Sy-Yen Kuo. 2022. Learning Multiple Adverse Weather Removal via Two-stage Knowledge Learning and Multi-contrastive Regularization: Toward a Unified Model. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 17632--17641.
[18]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2020. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. ArXiv, Vol. abs/2010.11929 (2020).
[19]
Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. 2023. EVA: Exploring the Limits of Masked Visual Representation Learning at Scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 19358--19369.
[20]
Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean. 2015. Distilling the Knowledge in a Neural Network. In NIPS Deep Learning and Representation Learning Workshop.
[21]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the 38th International Conference on Machine Learning, Vol. 139. 8748--8763.
[22]
Hongxiang Lv, Hai Zhu, Runzhe Zhu, Fei Wu, Chunyuan Wang, Meiyu Cai, and Kaiyu Zhang. 2024. Direction-Guided Multiscale Feature Fusion Network for Geo-Localization. IEEE Transactions on Geoscience and Remote Sensing, Vol. 62 (2024), 1--13.
[23]
Yireng Chen, Zihao Yang, and Quan Chen. 2023. A Cross-View Matching Method Based on Dense Partition Strategy for UAV Geolocalization. In Proceedings of the 31st ACM International Conference on Multimedia Workshop (UAVM '23). Association for Computing Machinery, 19--23.
[24]
Leyi Dong, Yuhui Wang, Junshi Huang, Xueming Qian, Mingyuan Fan, and Shenqi Lai. 2023. Dual Path Network for Cross-view Geo-Localization. In Proceedings of the 31st ACM International Conference on Multimedia Workshop (UAVM '23). Association for Computing Machinery, 45--49.
[25]
Ming Dai, Jianhong Hu, Jiedong Zhuang, and Enhui Zheng. 2022. A Transformer-Based Feature Segmentation and Region Alignment Method for UAV-View Geo-Localization. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 32, 7 (2022), 4376--4389.
[26]
Duc Viet Bui, Masao Kubo, and Hiroshi Sato. 2022. A Part-aware Attention Neural Network for Cross-view Geo-localization between UAV and Satellite. Journal of Robotics, Networking and Artificial Life, Vol. 9, 3 (2022), 275--284.
[27]
Runzhe Zhu, Mingze Yang, Ling Yin, Fei Wu, and Yuncheng Yang. 2023. UAV's Status Is Worth Considering: A Fusion Representations Matching Method for Geo-Localization. Sensors, Vol. 23, 2 (2023).
[28]
Tianrui Shen, Yingmei Wei, Lai Kang, Shanshan Wan, and Yee-Hong Yang. 2024. MCCG: A ConvNeXt-Based Multiple-Classifier Method for Cross-View Geo-Localization. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 34, 3 (2024), 1456--1468.

Index Terms

  1. WELN: Siamese Network-based Framework for Geo-localization in Extreme Weather

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    UAVM '24: Proceedings of the 2nd Workshop on UAVs in Multimedia: Capturing the World from a New Perspective
    October 2024
    41 pages
    ISBN:9798400712067
    DOI:10.1145/3689095
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 28 October 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. drone-satellite matching
    2. features
    3. geo-localization
    4. weather noise

    Qualifiers

    • Short-paper

    Funding Sources

    Conference

    MM '24
    Sponsor:
    MM '24: The 32nd ACM International Conference on Multimedia
    October 28 - November 1, 2024
    Melbourne VIC, Australia

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 62
      Total Downloads
    • Downloads (Last 12 months)62
    • Downloads (Last 6 weeks)10
    Reflects downloads up to 25 Jan 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media