skip to main content
10.1145/3689095.3689101acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
short-paper

MGAW: An Effective Method for Geo-localization in Adverse Weather

Published: 28 October 2024 Publication History

Abstract

Aerial-view geo-localization involves determining an unknown location by comparing drone-captured images with geo-tagged satellite images. Traditional deep learning-based methods falter under adverse weather conditions like rain and fog due to insufficient adaptation to the domain shift between training scenarios and diverse test environments. To overcome this limitation, we introduce a new data-driven framework designed to adapt to various domain transformations by effectively extracting features from complex environments. Our solution enhances model performance through adaptive data augmentation, achieving third place in the Multimedia Drone Satellite Matching Challenge across multiple environments in the University-160k-WX dataset.

References

[1]
Mayank Bansal, Harpreet S. Sawhney, Hui Cheng, and Kostas Daniilidis. 2011. Geo-localization of street views with aerial image databases. In Proceedings of the 19th ACM International Conference on Multimedia (Scottsdale, Arizona, USA) (MM '11). Association for Computing Machinery, New York, NY, USA, 1125--1128. https://doi.org/10.1145/2072298.2071954
[2]
Francesco Castaldo, Amir Zamir, Roland Angst, Francesco Palmieri, and Silvio Savarese. 2015. Semantic Cross-View Matching. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops.
[3]
Weijian Deng, Liang Zheng, Qixiang Ye, Guoliang Kang, Yi Yang, and Jianbin Jiao. 2018. Image-Image Domain Adaptation With Preserved Self-Similarity and Domain-Dissimilarity for Person Re-Identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[4]
Fabian Deuser, Konrad Habel, and Norbert Oswald. 2023. Sample4Geo: Hard Negative Sampling For Cross-View Geo-Localisation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 16847--16856.
[5]
Yuxin Fang, Quan Sun, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. 2023. EVA-02: A Visual Representation for Neon Genesis. arxiv: 2303.11331 [cs.CV] https://arxiv.org/abs/2303.11331
[6]
Jianping Gou, Baosheng Yu, Stephen J. Maybank, and Dacheng Tao. 2021. Knowledge Distillation: A Survey. International Journal of Computer Vision, Vol. 129, 6 (March 2021), 1789--1819. https://doi.org/10.1007/s11263-021-01453-z
[7]
Weijia Li, Haote Yang, Zhenghao Hu, Juepeng Zheng, Gui-Song Xia, and Conghui He. 2024. 3D Building Reconstruction from Monocular Remote Sensing Images with Multi-level Supervisions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 27728--27737.
[8]
Tsung-Yi Lin, Serge Belongie, and James Hays. 2013. Cross-View Image Geolocalization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[9]
Tsung-Yi Lin, Yin Cui, Serge Belongie, and James Hays. 2015. Learning Deep Representations for Ground-to-Aerial Geolocalization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[10]
Liu Liu and Hongdong Li. 2019. Lending Orientation to Neural Networks for Cross-View Geo-Localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
[11]
Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. 2022. A ConvNet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022).
[12]
Turgay Senlet and Ahmed Elgammal. 2011. A framework for global vehicle localization using stereo images and satellite and road maps. In 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops). 2034--2041. https://doi.org/10.1109/ICCVW.2011.6130498
[13]
Tianrui Shen, Yingmei Wei, Lai Kang, Shanshan Wan, and Yee-Hong Yang. 2024. MCCG: A ConvNeXt-Based Multiple-Classifier Method for Cross-View Geo-Localization. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 34, 3 (2024), 1456--1468. https://doi.org/10.1109/TCSVT.2023.3296074
[14]
Yicong Tian, Chen Chen, and Mubarak Shah. 2017. Cross-View Image Matching for Geo-Localization in Urban Environments. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[15]
Tingyu Wang, Zhedong Zheng, Yaoqi Sun, Chenggang Yan, Yi Yang, and Tat-Seng Chua. 2024. Multiple-environment Self-adaptive Network for Aerial-view Geo-localization. Pattern Recognition, Vol. 152 (2024), 110363.
[16]
Tingyu Wang, Zhedong Zheng, Chenggang Yan, Jiyong Zhang, Yaoqi Sun, Bolun Zheng, and Yi Yang. 2022. Each Part Matters: Local Patterns Facilitate Cross-View Geo-Localization. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 32, 2 (2022), 867--879. https://doi.org/10.1109/TCSVT.2021.3061265
[17]
Tingyu Wang, Zhedong Zheng, Zunjie Zhu, Yuhan Gao, Yi Yang, and Chenggang Yan. 2022. Learning Cross-view Geo-localization Embeddings via Dynamic Weighted Decorrelation Regularization. arxiv: 2211.05296 [cs.CV] https://arxiv.org/abs/2211.05296
[18]
Wenhui Wang, Hangbo Bao, Li Dong, Johan Bjorck, Zhiliang Peng, Qiang Liu, Kriti Aggarwal, Owais Khan Mohammed, Saksham Singhal, Subhojit Som, and Furu Wei. 2022. Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks. arxiv: 2208.10442 [cs.CV] https://arxiv.org/abs/2208.10442
[19]
Scott Workman and Nathan Jacobs. 2015. On the location dependence of convolutional neural network features. In 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 70--78. https://doi.org/10.1109/CVPRW.2015.7301385
[20]
Scott Workman, Richard Souvenir, and Nathan Jacobs. 2015. Wide-Area Image Geolocalization with Aerial Reference Imagery. In 2015 IEEE International Conference on Computer Vision (ICCV). 3961--3969. https://doi.org/10.1109/ICCV.2015.451
[21]
Menghua Zhai, Zachary Bessinger, Scott Workman, and Nathan Jacobs. 2017. Predicting Ground-Level Scene Layout From Aerial Imagery. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[22]
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. 2022. Scaling Vision Transformers. arxiv: 2106.04560 [cs.CV] https://arxiv.org/abs/2106.04560
[23]
Zhedong Zheng, Yujiao Shi, Tingyu Wang, Chen Chen, Pengfei Zhu, and Richard Hartley. 2024. The 2nd Workshop on UAVs in Multimedia: Capturing the World from a New Perspective. In Proceedings of the 32nd ACM International Conference on Multimedia Workshop.
[24]
Zhedong Zheng, Yunchao Wei, and Yi Yang. 2020. University-1652: A Multi-view Multi-source Benchmark for Drone-based Geo-localization. In Proceedings of the 28th ACM International Conference on Multimedia (Seattle, WA, USA) (MM '20). Association for Computing Machinery, New York, NY, USA, 1395--1403.
[25]
Zhedong Zheng, Liang Zheng, Michael Garrett, Yi Yang, Mingliang Xu, and Yi-Dong Shen. 2020. Dual-path Convolutional Image-Text Embeddings with Instance Loss. ACM Transactions on Multimedia Computing, Communications, and Applications, Vol. 16, 2 (May 2020), 1--23. https://doi.org/10.1145/3383184
[26]
Zhedong Zheng, Liang Zheng, and Yi Yang. 2017. A Discriminatively Learned CNN Embedding for Person Reidentification. ACM Trans. Multimedia Comput. Commun. Appl., Vol. 14, 1, Article 13 (dec 2017), 20 pages. https://doi.org/10.1145/3159171
[27]
Pengfei Zhu, Longyin Wen, Xiao Bian, Haibin Ling, and Qinghua Hu. 2018. Vision Meets Drones: A Challenge. arxiv: 1804.07437 [cs.CV]
[28]
Runzhe Zhu, Mingze Yang, Kaiyu Zhang, Fei Wu, Ling Yin, and Yujin Zhang. 2023. Modern Backbone for Efficient Geo-localization. In Proceedings of the 2023 Workshop on UAVs in Multimedia: Capturing the World from a New Perspective (Ottawa ON, Canada) (UAVM '23). Association for Computing Machinery, New York, NY, USA, 31--37. https://doi.org/10.1145/3607834.3616562
[29]
Runzhe Zhu, Ling Yin, Mingze Yang, Fei Wu, Yuncheng Yang, and Wenbo Hu. 2023. SUES-200: A Multi-Height Multi-Scene Cross-View Image Benchmark Across Drone and Satellite. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 33, 9 (2023), 4825--4839. https://doi.org/10.1109/TCSVT.2023.3249204
[30]
Sijie Zhu, Mubarak Shah, and Chen Chen. 2022. TransGeo: Transformer Is All You Need for Cross-View Image Geo-Localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 1162--1171.
[31]
Sijie Zhu, Taojiannan Yang, and Chen Chen. 2021. VIGOR: Cross-View Image Geo-Localization Beyond One-to-One Retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 3640--3649.

Index Terms

  1. MGAW: An Effective Method for Geo-localization in Adverse Weather

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    UAVM '24: Proceedings of the 2nd Workshop on UAVs in Multimedia: Capturing the World from a New Perspective
    October 2024
    41 pages
    ISBN:9798400712067
    DOI:10.1145/3689095
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 28 October 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. cross-view geo-localization
    2. deep learning
    3. image retrieval
    4. multi- source domain generalization
    5. multi-platform collaboration

    Qualifiers

    • Short-paper

    Funding Sources

    • National Natural Science Foundation of China (NSFC)

    Conference

    MM '24
    Sponsor:
    MM '24: The 32nd ACM International Conference on Multimedia
    October 28 - November 1, 2024
    Melbourne VIC, Australia

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 48
      Total Downloads
    • Downloads (Last 12 months)48
    • Downloads (Last 6 weeks)7
    Reflects downloads up to 16 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media