Qibin He
ABSTRACT
The ability to autonomously and precisely locate unmanned aerial vehicles (UAVs) is critical to successfully operate in complex and challenging environments. This paper addresses the challenge of location determination for UAVs in scenarios where GPS signals are weak or unavailable. The proposed solution introduces a novel multi-height localization system, leveraging the power of Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNNs) to process visual data captured by a UAV’s onboard camera. By analyzing visual information, this system enables UAVs to determine their positions at various altitudes accurately. When GPS signals are unreliable or obstructed, the proposed method offers a robust alternative, enhancing the overall reliability and autonomy of UAV missions. Experimental results demonstrate the real-time effectiveness of our multi-height localization system, showcasing its capability to accurately determine UAV locations at different altitudes.
- Seongha Ahn, Hosun Kang, and Jangmyung Lee. 2021. Aerial-Satellite Image Matching Framework for UAV Absolute Visual Localization using Contrastive Learning. In 2021 21st International Conference on Control, Automation and Systems (ICCAS). IEEE, 143–146. https://ieeexplore.ieee.org/abstract/document/9649999/Google Scholar
Digital Library
- Rodion O. Anisimov, Nikolay V. Goloburdin, Konstantin A. Kulagin, Tatiana Ya Gladkikh, and Yuri D. Vorobiev. 2022. Visual Localization System Algorithm for UAV. In 2022 International Conference on Information, Control, and Communication Technologies (ICCT). IEEE, 1–5. https://ieeexplore.ieee.org/abstract/document/9976692/Google Scholar
- Yuqing Cao, Kan Ren, and Qian Chen. 2023. Template matching based on convolution neural network for UAV visual localization. Optik 283 (2023), 170920. https://www.sciencedirect.com/science/article/pii/S0030402623004163Google ScholarCross Ref
- Andy Couturier and Moulay Akhloufi. 2020. Conditional probabilistic relative visual localization for unmanned aerial vehicles. In 2020 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE). IEEE, 1–4. https://ieeexplore.ieee.org/abstract/document/9255691/Google ScholarCross Ref
- Andy Couturier and Moulay A. Akhloufi. 2018. Relative visual localization (RVL) for UAV navigation. In Degraded Environments: Sensing, Processing, and Display 2018, Vol. 10642. SPIE, 213–226. https://www.spiedigitallibrary.org/conference-proceedings-of-spie/10642/106420O/Relative-visual-localization-RVL-for-UAV-navigation/10.1117/12.2304901.shortGoogle Scholar
- Andy Couturier and Moulay A. Akhloufi. 2021. A review on absolute visual localization for UAV. Robotics and Autonomous Systems 135 (2021), 103666. https://www.sciencedirect.com/science/article/pii/S0921889020305066Google ScholarCross Ref
- Jan Dufek and Robin Murphy. 2016. Visual pose estimation of USV from UAV to assist drowning victims recovery. In 2016 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR). IEEE, 147–153. https://ieeexplore.ieee.org/abstract/document/7784291/Google ScholarDigital Library
- Christoforos Kanellakis and George Nikolakopoulos. 2016. Evaluation of visual localization systems in underground mining. In 2016 24th Mediterranean Conference on Control and Automation (MED). IEEE, 539–544. https://ieeexplore.ieee.org/abstract/document/7535853/Google ScholarCross Ref
- Jouko Kinnari, Riccardo Renzulli, Francesco Verdoja, and Ville Kyrki. 2023. LSVL: Large-scale season-invariant visual localization for UAVs. Robotics and Autonomous Systems 168 (2023), 104497. https://www.sciencedirect.com/science/article/pii/S0921889023001367Google ScholarDigital Library
- Jouko Kinnari, Francesco Verdoja, and Ville Kyrki. 2022. Season-invariant gnss-denied visual localization for uavs. IEEE Robotics and Automation Letters 7, 4 (2022), 10232–10239. https://ieeexplore.ieee.org/abstract/document/9830867/Google ScholarCross Ref
- I. A. Kunina, L. M. Teplyakov, A. P. Gladkov, Timur M. Khanipov, and Dmitry P. Nikolaev. 2018. Aerial images visual localization on a vector map using color-texture segmentation. In Tenth International Conference on Machine Vision (ICMV 2017), Vol. 10696. SPIE, 465–472. https://www.spiedigitallibrary.org/conference-proceedings-of-spie/10696/106961T/Aerial-images-visual-localization-on-a-vector-map-using-color/10.1117/12.2310138.shortGoogle Scholar
- Chengyao Li and Steven L. Waslander. 2019. Visual measurement integrity monitoring for uav localization. In 2019 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR). IEEE, 22–29. https://ieeexplore.ieee.org/abstract/document/8848975/Google ScholarDigital Library
- Vincenzo Lippiello and Jonathan Cacace. 2021. Robust visual localization of a uav over a pipe-rack based on the lie group se (3). IEEE Robotics and Automation Letters 7, 1 (2021), 295–302. https://ieeexplore.ieee.org/abstract/document/9601262/Google ScholarCross Ref
- Yuncheng Lu, Zhucun Xue, Gui-Song Xia, and Liangpei Zhang. 2018. A survey on vision-based UAV navigation. Geo-spatial Information Science 21, 1 (Jan. 2018), 21–32. https://doi.org/10.1080/10095020.2017.1420509Google ScholarCross Ref
- Liqun Ma, Dongyuan Meng, Shuaihe Zhao, and Binbin An. 2023. Visual localization with a monocular camera for unmanned aerial vehicle based on landmark detection and tracking using YOLOv5 and DeepSORT. International Journal of Advanced Robotic Systems 20, 3 (May 2023), 172988062311648. https://doi.org/10.1177/17298806231164831Google ScholarCross Ref
- Hamza Merzi, Elena Stumm, Marcin Dymczyk, Roland Siegwart, and Igor Gilitschenski. 2017. Map quality evaluation for visual localization. In 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 3200–3206. https://ieeexplore.ieee.org/abstract/document/7989363/Google ScholarDigital Library
- Wided Souidene Mseddi, Mohamed Ali Sedrine, and Rabah Attia. 2021. YOLOv5 based visual localization for autonomous vehicles. In 2021 29th European Signal Processing Conference (EUSIPCO). IEEE, 746–750. https://ieeexplore.ieee.org/abstract/document/9616354/Google ScholarCross Ref
- Raul Mur-Artal, Jose Maria Martinez Montiel, and Juan D. Tardos. 2015. ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE transactions on robotics 31, 5 (2015), 1147–1163. https://ieeexplore.ieee.org/abstract/document/7219438/Google Scholar
- Bhavit Patel. 2019. Visual Localization for UAVs in Outdoor GPS-denied Environments. University of Toronto (Canada). https://search.proquest.com/openview/ff54ad672f25af4497a2176ba06f47a1/1?pq-origsite=gscholar&cbl=18750&diss=yGoogle Scholar
- Bhavit Patel, Timothy D. Barfoot, and Angela P. Schoellig. 2020. Visual localization with google earth images for robust global pose estimation of uavs. In 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 6491–6497. https://ieeexplore.ieee.org/abstract/document/9196606/Google ScholarCross Ref
- Bhavit Patel, Michael Warren, and Angela Schoellig. 2019. Point me in the right direction: Improving visual localization on UAVs with active gimballed camera pointing. In 2019 16th Conference on Computer and Robot Vision (CRV). IEEE, 105–112. https://ieeexplore.ieee.org/abstract/document/8781627/Google ScholarCross Ref
- Eduardo Perdices and José María Cañas. 2019. SDVL: Efficient and accurate semi-direct visual localization. Sensors 19, 2 (2019), 302. https://www.mdpi.com/1424-8220/19/2/302Google ScholarCross Ref
- Jae Hong Shim and Young Im Cho. 2017. A visual localization technique for unmanned ground and aerial robots. In 2017 First IEEE International Conference on Robotic Computing (IRC). IEEE, 399–403. https://ieeexplore.ieee.org/abstract/document/7926572/Google ScholarCross Ref
- Haoyang Tang, Jiakun Shi, Xin Miao, Ruichen Wu, and Dongfang Yang. 2022. UAV Visual Localization Technology Based on Heterogenous Remote Sensing Image Matching. In Proceedings of the 2022 5th International Conference on Artificial Intelligence and Pattern Recognition. ACM, 773–779. https://doi.org/10.1145/3573942.3574094 Place: Xiamen China.Google ScholarDigital Library
- Kailin Wen, Jie Chu, Jiayan Chen, Yu Chen, and Jueping Cai. 2022. MO SiamRPN with Weight Adaptive Joint MIoU for UAV Visual Localization. Remote Sensing 14, 18 (2022), 4467. https://www.mdpi.com/2072-4292/14/18/4467Google ScholarCross Ref
- Bayang Xue, Zhong Yang, Luwei Liao, Chi Zhang, Hao Xu, and Qiuyan Zhang. 2022. High precision visual localization method of UAV based on feature matching. Frontiers in Computational Neuroscience 16 (2022), 1037623. https://www.frontiersin.org/articles/10.3389/fncom.2022.1037623/fullGoogle ScholarCross Ref
Index Terms
- Multi-height Visual Drone Positioning Based on LSTM and Convolutional Neural Networks
Recommendations
Visual Track System Applied in Quadrotor Aerial Robot
ICDMA '12: Proceedings of the 2012 Third International Conference on Digital Manufacturing & AutomationUnmanned aerial vehicle (UAV) had developed for many years, with quad-rotor vertical take-off and landing (VTOL) unmanned aerial vehicle developing fastest. The equations of its flight attitude have been derived and verified by some scholars. Up to now, ...
Adaptive vision-based system for landing an autonomous hexacopter drone on a specific landing platform
Today, autonomous flight and precise landing of drones are crucial for many applications such as object detection, and delivery of services. The focus of this research is to present an adaptive vision-based system for landing an autonomous hexacopter ...
Towards Autonomous Autorotation Landing for Small Size Unmanned Helicopters
The consideration of safety issues in the operation of helicopters involves the capability to perform emergency descending maneuvers through the autorotation principle when the engine is no longer supplying power. When comparing manned and unmanned ...
Comments