An Improved Evolutionary Reinforcement Learning Algorithm for UAV Online Target Tracking
Pages 323 - 326
Abstract
Target tracking for unmanned aerial vehicles (UAVs) is significant in a variety of applications and has high research value. Due to the poor capability of responding to deceptive reward signals and lack of diverse exploration, common reinforcement learning (RL) has limitations in real-time decision-making tasks while evolutionary algorithms (EAs) can compensate for these shortcomings by utilizing fitness metrics and strong exploration ability. In this work, we propose an algorithm named improved evolutionary reinforcement learning (IERL) for online UAV target tracking. Firstly, a realistic UAV online target tracking problem is formulated, considering both the velocity change constraints and sparse reward. Based on this problem formulation, an improved selection operator and an improved interaction setting are proposed to achieve higher individual selection efficiency and higher optimization efficiency, respectively. Simulation results show that the proposed algorithm achieves better performance than the comparison algorithm in the sparse reward tracking tasks.
Supplemental Material
PDF File
Supplementary Material
- Download
- 695.49 KB
References
[1]
Paul G Fahlstrom, Thomas J Gleason, and Mohammad H Sadraey. Introduction to UAV systems. John Wiley & Sons, 2022.
[2]
Qiongjian Fan, Fengxian Wang, Xiqiang Shen, and Delin Luo. Path planning for a reconnaissance uav in uncertain environment. In 2016 12th IEEE International Conference on Control and Automation (ICCA), pages 248--252. IEEE, 2016.
[3]
Hamid Menouar, Ismail Guvenc, Kemal Akkaya, A Selcuk Uluagac, Abdullah Kadri, and Adem Tuncer. Uav-enabled intelligent transportation systems for the smart city: Applications and challenges. IEEE Communications Magazine, 55(3):22--28, 2017.
[4]
Jingxuan Hao, Yimin Zhou, Guoshan Zhang, Qin Lv, and Qingtian Wu. A review of target tracking algorithm based on uav. In 2018 IEEE International Conference on Cyborg and Bionic Systems (CBS), pages 328--333. IEEE, 2018.
[5]
Tian Wang, Ruoxi Qin, Yang Chen, Hichem Snoussi, and Chang Choi. A reinforcement learning approach for uav target searching and tracking. Multimedia Tools and Applications, 78:4347--4364, 2019.
[6]
Lihong Liu, Bailing Tian, Xinyi Zhao, and Qun Zong. Uav autonomous trajectory planning in target tracking tasks via a dqn approach. In 2019 IEEE International Conference on Real-time Computing and Robotics (RCAR), pages 277--282, 2019.
[7]
Yintao Zhang, Youmin Zhang, and Ziquan Yu. Path following control for uav using deep reinforcement learning approach. Guidance, Navigation and Control, 01(01):2150005, 2021.
[8]
Somdeb Majumdar, Shauharda Khadka, Santiago Miret, Stephen McAleer, and Kagan Tumer. Evolutionary reinforcement learning for sample-efficient multiagent coordination. In International Conference on Machine Learning, pages 6651--6660. PMLR, 2020.
[9]
L Shi and K Rasheed. A survey of fitness approximation methods applied in evolutionary algorithms. In Computational intelligence in expensive optimization problems, pages 3--28. Springer, 2010.
[10]
Matej Črepinšek, Shih-Hsi Liu, and Marjan Mernik. Exploration and exploitation in evolutionary algorithms: A survey. ACM computing surveys (CSUR), 45(3):1--33, 2013.
[11]
Shauharda Khadka and Kagan Tumer. Evolutionary reinforcement learning. arXiv preprint arXiv:1805.07917, 223, 2018.
[12]
Richard S. Sutton and Andrew G. Barto. Reinforcement learning: An introduction. IEEE Trans. Neural Networks, 9:1054--1054, 1998.
[13]
David B. Fogel. Evolutionary Computation: Toward a New Philosophy of Machine Intelligence. IEEE Press, 1995.
[14]
William Spears, Kenneth De Jong, Thomas Bäck, David Fogel, and Dr. Hugo de GARIS. An overview of evolutionary computation. volume 667, pages 442--459, 04 1993.
Index Terms
- An Improved Evolutionary Reinforcement Learning Algorithm for UAV Online Target Tracking
Recommendations
Adaptive Evolutionary Reinforcement Learning Algorithm with Early Termination Strategy
AAMAS '24: Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent SystemsEvolutionary reinforcement learning algorithms (ERLs), which combine evolutionary algorithms (EAs) with reinforcement learning (RL), have demonstrated significant success in enhancing RL performance. However, most ERLs rely heavily on Gaussian mutation ...
An Efficient Evaluation Mechanism for Evolutionary Reinforcement Learning
Intelligent Computing Theories and ApplicationAbstractIn recent years, many algorithms use Evolutionary Algorithms (EAs) to help Reinforcement Learning (RL) jump out of local optima. Evolutionary Reinforcement Learning (ERL) is a popular algorithm in this field. However, ERL evaluate the population ...
A survey on Evolutionary Reinforcement Learning algorithms
AbstractReinforcement Learning (RL) has proven to be highly effective in various real-world applications. However, in certain scenarios, Evolutionary Algorithms (EAs) have been utilized as an alternative to RL algorithms. Recently, Evolutionary ...
Highlights- This paper provides an extensive survey of ERLs.
- The paper classifies ERLs by their frameworks and evaluates strengths and limitations.
- Various experiments showcase the methods’ performance across diverse environments.
Comments
Information & Contributors
Information
Published In
July 2024
2187 pages
Copyright © 2024 Copyright held by the owner/author(s).
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 01 August 2024
Check for updates
Author Tags
Qualifiers
- Poster
Funding Sources
- National Natural Science Foundation of China
- Guangdong Regional Joint Foundation Key Project
Conference
GECCO '24 Companion
Sponsor:
GECCO '24 Companion: Genetic and Evolutionary Computation Conference Companion
July 14 - 18, 2024
VIC, Melbourne, Australia
Acceptance Rates
Overall Acceptance Rate 1,669 of 4,410 submissions, 38%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 50Total Downloads
- Downloads (Last 12 months)50
- Downloads (Last 6 weeks)7
Reflects downloads up to 01 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in