Yifei Wu
ABSTRACT
With the Artificial Intelligence (AI) widely used in air combat simulation system, the decision-making system of fighter has reached a high level of complexity. Traditionally, the pure theoretical analysis and the rule-based system are not enough to represent the cognitive behavior of pilots. In order to properly specify the autonomous decision-making of fighter, hence, we proposed a unified framework which combines the combat simulation and machine learning in this paper. This framework adopts deep reinforcement learning modelling by using the supervised learning and the Deep Q-Network (DQN) methods. As a proof of concept, we built an autonomous decision-making training scenario based on the Weapon Effectiveness Simulation System (WESS). The simulation results show that the intelligent decision-making model based on the proposed framework has better combat effects than the traditional decision-making model based on knowledge engineering.
- GAO Zhen-yang, QIN Bin. Progress in Deep Reinforcement Learning research[J]. Computer knowledge and technology, 2019,15(04): 163-165+179.Google Scholar
- B Xue, Y He, F Jing, Robot Target Recognition using Deep Federated Learning [J]. International Journal of Intelligent Systems, 2021, 36(12): 7754-7769.Google ScholarCross Ref
- Lu Y, Xu X, Zhang X, Hierarchical Reinforcement Learning for Autonomous Decision Making and Motion Planning of Intelligent Vehicles[J]. IEEE Access, 2020, PP(99):1-1.Google ScholarCross Ref
- Peng X, Chen R, Zhang J, Enhanced Autonomous Navigation of Robots by Deep Reinforcement Learning Algorithm with Multistep Method[J]. Sensors and Materials, 2021, 33(2):825.Google ScholarCross Ref
- ZUO Jialiang, YANG Rennong, ZHANG Ying, LI Zhonglin, WU Meng. Intelligent decision-making in air combat maneuvering based on heuristic reinforcement learning[J]. Acta Aeronautica et Astronautica Sinica, 2017(10):217-230.Google Scholar
- LI ANG ZHANG, JIA XU, DARA GOLD, JEFF HAGEN, AJAY K. KOCHHAR, ANDREW J. LOHN, OSONDE A. OSOBA. Air Dominance Through Machine Learning.Google Scholar
- Li K, Zhang K, Zhang Z, A UAV Maneuver Decision-Making Algorithm for Autonomous Airdrop Based on Deep Reinforcement Learning[J]. Sensors, 2021, 21(6): 2233.Google ScholarCross Ref
- Yang Qiming, Zhang Jiandong, Shi Guoqing, Hu Jinwen, Wu Yong. Maneuver Decision of UAV in Short-Range Air Combat Based on Deep Reinforcement Learning[J].IEEE ACCESS, 2020: 363-378.Google Scholar
- Song Xiagan, Jiang Ju. Research on intelligent air combat decision under uncertain environment[D]. Nanjing: Nanjing University of Aeronautics and Astronautics, 2017.03.Google Scholar
- LIU Chao, LI Qingwei. Design of Simulation Software System for Intelligent Decision – Making in Tactical – Level Air Combat[A]. 2019 China System Simulation and Virtual Reality Technology High-level Forum[C],2019.Google Scholar
- Lei Yonglin, Yao Jian, Zhu Ning, Zhu Yifan, Wang Weiping. Weapon Effectiveness Simulation System (WESS)[J]. Journal of System Simulation. 2017,29(6): 1244-1252.Google Scholar
- AUSTIN F, CARBONE G, MICHAEL F, el a1. Game Theory for Automated Maneuvering during Air-to—Air Combat[J].Journal of Guidance 1990,13(6):1143—1147.Google Scholar
- VIRLANEN K, RAIVIO T, HAMALAINEN R P. Decision Theoretical Approach to Pilot Simulation[J].Journal of Aircraft, 1999,27(4):632—641.Google Scholar
- HORIE K, CONWAY B. Optimal Fighter Pursuit-Evasion Maneuvers Found via Two-Sided Optimization[J]. Journal of Guidance, Control and Dynamics, 2006, 29(1): 105—112.Google ScholarCross Ref
- ZHU Shihu, DONG Chaoyang, ZHANG Jinpeng, CHEN Zongji. An intelligent decision-making system based on neural networks and expert system [J].Electro-optical and control, 2006,13(1): 8-11.Google Scholar
- SMITH R E, DIKE B A, MEHRA R K, Classifier systems in combat: two-sided learning of maneuvers for advanced fighter aircraft[J].Computer Methods in Applied Mechanics and Engineering, 2000,186(2): 421.DOI: 10.1016/S0045−7825(99) 00395−3.Google ScholarCross Ref
- NICHOLAS E, COHEN K, SCHUMACHER D. Collaborative tasking of UAVs using a genetic fuzzy approach[C]. /51st AIAA Aerospace Sciences Meeting including the New Horizons Forum and Aerospace Exposition. Grapevine: AIAA, 2013: 1032. DOI: 10.2514 /6. 2013-1032.Google Scholar
Recommendations
Infrared Air Combat Simulation Model for Deep Reinforcement Learning
CSAE '21: Proceedings of the 5th International Conference on Computer Science and Application EngineeringAiming at the problem of lacking credible and realistic infrared air combat simulation platform for applying the deep reinforcement learning method, this paper explored the design requirements for the construction of simulation system, built the overall ...
Autonomous air combat decision‐making of UAV based on parallel self‐play reinforcement learning
AbstractAiming at addressing the problem of manoeuvring decision‐making in UAV air combat, this study establishes a one‐to‐one air combat model, defines missile attack areas, and uses the non‐deterministic policy Soft‐Actor‐Critic (SAC) algorithm in ...
Multi-intent autonomous decision-making for air combat with deep reinforcement learning
AbstractAutonomous decision-making in unmanned aerial vehicle (UVA) confrontations presents challenges in making optimal strategy. Therefore, deep reinforcement learning (DRL) has been adopted to address these issues. However, existing DRL decision-making ...
Comments