Conghui Huang
ABSTRACT
Aiming at the policy problem of unmanned combat aerial vehicle (UCAV) evading infrared air-to-air missiles in close air combat, based on the establishment of an infrared offensive and defensive confrontation simulation system, the application of soft actor critic algorithm is studied to train the agent to learn the escape maneuver policy and decoy launching policy of UCAV to evade missiles. The three-dimensional coordinates of the missile in the UCAV body coordinate system and the number of remaining decoys are taken as the input states. The joystick, throttle stick control stroke and the decoy launching pulse are taken as the output actions. The dense reward composed of relative situational parameters and flight parameters and the sparse reward constituted by the result of the decoy interference and the result of the engagement are designed. The soft actor critic (SAC) algorithm is improved to adapt to the action space of mixed continuous action and discrete action, and finally the end-to-end UCAV escape maneuver policy and decoy launching policy from state input to control output is obtained. Through simulation verification, the escape rates of the UCAV with and without the decoys are compared, and the results show that the escape rate with escape maneuver policy realized by the agent can reach 59.0%, and the escape rate combined with the decoy launching policy will increase by 6.7%, finally the UCAV escape rate can reach 65.7%.
- Fumiaki Imado. High-g Barrel Roll Maneuvers Against Proportional Navigation from Optimal Control Viewpoint[J]. JOURNAL OF GUIDANCE, CONTROL, AND DYNAMICS, 1998, 21(6): 876-881.Google Scholar
- Remzi AKDAG, D. Turgay ALTILAR. A Comparative Study on Practical Evasive Maneuvers Against Proportional Navigation Missiles[C]. Proceedings of the AIAA Guidance, Navigation, and Control Conference and Exhibit, 2006, Keystone, Colorado.Google Scholar
- WANG Si-cai, NAN Ying, LIU Jing-wei. Optimal Escape Strategy of Fighter against Oncoming Missiles [J]. Aero Weaponry, 2009,4, 28-32.Google Scholar
- WANG Jie, DING Dali, CHEN Cheng, UCAV trial maneuvering decision under missile attack state assessment [J]. Journal of Harbin Institute of Technology, 2021,53(6): 118-127.Google Scholar
- Jiang Chao, Wang Weijia, Wang Hao. The Evolutionary Expert System for the Missile Avoidance System[C]. Proceedings of the Fourth China Aviation Science and Technology Conference,2019,132-144.Google Scholar
- Zhang Hongpeng, Huang Changqiang, Wei Zhenglei, Generation of optimal evasive maneuvers of UCAV based on BP neural networks[J]. Flight Dynamics, 2020, 38(3): 46-51.Google Scholar
- Song Hongchuan, Zhan Hao, Xia Lu, The Study on a Fighter Against a Medium-range Air-to-air Missile Based on Deep Deterministic Policy Gradient Algorithm[J]. Advances in Aeronautical Science And Engineering, 2021, 12(3): 85-94.Google Scholar
- Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor[C]. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, PMLR 80, 2018.Google Scholar
- Huang Cong-hui, Wang Chao-zhe, Tong Qi. Infrared Air Combat Simulation Model for Deep Reinforcement Learning[C]. Proceeding of the 5th International Conference on Computer Science and Application Engineering, Sanya China, October 19 - 21, 2021.Google ScholarDigital Library
- Fang Zhenping, Chen Wangchun, Zhang Shugang. Aerospace Vehicle Flight Dynamics [M]. Beijing University of Aeronautics and Astronautics Press, 2005.Google Scholar
- Zhang Yuansheng. Development of airborne Electro-Optical warning system[J]. Electronics Optics & Control, 2015, 22(6): 52-55.Google Scholar
- Zhen Yang , Deyun Zhou , Haiyin Piao, Evasive Maneuver Strategy for UCAV in Beyond-Visual-Range Air Combat Based on Hierarchical Multi-Objective Evolutionary Algorithm[J]. IEEE Access, 2020,8: 46605-46623.Google ScholarCross Ref
- Hesong Huang, Zhongxiang Tong, Taorui Li, Defense Strategy of Aircraft Confronted with IR Guided Missile[J]. Mathematical Problems in Engineering, 2017:1-9.Google Scholar
Index Terms
- Research on Evasion Policy of UCAV Against Infrared Air-to-Air Missile Based on Soft Actor Critic Algorithm
Recommendations
Manoeuvre decision‐making of unmanned aerial vehicles in air combat based on an expert actor‐based soft actor critic algorithm
AbstractThe demand for autonomous motion control of unmanned aerial vehicles in air combat is boosted as taking the initiative in combat appears more and more crucial. Unmanned aerial vehicles inability to manoeuvre autonomously during air combat that ...
Aerodynamic mechanisms in bio‐inspired micro air vehicles: a review in the light of novel compound layouts
Modern designs of micro air vehicles (MAVs) are mostly inspired by nature's flyers, such as hummingbirds and flying insects, which results in the birth of bio‐inspired MAVs. The history and recent progress of the aerodynamic mechanisms in bio‐inspired ...
Comments