Skip to main content
SpringerLink
Log in

Morphing aircraft acceleration and deceleration task morphing strategy using a reinforcement learning method

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

This paper proposes a design scheme for a whole morphing strategy based on the reinforcement learning (RL) method. A novel morphing aircraft is designed, and its nonlinear dynamic equations are established based on the calculated aerodynamic data. Further, a soft actor critic (SAC) approach is utilized to design the scheme, whose structure consists of the environment, the agent, and the reward function. In the environment design part, the incremental backstepping approach is employed to design the morphing aircraft controller. The safety and feasibility of deployment are verified. In the agent design part, in addition to using the entropy regularization RL algorithm, the generalization ability of the agent is enhanced in three ways: adding environmental noise, adding control command randomness, and adding output momentum terms. For the reward function, a structure with dynamic and steady-state performance is designed to accurately describe the aircraft dynamics. Finally, the designed SAC strategy is verified under the acceleration and deceleration tasks and compared with a GA and PPO strategy. Simulation results validate the effectiveness and superiority of the designed SAC scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Tsushima N, Tamayama M (2019) Recent researches on morphing aircraft technologies in Japan and other countries. Mech Eng Rev 6(2):19–00197–19–00197. https://doi.org/10.1299/mer.19-00197

    Article  Google Scholar 

  2. Kölbl M, Ermanni P (2022) Structural design and analysis of an anisotropic, bi-axially morphing skin concept. Aerosp Sci Technol 120. https://doi.org/10.1016/j.ast.2021.107292

  3. Hao F, Tang T, Gao Y, Li Y, Yi S, Lu J (2021) Continuous morphing trailing-edge wing concept based on multi-stable nanomaterial. Chin J Aeronaut 34(7):219–231. https://doi.org/10.1016/j.cja.2020.03.041

    Article  Google Scholar 

  4. Kammegne MJT, Botez RM, Grigorie LT, Mamou M, Mébarki Y (2017) Proportional fuzzy feed-forward architecture control validation by wind tunnel tests of a morphing wing. Chin J Aeronaut 30(2):561–576. https://doi.org/10.1016/j.cja.2017.02.001

    Article  Google Scholar 

  5. Xu W, Li Y, Pei B, Yu Z (2022) Coordinated intelligent control of the flight control system and shape change of variable sweep morphing aircraft based on dueling-DQN. Aerosp Sci Technol 130. https://doi.org/10.1016/j.ast.2022.107898

  6. Li D et al (2018) A review of modelling and analysis of morphing wings. Progress Aerospace Sci 100:46–62. https://doi.org/10.1016/j.paerosci.2018.06.002

    Article  Google Scholar 

  7. Wu M, Xiao T, Ang H, Li H (2018) Optimal flight planning for a Z-shaped morphing-wing solar-powered unmanned aerial vehicle. J Guid Control Dyn 41(2):497–505. https://doi.org/10.2514/1.G003000

    Article  Google Scholar 

  8. Xu W, Li Y, Lv M, Pei B (2022) Modeling and switching adaptive control for nonlinear morphing aircraft considering actuator dynamics. Aerosp Sci Technol 122. https://doi.org/10.1016/j.ast.2022.107349

  9. Qiao F, Shi J, Qu X, Lyu Y (2022) Adaptive Back-stepping neural control for an embedded and Tiltable V-tail morphing aircraft. Int J Control Autom Syst 20(2):678–690. https://doi.org/10.1007/s12555-020-0694-0

    Article  Google Scholar 

  10. Wu Z, Lu J, Shi J, Liu Y, Zhou Q (2017) Robust adaptive neural control of morphing aircraft with prescribed performance. Math Probl Eng 2017. https://doi.org/10.1155/2017/1401427

  11. Hammerton JR, Su W, Zhu G, Swei SSM (2018) Optimum distributed wing shaping and control loads for highly flexible aircraft. Aerosp Sci Technol 79:255–265. https://doi.org/10.1016/j.ast.2018.05.045

    Article  Google Scholar 

  12. Gu X, Yang K, Wu M, Zhang Y, Zhu J, Zhang W (2021) Integrated optimization design of smart morphing wing for accurate shape control. Chin J Aeronaut 34(1):135–147. https://doi.org/10.1016/j.cja.2020.08.048

    Article  Google Scholar 

  13. Mansell JR, Grant MJ (2018) Adaptive continuation strategy for indirect hypersonic trajectory optimization. J Spacecr Rocket 55(4):818–828. https://doi.org/10.2514/1.A34013

    Article  Google Scholar 

  14. Li W, Wang W, Huang X, Zhang S, Li C (2021) Roll control of morphing aircraft with synthetic jet actuators at a high angle of attack. Appl Sci 11(2):1–12. https://doi.org/10.3390/app11020505

    Article  Google Scholar 

  15. Themistokleous C, Markatos NG, Prospathopoulos J, Riziotis V, Sieros G, Papadakis G. A high-lift optimization methodology for the design of leading and trailing edges on morphing wings. Appl Sci 11(6). https://doi.org/10.3390/app11062822

  16. Gong C, Ma BF (2019) Shape optimization and sensitivity analysis of a morphing-wing aircraft. Int J Aeronaut Sp Sci 20(1):57–69. https://doi.org/10.1007/s42405-018-0110-7

    Article  Google Scholar 

  17. Dai P, Yan B, Liu R, Liu S, Wang M (2021) Integrated morphing strategy and trajectory optimization of a morphing Waverider and its online implementation based on the neural network. IEEE Access 9:59383–59393. https://doi.org/10.1109/ACCESS.2021.3073207

    Article  Google Scholar 

  18. Silver D et al (2017) Mastering the game of go without human knowledge. Nature 550(7676):354–359. https://doi.org/10.1038/nature24270

    Article  Google Scholar 

  19. Muthusamy PK, Garratt M, Pota HR, Muthusamy R (2021) Realtime adaptive intelligent control system for quadcopter UAV with payload uncertainties. IEEE Trans Ind Electron. https://doi.org/10.1109/TIE.2021.3055170

  20. Jardine PT, Givigi SN, Yousefi S (2021) Leveraging data engineering to improve unmanned aerial vehicle control design. IEEE Syst J 15(2):2595–2606. https://doi.org/10.1109/JSYST.2020.3003203

    Article  Google Scholar 

  21. Lin X, Liu J, Yu Y, Sun C (2020) Event-triggered reinforcement learning control for the quadrotor UAV with actuator saturation. Neurocomputing 415:135–145. https://doi.org/10.1016/j.neucom.2020.07.042

    Article  Google Scholar 

  22. Hajijamali Arani A, Azari MM, Hu P, Zhu Y, Yanikomeroglu H, Safavi-Naeini S (2022) Reinforcement learning for energy-efficient trajectory design of UAVs. IEEE Internet Things J 9(11):9060–9070. https://doi.org/10.1109/JIOT.2021.3118322

    Article  Google Scholar 

  23. Liu Y, Wang H, Wu T, Lun Y, Fan J, Wu J (2022) Attitude control for hypersonic reentry vehicles: An efficient deep reinforcement learning method. Appl Soft Comput 123. https://doi.org/10.1016/j.asoc.2022.108865

  24. Lu K, Fu Q, Cao R, Peng J, Wang Q (2022) Asymmetric Airfoil Morphing via Deep Reinforcement Learning. Biomimetics 7(4):188. https://doi.org/10.3390/biomimetics7040188

    Article  Google Scholar 

  25. Haughn KPT, Gamble LL, Inman DJ (2022) Deep reinforcement learning achieves multifunctional morphing airfoil control. J Compos Mater. https://doi.org/10.1177/00219983221137644

  26. Xu D, Hui Z, Liu Y, Chen G (2019) Morphing control of a new bionic morphing UAV with deep reinforcement learning. Aerosp Sci Technol 92:232–243. https://doi.org/10.1016/j.ast.2019.05.058

    Article  Google Scholar 

  27. Che H, Chen J, Wang Y, Wang J, Luo Y (2022) Data-driven model-free adaptive attitude control for morphing vehicles. IET Control Theory Appl 16(16):1696–1707. https://doi.org/10.1049/cth2.12335

    Article  MathSciNet  Google Scholar 

  28. Seigler TM, Neal DA (2009) Analysis of transition stability for morphing aircraft. J Guid Control Dyn 32(6):1947–1953. https://doi.org/10.2514/1.44108

    Article  Google Scholar 

  29. Haarnoja T, Zhou A, Abbeel P, Levine S (2018) Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: Dy J, Krause A (eds) Proceedings of the 35th International Conference on Machine Learning, vol 80 of Proceedings of Machine Learning Research. PMLR, pp 1861–1870. https://proceedings.mlr.press/v80/haarnoja18b.html

Download references

Acknowledgements

The authors would like to express their gratitude to the Shaanxi Province Key Laboratory of Flight Control and Simulation Technology for supporting this research. This research work is funded by the National Natural Science Foundation of China (No. 62073266) and the Aeronautical Science Foundation of China (No. 201905053003).

Author information

Authors and Affiliations

  1. Northwestern Polytechnical University, Xi’an, Shaanxi, China

    Ruichen Ming, Xiaoxiong Liu, Yu Li, Yi Yin & WeiGuo Zhang

Authors
  1. Ruichen Ming
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoxiong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yi Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. WeiGuo Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoxiong Liu.

Ethics declarations

Competing of interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ming, ., Liu, X., Li, Y. et al. Morphing aircraft acceleration and deceleration task morphing strategy using a reinforcement learning method. Appl Intell 53, 26637–26654 (2023). https://doi.org/10.1007/s10489-023-04876-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04876-y

Keywords

Search

Navigation