Skip to main content
SpringerLink
Log in

Development of reinforced learning based non-linear controller for unmanned aerial vehicle

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Design complexities of trending UAVs and the operational harsh environments necessitates Control Law formulation utilizing intelligent techniques that are both robust, model-free and adaptable. In this research, an intelligent control architecture for an experimental Unmanned Aerial Vehicle (UAV) having an unconventional inverted V-tail design, is presented. Due to unique design of the vehicle strong roll and yaw coupling exists, making the control of vehicle challenging. To handle UAV’s inherent control complexities, while keeping them computationally acceptable, a variant of distinct Deep Reinforcement learning (DRL) algorithm, namely Reformed Deep Deterministic Policy Gradient (R-DDPG) is proposed. Conventional DDPG algorithm after being modified in its learning architecture becomes capable of intelligently handling the continuous state and control space domains besides controlling the platform in its entire flight regime. The paper illustrates the application of modified DDPG algorithm (namely R-DDPG) towards the design, while the performance of the resulting controller is assessed in simulation using dynamic model of the vehicle. Nonlinear simulations were then performed to analyze UAV performance under different environmental and launch conditions. The effectiveness of the proposed strategy is further demonstrated by comparing the results with the linear controller for the same UAV whose feedback loop gains are optimized by employing technique of optimal control theory achieved through application of Linear quadratic regulator (LQR) based control strategy. The efficacy of the results and performance characteristics, demonstrated the ability of the presented algorithm to dynamically adapt to the changing environment, thereby making it suitable for UAV applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Abbreviations

API:

Application programming interface

b :

Wing span (m)

\(\tilde{c}\) :

Mean aerodynamic chord (m)

CAD:

Computer aided design

CFD:

Computational fluid dynamics

\(C_{M_x}\) :

Rolling moment coefficient

\(C_{M_y}\) :

Pitching moment coefficient

\(C_{M_z}\) :

Yawing moment coefficient

\(C_{F_x}\) :

X-direction force coefficient

\(C_{F_y}\) :

Y-direction force coefficient

\(C_{F_z}\) :

Z-direction force coefficient

DDPG:

Deep deterministic policy gradient

DoF:

Degree of freedom

g :

Acceleration due Gravity (m/s\(^2\))

h :

Altitude (m)

LCF:

Left control fin

ML:

Machine learning

O-PPO:

Optimal proximal policy optimization

POMDP:

Partial observable Markov decision process

R-DDPG:

Reformed deep deterministic policy gradient

m :

Vehicle’s mass (kg)

P :

Roll rate (deg/s)

\(P_E\) :

Position vector—east (km)

\(P_N\) :

Position vector—north (km)

Q :

Pitch rate (deg/s)

Parm :

Parameter

R :

Yaw rate (deg/s)

RL:

Reinforcement Learning

RCF:

Right control fin

S :

Wing area (m\(^2\))

UAV:

Unmanned aerial vehicle

\(V_T\) :

Free stream velocity (m/s)

NNs :

Neural networks

\(wt_i\) :

Numerical weight (ith number)

Xcut :

Current X-position (m)

Ycut :

Current Y-position (m)

Zcut :

Current Z-position (m)

R :

Instantaneous reward

TR:

Total reward

Py:

Penalty

\(\alpha\) :

Angle of attack (deg)

\(\beta\) :

Sideslip angle (deg)

\(\gamma\) :

Flight path angle (deg)

\(\psi\) :

Yaw angle (deg)

\(\phi\) :

Roll angle (deg)

\(\theta\) :

Theta angle (deg)

\(\delta _L\) :

LCF deflection (deg)

\(\delta _R\) :

RCF deflection (deg)

\(\rho\) :

Density of air (kg/m\(^3\))

References

  • Aboelezz A, Mohamady O, Hassanalian M, Elhadidi B (2021) Nonlinear flight dynamics and control of a fixed-wing micro air vehicle: numerical, system identification and experimental investigations. J Intell Robot Syst 101(3):1–18

    Google Scholar 

  • Adams RJ, Banda SS (1993) Robust flight control design using dynamic inversion and structured singular value synthesis. IEEE Trans Control Syst Technol 1(2):80–92

    Google Scholar 

  • Adams RJ, Buffington JM, Banda SS (1994) Design of nonlinear control laws for high-angle-of-attack flight. J Guid Control Dyn 17(4):737–746

    MATH  Google Scholar 

  • Adams RJ, Buffington JM, Sparks AG, Banda SS (2012) Robust multivariable flight control. Springer Science & Business Media, Berlin

    Google Scholar 

  • Araar O, Aouf N (2014) Full linear control of a quadrotor uav, lq vs hinf. In: 2014 UKACC International Conference on control (CONTROL), IEEE, pp 133–138

  • Azar AT, Koubaa A, Ali Mohamed N, Ibrahim HA, Ibrahim ZF, Kazim M, Ammar A, Benjdira B, Khamis AM, Hameed IA et al (2021) Drone deep reinforcement learning: a review. Electronics 10(9):999

    Google Scholar 

  • Bansal T, Pachocki J, Sidor S, Sutskever I, Mordatch I (2017) Emergent complexity via multi-agent competition. arXiv preprint arXiv:1710.03748

  • Bouhamed O, Ghazzai H, Besbes H, Massoud Y (2020) Autonomous UAV navigation: a DDPG-based deep reinforcement learning approach. In: 2020 IEEE International Symposium on circuits and systems (ISCAS), vol 1. IEEE, pp 1–5

  • Brière D, Traverse P (1993) Airbus a320/a330/a340 electrical flight controls-a family of fault-tolerant systems. In: FTCS-23 The Twenty-Third International Symposium on fault-tolerant computing, vol 1. IEEE, pp 616–623

  • Buning PG, Gomez RJ, Scallion WI (2004) Cfd approaches for simulation of wing-body stage separation. AIAA Paper 4838:2004

    Google Scholar 

  • Cai G, Dias J, Seneviratne L (2014) A survey of small-scale unmanned aerial vehicles: Recent advances and future development trends. Unmanned Syst 2(02):175–199

    Google Scholar 

  • Chen J, Xiao Z, Xing H, Dai P, Luo S, Iqbal MA (2020) Stdpg: a spatio-temporal deterministic policy gradient agent for dynamic routing in sdn. In: ICC 2020-2020 IEEE International Conference on communications (ICC), vol 1. IEEE, pp 1–6

  • Chen J, Xing H, Xiao Z, Xu L, Tao T (2021) A drl agent for jointly optimizing computation offloading and resource allocation in mec. IEEE Internet Things J 8(24):17508–17524

    Google Scholar 

  • Chowdhary G, Frazzoli E, How J, Liu H (2014) Nonlinear flight control techniques for unmanned aerial vehicles. In: Handbook of unmanned aerial vehicles, Springer, Houten

  • CS231n S (2017) Convolutional neural networks for visual recognition. https://www.cs231ngithubio/neural-networks-3/#baby. Accessed 1 Sept 2020

  • Dalal G, Dvijotham K, Vecerik M, Hester T, Paduraru C, Tassa Y (2018) Safe exploration in continuous action spaces. arXiv preprint arXiv:1801.08757

  • Derafa L, Ouldali A, Madani T, Benallegue A (2011) Non-linear control algorithm for the four rotors uav attitude tracking problem. Aeronaut J 115(1165):175–185

    Google Scholar 

  • Din AFU, Mir I, Gul F, Mir S, Alhady SSN, Nasar A, Rustom M, Alkhazaleh HA, Abualigah L (2022) Robust flight control system design of a fixed wing uav using optimal dynamic programming. Soft Comput. https://doi.org/10.1007/s00500-022-07484-z

  • Dong Y, Zou X (2020) Mobile robot path planning based on improved ddpg reinforcement learning algorithm. In: 2020 IEEE 11th International Conference on software engineering and service science (ICSESS), vol 1. IEEE, pp 52–56

  • Doyle J, Lenz K, Packard A (1987) Design examples using \(\mu\)-synthesis: space shuttle lateral axis fcs during reentry. In: Modelling, robustness and sensitivity reduction in control systems. Springer, pp 127–154

  • Dutoi B, Richards N, Gandhi N, Ward D, Leonard J (2008) Hybrid robust control and reinforcement learning for optimal upset recovery. In: AIAA Guidance, Navigation and Control Conference and Exhibit, vol 1. p 6502

  • Elmeseiry N, Alshaer N, Ismail T (2021) A detailed survey and future directions of unmanned aerial vehicles (uavs) with potential applications. Aerospace 8(12):363

    Google Scholar 

  • Enomoto K, Yamasaki T, Takano H, Baba Y (2013) Guidance and control system design for chase uav. In: AIAA Guidance, Navigation and Control Conference and exhibit, vol 1. p 6842

  • Escareno J, Salazar-Cruz S, Lozano R (2006) Embedded control of a four-rotor uav. In: 2006 American Control Conference, IEEE, vol 1. pp 6–pp

  • Finck R, (US) AFFDL, Hoak D (1978) USAF stability and control DATCOM. Eng Doc

  • Giordan D, Adams MS, Aicardi I, Alicandro M, Allasia P, Baldo M, De Berardinis P, Dominici D, Godone D, Hobbs P et al (2020) The use of unmanned aerial vehicles (uavs) for engineering geology applications. Bull Eng Geol Env 79(7):3437–3481

    Google Scholar 

  • Golian M, Katibeh H, Singh VP, Ostad-Ali-Askari K, Rostami HT (2020) Prediction of tunnelling impact on flow rates of adjacent extraction water wells. Q J Eng Geol Hydrogeol 53(2):236–251

    Google Scholar 

  • Gul F, Rahiman W (2019) An integrated approach for path planning for mobile robot using bi-rrt. In: IOP Conference Series: materials science and engineering, IOP Publishing, vol 697, p 012022

  • Gul F, Rahiman W, Nazli Alhady SS (2019) A comprehensive study for robot navigation techniques. Cogent Eng 6(1):1632046

    Google Scholar 

  • Gul F, Alhady SSN, Rahiman W (2020a) A review of controller approach for autonomous guided vehicle system. Indones J Electr Eng Comput Sci 20(1):552–562

    Google Scholar 

  • Gul F, Rahiman W, Alhady SN, Ali A, Mir I, Jalil A (2020b) Meta-heuristic approach for solving multi-objective path planning for autonomous guided robot using pso–gwo optimization algorithm with evolutionary programming. J Ambient Intell Humaniz Comput pp 1–18

  • Gul F, Mir I, Abualigah L, Sumari P (2021a) Multi-robot space exploration: an augmented arithmetic approach. IEEE Access 9:107738–107750

    Google Scholar 

  • Gul F, Mir I, Abualigah L, Sumari P, Forestiero A (2021b) A consolidated review of path planning and optimization techniques: technical perspectives and future directions. Electronics 10(18):2250

    Google Scholar 

  • Gul F, Mir I, Rahiman W, Islam TU (2021c) Novel implementation of multi-robot space exploration utilizing coordinated multi-robot exploration and frequency modified whale optimization algorithm. IEEE Access 9:22774–22787

    Google Scholar 

  • Gul F, Rahiman W, Alhady S, Ali A, Mir I, Jalil A (2021d) Meta-heuristic approach for solving multi-objective path planning for autonomous guided robot using pso-gwo optimization algorithm with evolutionary programming. J Ambient Intell Humaniz Comput 12(7):7873–7890

    Google Scholar 

  • Gul F, Mir I, Abualigah L, Mir S, Altalhi M (2022a) Cooperative multi-function approach: a new strategy for autonomous ground robotics. Future Gener Comput Syst 134:361–373

  • Gul F, Mir I, Alarabiat D, Alabool HM, Abualigah L, Mir S (2022b) Implementation of bio-inspired hybrid algorithm with mutation operator for robotic path planning. J Parallel Distrib Comput 169:171–184

    Google Scholar 

  • Gul F, Mir S, Mir I (2022c) Coordinated multi-robot exploration: hybrid stochastic optimization approach. In: AIAA SCITECH 2022 Forum, p 1414

  • Gul F, Mir S, Mir I (2022d) Multi robot space exploration: a modified frequency whale optimization approach. In: AIAA SCITECH 2022 Forum, p 1416

  • Hafner R, Riedmiller M (2011) Reinforcement learning in feedback control. Mach Learn 84(1–2):137–169

    MathSciNet  Google Scholar 

  • Heess N, Hunt JJ, Lillicrap TP, Silver D (2015) Memory-based control with recurrent neural networks. arXiv preprint arXiv:1512.04455

  • Henderson P, Islam R, Bachman P, Pineau J, Precup D, Meger D (2018) Deep reinforcement learning that matters. In: Proceedings of the AAAI Conference on artificial intelligence, vol 32. pp 1–12

  • Hou Z, Lu P, Tu Z (2020) Nonsingular terminal sliding mode control for a quadrotor uav with a total rotor failure. Aerosp Sci Technol 98:105716

    Google Scholar 

  • Hu H, Ql Wang (2020) Proximal policy optimization with an integral compensator for quadrotor control. Front Inf Technol Electron Eng 21(5):777–795

    Google Scholar 

  • Hussain A, Hussain I, Mir I, Afzal W, Anjum U, Channa BA (2020) Target parameter estimation in reduced dimension stap for airborne phased array radar. In: 2020 IEEE 23rd International Multitopic Conference (INMIC), IEEE, pp 1–6

  • Hussain A, Anjum U, Channa BA, Afzal W, Hussain I, Mir I (2021) Displaced phase center antenna processing for airborne phased array radar. In: 2021 International Bhurban Conference on applied sciences and technologies (IBCAST), IEEE, pp 988–992

  • Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285

    Google Scholar 

  • Kim D, Oh G, Seo Y, Kim Y (2017) Reinforcement learning-based optimal flat spin recovery for unmanned aerial vehicle. J Guid Control Dyn 40(4):1076–1084

    Google Scholar 

  • Kimathi S (2017) Application of reinforcement learning in heading control of a fixed wing uav using x-plane platform

  • Kinga DA (2015) A method for stochastic optimization. In: Anon International Conference on learning representations, San Diego: ICLR

  • Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  • Koch W, Mancuso R, West R, Bestavros A (2019) Reinforcement learning for uav attitude control. ACM Trans Cyber-Phys Syst 3(2):1–21

    Google Scholar 

  • Kroezen D (2019) Online reinforcement learning for flight control: an adaptive critic design without prior model knowledge

  • Kulcsar B (2000) Lqg/ltr controller design for an aircraft model. Period Polytech Transp Eng 28(1–2):131–142

    Google Scholar 

  • Labbadi M, Cherkaoui M (2019) Robust adaptive backstepping fast terminal sliding mode controller for uncertain quadrotor uav. Aerosp Sci Technol 93:105306

    Google Scholar 

  • Laroche R, Feraud R (2017) Reinforcement learning algorithm selection. arXiv preprint arXiv:1701.08810

  • Lei C (2021) Deep reinforcement learning. In: Deep learning and practice with MindSpore, Springer, pp 217–243

  • Lillicrap TP, Hunt JJ, Pritzel A, Heess N, Erez T, Tassa Y, Silver D, Wierstra D (2015) Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971

  • Lin X, Liu J, Yu Y, Sun C (2020) Event-triggered reinforcement learning control for the quadrotor uav with actuator saturation. Neurocomputing 415:135–145

    Google Scholar 

  • Luo X, Zhang Y, He Z, Yang G, Ji Z (2019) A two-step environment-learning-based method for optimal uav deployment. IEEE Access 7:149328–149340

    Google Scholar 

  • Mir I, Maqsood A, Akhtar S (2017a) Dynamic modeling & stability analysis of a generic uav in glide phase. In: MATEC Web of Conferences, EDP Sciences, vol 114, p 01007

  • Mir I, Maqsood A, Akhtar S (2017b) Optimization of dynamic soaring maneuvers to enhance endurance of a versatile uav. In: IOP Conference series: materials science and engineering, IOP Publishing, vol 211, p 012010

  • Mir I, Maqsood A, Akhtar S (2017c) Optimization of dynamic soaring maneuvers to enhance endurance of a versatile uav. In: IOP Conference series: materials science and engineering, IOP Publishing, vol 211, p 012010

  • Mir I, Eisa SA, Maqsood A (2018a) Review of dynamic soaring: technical aspects, nonlinear modeling perspectives and future directions. Nonlinear Dyn 94(4):3117–3144

    Google Scholar 

  • Mir I, Maqsood A, Akhtar S (2018b) Biologically inspired dynamic soaring maneuvers for an unmanned air vehicle capable of sweep morphing. Int J Aeronaut Sp Sci 19(4):1006–1016

    Google Scholar 

  • Mir I, Maqsood A, Eisa SA, Taha H, Akhtar S (2018c) Optimal morphing-augmented dynamic soaring maneuvers for unmanned air vehicle capable of span and sweep morphologies. Aerosp Sci Technol 79:17–36

    Google Scholar 

  • Mir I, Taha H, Eisa SA, Maqsood A (2018d) A controllability perspective of dynamic soaring. Nonlinear Dyn 94(4):2347–2362

    MATH  Google Scholar 

  • Mir I, Akhtar S, Eisa S, Maqsood A (2019a) Guidance and control of standoff air-to-surface carrier vehicle. Aeronaut J 123(1261):283–309

    Google Scholar 

  • Mir I, Maqsood A, Taha HE, Eisa SA (2019b) Soaring energetics for a nature inspired unmanned aerial vehicle. In: AIAA Scitech 2019 Forum, p 1622

  • Mir I, Eisa SA, Taha H, Maqsood A, Akhtar S, Islam TU (2021a) A stability perspective of bio-inspired uavs performing dynamic soaring optimally. Bioinspir Biomim 16(6):066010

  • Mir I, Eisa SA, Taha H, Maqsood A, Akhtar S, Islam TU (2021b) A stability perspective of bioinspired unmanned aerial vehicles performing optimal dynamic soaring. Bioinspir Biomim 16(6):066010

    Google Scholar 

  • Mir I, Eisa S, Maqsood A, Gul F (2022a) Contraction analysis of dynamic soaring. In: AIAA SCITECH 2022 Forum, p 0881

  • Mir I, Eisa S, Taha HE, Gul F (2022b) On the stability of dynamic soaring: Floquet-based investigation. In: AIAA SCITECH 2022 Forum, p 0882

  • Mir I, Gul F, Mir S, Khan MA, Saeed N, Abualigah L, Abuhaija B, Gandomi AH (2022c) A survey of trajectory planning techniques for autonomous systems. Electronics 11(18):2801

    Google Scholar 

  • Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533

    Google Scholar 

  • Napolitano MR, An Y, Seanor BA (2000) A fault tolerant flight control system for sensor and actuator failures using neural networks. Aircraft Design 3(2):103–128

  • Nikolakopoulos KG, Soura K, Koukouvelas IK, Argyropoulos NG (2017) Uav vs classical aerial photogrammetry for archaeological studies. J Archaeol Sci Rep 14:758–773

    Google Scholar 

  • Novati G, Mahadevan L, Koumoutsakos P (2018) Deep-reinforcement-learning for gliding and perching bodies. arXiv preprint arXiv:1807.03671

  • Nurbani ES (2018) Environmental protection in international humanitarian law. Unram Law Rev 2(1):1–12

    Google Scholar 

  • Ostad-Ali-Askari K, Shayannejad M, Ghorbanizadeh-Kharazi H (2017) Artificial neural network for modeling nitrate pollution of groundwater in marginal area of zayandeh-rood river, isfahan, iran. KSCE J Civ Eng 21(1):134–140

    Google Scholar 

  • Pan J, Wang X, Cheng Y, Yu Q (2018) Multisource transfer double dqn based on actor learning. IEEE Trans Neural Netw Learn Syst 29(6):2227–2238

    Google Scholar 

  • Paucar C, Morales L, Pinto K, Sánchez M, Rodríguez R, Gutierrez M, Palacios L (2018) Use of drones for surveillance and reconnaissance of military areas. In: International Conference of research applied to defense and security, Springer, pp 119–132

  • Peng K (2021) Autonomous mission management based nonlinear flight control design for a class of hybrid unmanned aerial vehicles. Guid Navig Control 1(02):2150009

    Google Scholar 

  • Petterson K (2006) Cfd analysis of the low-speed aerodynamic characteristics of a ucav. AIAA Paper 1259:2006

    Google Scholar 

  • Pi CH, Ye WY, Cheng S (2021) Robust quadrotor control through reinforcement learning with disturbance compensation. Appl Sci 11(7):3257

    Google Scholar 

  • Pirnazar M, Hasheminasab H, Karimi AZ, Ostad-Ali-Askari K, Ghasemi Z, Haeri-Hamedani M, Mohri-Esfahani E, Eslamian S (2018) The evaluation of the usage of the fuzzy algorithms in increasing the accuracy of the extracted land use maps. Int J Glob Environ Issues 17(4):307–321

    Google Scholar 

  • Poksawat P, Wang L, Mohamed A (2017) Gain scheduled attitude control of fixed-wing uav with automatic controller tuning. IEEE Trans Control Syst Technol 26(4):1192–1203

    Google Scholar 

  • Rastogi D (2017) Deep reinforcement learning for bipedal robots, Student Thesis

  • Rinaldi F, Chiesa S, Quagliotti F (2013) Linear quadratic control for quadrotors uavs dynamics and formation flight. J Intell Robot Syst 70(1–4):203–220

    Google Scholar 

  • Roaskam J (2001) Airplane flight dynamics and automatic flight controls. vol

  • Rodriguez-Ramos A, Sampedro C, Bavle H, De La Puente P, Campoy P (2019) A deep reinforcement learning strategy for uav autonomous landing on a moving platform. J Intell Robot Syst 93(1–2):351–366

    Google Scholar 

  • Rosales C, Soria CM, Rossomando FG (2019) Identification and adaptive pid control of a hexacopter uav based on neural networks. Int J Adapt Control Signal Process 33(1):74–91

    MathSciNet  MATH  Google Scholar 

  • Ruder S (2016) An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747

  • Schulman J, Levine S, Abbeel P, Jordan M, Moritz P (2015) Trust region policy optimization. In: International Conference on machine learning, PMLR, pp 1889–1897

  • Schulman J, Wolski F, Dhariwal P, Radford A, Klimov O (2017) Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

  • Silver D (2015) Lecture 2: Markov decision processes. UCL Retrieved from www.0csuclacuk/staff/dsilver/web/Teaching_files/MDPpdf

  • Silver D, Lever G, Heess N, Degris T, Wierstra D, Riedmiller M (2014) Deterministic policy gradient algorithms

  • Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M et al (2016) Mastering the game of go with deep neural networks and tree search. Nature 529(7587):484–489

    Google Scholar 

  • Sutton RS, Barto AG (1998) Planning and learning. In: Reinforcement learning: an introduction., ser. Adaptive computation and machine learning. A Bradford Book, pp 227–254

  • Szczepanski R, Tarczewski T, Grzesiak LM (2019) Adaptive state feedback speed controller for pmsm based on artificial bee colony algorithm. Appl Soft Comput 83:105644

    Google Scholar 

  • Szczepanski R, Bereit A, Tarczewski T (2021) Efficient local path planning algorithm using artificial potential field supported by augmented reality. Energies 14(20):6642

    Google Scholar 

  • Szczepanski R, Tarczewski T (2021) Global path planning for mobile robot based on artificial bee colony and dijkstra’s algorithms. In: 2021 IEEE 19th International Power Electronics and Motion Control Conference (PEMC), IEEE, pp 724–730

  • Tal E, Karaman S (2020) Accurate tracking of aggressive quadrotor trajectories using incremental nonlinear dynamic inversion and differential flatness. IEEE Trans Control Syst Technol 29(3):1203–1218

    Google Scholar 

  • Tang Y (2016) Tf. learn: tensorflow’s high-level module for distributed machine learning. arXiv preprint arXiv:1612.04251

  • Thorndike EL (1898) Animal intelligence. Nature 58(1504):390

  • Verma S (2020) A survey on machine learning applied to dynamic physical systems. arXiv preprint arXiv:2009.09719

  • Wang S, Jia D, Weng X (2018) Deep reinforcement learning for autonomous driving. arXiv preprint arXiv:1811.11329

  • Werbos PJ, Miller W, Sutton R (1990) A menu of designs for reinforcement learning over time. In: Neural networks for control, pp 67–95

  • Wickenheiser AM, Garcia E (2008) Optimization of perching maneuvers through vehicle morphing. J Guid Control Dyn 31(4):815–823

    Google Scholar 

  • Winkler S, Zeadally S, Evans K (2018) Privacy and civilian drone use: The need for further regulation. IEEE Secur Privacy 16(5):72–80

    Google Scholar 

  • Wu Y, Mansimov E, Grosse RB, Liao S, Ba J (2017) Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation. In: Advances in neural information processing systems, p 30

  • Xenou K, Chalkiadakis G, Afantenos S (2018) Deep reinforcement learning in strategic board game environments. In: European Conference on multi-agent systems, Springer, pp 233–248

  • Xiao Z, Xu X, Xing H, Luo S, Dai P, Zhan D (2021) Rtfn: a robust temporal feature network for time series classification. Inf Sci 571:65–86

    MathSciNet  Google Scholar 

  • Xu D, Hui Z, Liu Y, Chen G (2019) Morphing control of a new bionic morphing uav with deep reinforcement learning. Aerosp Sci Technol 92:232–243

    Google Scholar 

  • Yang Y, Zhu J, Yang J (2020) Indi-based transitional flight control and stability analysis of a tail-sitter uav. In: 2020 IEEE International Conference on systems, man, and cybernetics (SMC), IEEE, pp 1420–1426

  • Yanushevsky R (2011) Guidance of unmanned aerial vehicles. CRC Press, Boca Raton

    Google Scholar 

  • Zhou Y (2018) Online reinforcement learning control for aerospace systems

  • Zhou C, He H, Yang P, Lyu F, Wu W, Cheng N, Shen X (2019) Deep rl-based trajectory planning for aoi minimization in uav-assisted iot. In: 2019 11th International Conference on wireless communications and signal processing (WCSP), IEEE, pp 1–6

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical & Aerospace Engineering, Institute of Avionics & Aeronautics, Air University, Islamabad, Pakistan

    Adnan Fayyaz Ud Din

  2. School of Avionics & Electrical Engineering, College of Aeronautical Engineering, NUST, Risalpur, Pakistan

    Imran Mir

  3. Department of Electrical Engineering, Air University Aerospace & Aviation, Campus Kamra, Islamabad, Pakistan

    Faiza Gul

  4. Department of Aeronautical Engineering, Institute of Space Technologies, Islamabad, Pakistan

    Suhail Akhtar

Authors
  1. Adnan Fayyaz Ud Din
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Imran Mir
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Faiza Gul
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Suhail Akhtar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Imran Mir.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Din, A.F.U., Mir, I., Gul, F. et al. Development of reinforced learning based non-linear controller for unmanned aerial vehicle. J Ambient Intell Human Comput 14, 4005–4022 (2023). https://doi.org/10.1007/s12652-022-04467-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-022-04467-8

Keywords

Search

Navigation