Abstract
In this paper, the problem of adaptive optimized tracking control design is addressed for a class of nonlinear systems in strict-feedback form. The system under consideration contains input delay and has unmeasurable and restricted states within predefined compact sets. First, neural networks (NNs) are employed to approximate the unknown nonlinear dynamics, and an adaptive neural network (NN) state observer is constructed to compensate for the absence of state information. Additionally, by utilizing an auxiliary system compensation method alongside the backstepping technique, the impact of input delay is eliminated, and the generation of intermediate variables is prevented. Second, tan-type barrier optimal cost functions are established for each subsystem within the backstepping method to prevent the state variables from exceeding preselected sets. Moreover, by establishing both actor and critic NNs to execute a reinforcement learning algorithm, the optimal controller and optimal performance index function are evaluated, while relaxing the persistence of excitation condition. According to the Lyapunov stability theorem, it is demonstrated that all signals in the closed-loop system are semi-globally uniformly ultimately bounded (SGUUB), and the output signal accurately tracks a reference trajectory with the desired precision. Finally, a practical simulation example is provided to verify the effectiveness of the proposed control strategy, demonstrating its potential for real-world implementation.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability and access
Not applicable.
Code availability
Not applicable.
References
Yuan X, Yang B, Zhao XD (2023) Observer-based fuzzy adaptive tracking control of nonlinear strict-feedback systems via a unique event triggering approach. IEEE Trans Fuzzy Syst 31(8):2734–2746
Wan P, Zeng ZG (2023) Adaptive tracking control of state-constrained strict-feedback nonlinear systems using direct method. IEEE Trans. Syst. Man Cybern. Syst. 53(8):5116–5126
Li YB, Wang DS, Tang LK (2024) Adaptive fuzzy asymptotic predefined-time tracking control of uncertain nonlinear systems based on event-trigger. Commun. Nonlinear Sci. 137:108144. https://doi.org/10.1016/j.cnsns.2024.108144
Sun K, Ma ZY, Dong GW, Gong P (2023) Adaptive fuzzy fault-tolerant control of uncertain fractional-order nonlinear systems with sensor and actuator faults. Fractal and Fractional. 7(12):862. https://doi.org/10.3390/fractalfract7120862
Gao ZR, Wang ZY, Wu XJ, Wang WC, Liu YL (2020) Finite-time stability analysis of a class of discrete-time switched nonlinear systems with partial finite-time unstable modes. Asian J. Control. 24(1):309–319
Zhao JG, Chen FF (2022) Off-policy integral reinforcement learning-based optimal tracking control for a class of nonzero-sum game systems with unknown dynamics. Optim. Contr. Appl. Met. 43(6):1623–1644
Zhao JG (2023) Adaptive dynamic programming-based adaptive optimal tracking control of a class of strict-feedback nonlinear system. Int J Control Autom Syst 21(4):1349–1360
Wei W, Min W (2024) Adaptive neural event-triggered output-feedback opti-mal tracking control for discrete-time pure-feedback nonlinear systems. International Journal of Network Dynamics and Intelligence 100010
Wen GX, Chen CLP, Ge SS, Yang HL, Liu XG (2019) Optimized adaptive nonlinear tracking control using actor-critic reinforcement learning strategy. IEEE Trans. Ind. Inf. 15(9):4969–4977
Wen GX, Ge SS, Tu FW (2018) Optimized backstepping for tracking control of strict-feedback systems. IEEE Trans. Neural Netw. Learn. Syst. 29(8):3850–3862
Tang L, Zhang L, Xu N (2024) Optimized backstepping based finite-time containment control for nonlinear multi-agent systems with prescribed performance. Optim Contr Appl Met 45(5):2364–2382
Li YL, Fan YL, Li KW, Liu W, Tong SC (2022) Adaptive optimized backstepping control-based RL algorithm for stochastic nonlinear systems with state constraints and its application. IEEE Trans. Cybern. 52(10):10542–10555
Wen GX, Chen CLP (2021) Optimized backstepping consensus control using reinforcement learning for a class of nonlinear strict-feedback-dynamic multi-agent systems. IEEE Trans. Neural Netw. Learn. Syst. PP(3):1524–1536
Li ZJ, Song YF, Wen GX (2023) Reinforcement learning based optimized sliding-mode consensus control of high-order nonlinear canonical dynamic multiagent system. IEEE Syst J 1–10
Wen GX, Hao W, Feng WW, Gao KZ (2022) Optimized backstepping tracking control using reinforcement learning for quadrotor unmanned aerial vehicle system. IEEE Trans. Syst. Man Cybern. Syst. 52(8):5004–5015
Maleki N, Moradi E, Khooban MH (2022) Robust \(h_\infty \) control of dc motor in the presence of input delay and disturbance by the predictor-based method. Sfi. S. Sci. C. 2022(1):1902166. https://doi.org/10.1155/2022/1902166
Wu XJ, Ding S, Xu N, Niu B, Zhao XD (2024) Periodic event-triggered bipartite containment control for nonlinear multi-agent systems with input delay. Int J Syst Sci 55(10):2008–2022
Liu SW, Wang HQ, Li TS (2023) Adaptive composite dynamic surface neural control for nonlinear fractional-order systems subject to delayed input. ISA Trans 134:122–133
Xia XN, Zhang TP, Kang GP, Fang Y (2021) Adaptive event-triggered control of pure-feedback systems with quantized input and unknown input delay. Int. J. Robust Nonlin. 31(18):9074–9093
Zhai JC, Wang HQ, Tao JQ, He ZW (2023) Finite time adaptive neural tracking control for non-strict-feedback uncertain non-linear systems with disturbance and input delay. Trans Inst Meas Control 45(2):374–390. https://doi.org/10.1177/01423312221110437
Li HY, Wang FL (2023) Adaptive output feedback reinforcement learning control for continuous time switched stochastic nonlinear systems with unknown control coefficients and full-state constraints. Int J Syst Sci 55(2):332–354. https://doi.org/10.1080/00207721.2023.2272217
Wang HQ, Liu W, Tong M (2024) Adaptive fuzzy fast finite-time output-feedback tracking control for switched nonlinear systems with full-state constraints. IEEE Trans Fuzzy Syst 32(3):958–968. https://doi.org/10.1109/TFUZZ.2023.3315458
Zhang YX, Liang XL, Li DY, Ge SS, Gao BZ, Chen H, Lee TH (2022) Barrier lyapunov function-based safe reinforcement learning for autonomous vehicles with optimized backstepping. IEEE Trans. Neural Netw. Learn. Syst. PP(2):2066–2080
Wang JF, Zhang P, Wang Y, Ji ZC (2023) Adaptive dynamic programming-based optimal control for nonlinear state constrained systems with input delay. Nonlinear Dynam. 111(20):19133–19149
Zhu HY, Li YX, Tong SC (2023) Fuzzy adaptive tracking of constrained nonlinear systems with event-sampling reinforcement learning. IEEE Trans Fuzzy Syst 1–10
Song XN, Sun P, Ahn CK, Song S (2024) Reinforcement learning-based event-triggered predefined-time optimal fuzzy control for nonlinear constrained systems. IEEE Trans Fuzzy Syst. 1–13
Li DD, Dong JX (2024) Fuzzy weight-based reinforcement learning for event-triggered optimal backstepping control of fractional-order nonlinear systems. IEEE Trans Fuzzy Syst 32(1):214–225
Li DD, Dong JX (2023) Fuzzy control based on reinforcement learning and subsystem error derivatives for strict-feedback systems with an observer. IEEE Trans Fuzzy Syst 31(8):2509–2521
Li DD, Dong JX (2024) Approximate optimal robust tracking control based on state error and derivative without initial admissible input. IEEE Trans. Syst. Man Cybern. Syst. 54(2):1059–1069
Zhao H, Zong GD, Zhao XD, Wang HQ, Xu N, Zhao N (2023) Hierarchical sliding-mode surface-based adaptive critic tracking control for nonlinear multiplayer zero-sum games via generalized fuzzy hyperbolic models. IEEE Trans Fuzzy Syst 31(11):4010–4023
Zhao H, Wang HQ, Niu B, Zhao XD, Alharbi KH (2023) Event-triggered fault-tolerant control for input-constrained nonlinear systems with mismatched disturbances via adaptive dynamic programming. Neural Netw 164:508–520
Chen L, Dai SL, Dong C (2022) Adaptive optimal tracking control of an underactuated surface vessel using actor-critic reinforcement learning. IEEE Trans. Neural Netw. Learn. Syst. PP(6):7520–7533
Zhao H, Wang H, Chang X, Ahmad AM, Zhao X (2024) Neural network-based adaptive critic control for saturated nonlinear systems with full state constraints via a novel event-triggered mechanism. Inform Sci 675:120756
Zhao H, Wang HQ, Niu B, Zhao XD, Xu N (2024) Adaptive fuzzy decentralized optimal control for interconnected nonlinear systems with unmodeled dynamics via mixed data and event driven method. Fuzzy Set Syst 474:108735
Liu SH, Niu B, Xu N, Zhao XD (2024) Zero-sum game-based decentralized optimal control for saturated nonlinear interconnected systems via a data and event driven approach. IEEE Syst J 18(1):758–769
Zhang JX, Fu Y, Fu J (2024) Optimal formation control of second-order heterogeneous multiagent systems using adaptive predefined-time strategy. IEEE Trans Fuzzy Syst 32(4):2390–2402
Wang T, Zong G, Zhao X, Xu N (2024) Data-driven-based sliding-mode dynamic event-triggered control of unknown nonlinear systems via reinforcement learning. Neurocomputing 601:128176
Zhu B, Zhang L, Niu B, Zhao N (2024) Adaptive reinforcement learning for fault-tolerant optimal consensus control of nonlinear canonical multiagent systems with actuator loss of effectiveness. IEEE Syst J, https://doi.org/10.1109/JSYST.2024.3433023
Huang S, Zong GD, Niu B, Xu N, Zhao XD (2024) Dynamic self-triggered fuzzy bipartite time-varying formation tracking for nonlinear multiagent systems with deferred asymmetric output constraints. IEEE Trans Fuzzy Syst 32(5):2700–2712
Wu X, Ding S, Wang H, Xu N, Zhao X, Wang W (2025) Dual-channel event-triggered prescribed performance adaptive fuzzy time-varying formation tracking control for nonlinear multi-agent systems. Fuzzy Sets Syst 498:109140
Zhu B, Liang H, Niu B, Wang H, Zhao N, Zhao X (2025) Observer-based reinforcement learning for optimal fault-tolerant consensus control of nonlinear multi-agent systems via a dynamic event-triggered mechanism. Inform Sci 689:121350
Zheng XY (2023) Adaptive neural control for non-strict feedback stochastic nonlinear systems with input delay. Trans Inst Meas Control 46(1):104–115
Jain AK, Katiyar A (2023) Robust adaptive control design of nonlinear systems with input delay. Int. J. Adapt. Control. 37(5):1193–1202. https://doi.org/10.1002/acs.3570
Wei F, Niu B, Zong G, Zhao X (2024) Adaptive neural self-triggered bipartite consensus control for nonlinear fractional-order multi-agent systems with actuator fault. Nonlinear Dyn, https://doi.org/10.1007/s11071-024-10234-5
Liu MG, Xu N (2024) Adaptive neural predefined-time hierarchical sliding mode control of switched under-actuated nonlinear systems subject to bouc-wen hysteresis. Int J Syst Sci 1–18
Bai WW, Zhou Q, Li TS, Li HY (2020) Adaptive reinforcement learning neural network control for uncertain nonlinear system with input saturation. IEEE Trans. Cybern. 50(8):3433–3443
Huang ZS, Bai WW, Li TS, Long Y, Chen CLP, Liang HJ, Yang HQ (2023) Adaptive reinforcement learning optimal tracking control for strict-feedback nonlinear systems with prescribed performance. Inform Sci 621:407–423
Li DP, Liu YJ, Tong S, Chen CLP, Li DJ (2019) Neural networks-based adaptive control for nonlinear state constrained systems with input delay. IEEE Trans. Cybern. 49(4):1249–1258
Funding
This research was supported by the National Natural Science Foundation of China, under Grant 62203064.
Author information
Authors and Affiliations
Contributions
B.Z. provided methodology, validation, and writing-original draft preparation; H.R.K. and X.Z. provided conceptualization, writing-review; L.Z. provided supervision; X.Z. provided funding support. All authors have read and agreed to the published version of the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no competing interests to declare that are relevant to the content of this article.
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Materials availability
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhu, B., Karimi, H.R., Zhang, L. et al. Neural network-based adaptive reinforcement learning for optimized backstepping tracking control of nonlinear systems with input delay. Appl Intell 55, 129 (2025). https://doi.org/10.1007/s10489-024-05932-x
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-05932-x