Abstract
In this paper, for the multiplayer mixed zero-sum game (MZSG) problem of the constrained-states nonlinear systems with matched input disturbances, an adaptive robust control method without initial stabilizing is presented on account of barrier function (BF) transformation. Firstly, the original system with state constraints is converted to a transformed system without state constraints by barrier function transformation. Secondly, to overcome the influence of matched input disturbances, considering the nominal system related to the transformation system, the cost function corresponding to each player is appropriately selected, and the robust regulation scheme with matched input disturbances is converted to the optimal regulation scheme. In addition, a novel weight tuning law is designed for the critic neural network (NN) by combining the experience replay (ER) mechanism and the index function. Then, the corresponding cost function of each player is approximated by the critic NN without requiring initial stabilizing control. Utilizing the Lyapunov stability theory, under the influence of state constraints and matched input disturbances, the critic NN weights and states within the multiplayer system are ensured to be uniformly ultimately bounded (UUB). Ultimately, the validity of the proposed method is verified by two simulation examples.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
The authors can confirm that all relevant data are included in the article.
References
Nguyen Q, Sreenath K (2022) Robust safety-critical control for dynamic robotics. IEEE Trans Automatic Control 67(3):1073–1088
Chen Y, Grizzle J (2018) Obstacle avoidance for low-speed autonomous vehicles with barrier function. IEEE Trans Control Syst Technol 26(1):194–206
Shaw C, Oetomo D, Manzie C, Choong P (2021) Control barrier functions for mechanical systems: Theory and application to robotic grasping. IEEE Trans Control Syst Technol 29(2):530–545
Tee KP, Ge SS, Tay EH (2009) Barrier lyapunov functions for the control of output-constrained nonlinear systems. Automatica 45(4):918–927
Qin C, Zhang Z, Shang Z, Zhang J, Zhang D (2023) Adaptive optimal safety tracking control for multiplayer mixed zero-sum games of continuous-time systems. Appl Intell 53:17460–17475
Ames AD, Xu X, Grizzle JW, Tabuada P (2017) Barrier lyapunov functions for the control of output-constrained nonlinear systems. IEEE Trans Automatic Control 62(8):3861–3876
Lv Y, Ren X (2019) Approximate nash solutions for multiplayer mixed zero-sum game with reinforcement learning. IEEE Trans Syst Man, Cybernetics: Syst 49(12):2739–2750
Wang D, Zhao M, Ha M, Qiao J (2023) Stability and admissibility analysis for zero-sum games under general value iteration formulation. IEEE Trans Neural Netw Learn Syst 34(11):8707–8718
Qin C, Zhang H, Luo Y (2016) Neural network-based online \( {H}_{ {\infty }}\) control for discrete-time affine nonlinear system using adaptive dynamic programming. Neurocomputing 198:91–99
Zhang H, Cui L, Luo Y (2013) Near-optimal control for nonzero-sum differential games of continuous-time nonlinear systems using single-network adp. IEEE Trans Cybernetics 43(1):206–216
Liu D, Xue X, Zhao B, Luo B, Wei Q (2021) Adaptive dynamic programming for control: A survey and recent advances. IEEE Trans Syst Man, Cybernetics: Syst 51(1):142–160
Wan T, Zhang H, Luo Y (2018) Stochastic linear quadratic optimal control for model-free discrete-time systems based on q-learning algorithm. Neurocomputing 312:1–8
Xue S, Luo B, Liu D (2020) Event-triggered adaptive dynamic programming for zero-sum game of partially unknown continuous-time nonlinear systems. IEEE Trans Neural Netw Learn Syst 50(9):3189–3199
Huo Y, Wang D, Qiao J, Li M (2023) Adaptive critic design for nonlinear multi-player zero-sum games with unknown dynamics and control constraints. Nonlinear Dynamics 111:11671–11683
Hou J, Wang D, Liu D, Zhao D, Zhang Y (2020) Model-free \( {H}_{ {\infty }}\) optimal tracking control of constrained nonlinear systems via an iterative adaptive learning algorithm. IEEE Trans Syst Man, Cybernetics: Syst 50(11):4097–4108
Yang X, He H, Liu D (2019) Event-triggered optimal neuro-controller design with reinforcement learning for unknown nonlinear systems. IEEE Trans Syst, Man, Cybernetics 49(9):1866–1878
Wang K, Mu C (2022) Asynchronous learning for actor–critic neural networks and synchronous triggering for multiplayer system. ISA Trans 129:295–308
Lin H, Zhao B, Liu D, Liu D, Alippi C (2020) Data-based fault tolerant control for affine nonlinear systems through particle swarm optimized neural networks. IEEE/CAA J Automatica Sinica 7(4):954–964
Sun T, Sun X (2021) An adaptive dynamic programming scheme for nonlinear optimal control with unknown dynamics and its application to turbofan engines. IEEE Trans Industrial Inf 17(1):367–376
El-Sousy FFM, Amin MM, Al-Durra A (2021) Adaptive optimal tracking control via actor-critic-identifier based adaptive dynamic programming for permanent-magnet synchronous motor drive system. IEEE Trans Industry Appl 57(6):6577–6591
Wang D, Mu C (2018) Adaptive-critic-based robust trajectory tracking of uncertain dynamics and its application to a spring-mass-damper system. IEEE Trans Industrial Electron 65(1):654–663
Yang X, He H, Zhong X (2018) Adaptive dynamic programming for robust regulation and its application to power systems. IEEE Trans Industrial Electron 65(7):5722–5732
Wang D, He H, Zhong X, Liu D (2017) Event-driven nonlinear discounted optimal regulation involving a power system application. IEEE Trans Industrial Electron 64(10):8177–8186
Yang X, Xu M, Wei Q (2023) Adaptive dynamic programming for nonlinear-constrained \({H}_{ {\infty }}\) control. IEEE Trans Syst, Man, Cybernetics: Syst 53(7):4393–4403
Wu N, Luo B (2012) Neural network based online simultaneous policy update algorithm for solving the HJI equation in nonlinear \({H}_{ {\infty }}\) control. IEEE Trans Neural Netw Learn Syst 23(12):1884–1895
Ren H, Jiang B, Ma Y (2024) Zero-sum differential game-based fault-tolerant control for a class of affine nonlinear systems. IEEE Trans Cybernetics 54(2):1272–1282
Zhao D, Zhang Q, Wang D, Zhu Y (2016) Experience replay for optimal control of nonzero-sum game systems with unknown dynamics. IEEE Trans Cybernetics 46(3):854–865
Zhang H, Zhang K, Luo Y (2019) Event-triggered adaptive dynamic programming algorithm for non-zero-sum games of unknown nonlinear systems via generalized fuzzy hyperbolic models. IEEE Trans Fuzzy Syst 27(11):2202–2214
Zhang Q, Zhao D (2019) Data-based reinforcement learning for nonzero-sum games with unknown drift dynamics. IEEE Trans Cybernetics 49(8):2874–2885
Qiao J, Li M, Wang D (2024) Asymmetric constrained optimal tracking control with critic learning of nonlinear multiplayer zero-sum games. IEEE Trans Neural Netw Learn Syst 35(4):3081–3092
Xu S, He B, Zhang X, Luo Y (2023) Robust adaptive fuzzy fault tolerant control of robot manipulators with unknown parameters. IEEE Trans Fuzzy Syst 31(9):190–195
Yang X, Zhou Y, Gao Z (2023) Reinforcement learning for robust stabilization of nonlinear systems with asymmetric saturating actuators. Neural Netw 158:132–141
Wei Q, Song R, Yan P (2016) Data-driven zero-sum neuro-optimal control for a class of continuous-time unknown nonlinear systems with disturbance using adp. IEEE Trans Neural Netw Learn Syst 27(2):444–458
Vamvoudakis KG, Lewis FL (2011) Multi-player non-zero-sum games: Online adaptive learning solution of coupled hamilton–jacobi equations. Automatica 47(8):1556–1569
Qu Q, Zhang H, Luo C, Yu R (2019) Robust control design for multi-player nonlinear systems with input disturbances via adaptive dynamic programming. Neurocomputing 334:1–10
Zhang Y, Zhao B, Liu D, Zhang S (2023) Adaptive dynamic programming-based event-triggered robust control for multiplayer nonzero-sum games with unknown dynamics. IEEE Trans Cybernetics 53(8):5151–5164
Cui X, Zhang H, Luo Y, Zu P (2016) Online finite-horizon optimal learning algorithm for nonzero-sum games with partially unknown dynamics and constrained inputs. Neurocomputing 185:37–44
Yang Y, Vamvoudakis KG, Modares H (2020) Safe reinforcement learning for dynamical games. Int J Robust Nonlinear Control 30(9):3706–3726
Yang Y, Ding D, Xiong H, Yin Y, Wunsch DC (2020) Online barrier-actor-critic learning for \({H}_{ {\infty }}\) control with full-state constraints and input saturation. J Franklin Institute 357(6):3316–3344
Yang Y, Yin Y, He W, Vamvoudakis KG, Modares H, Wunsch DC (2019) Safety-aware reinforcement learning framework with an actor-critic-barrier structure. American Control Conference (ACC), 2352–2358
Xu J, Wang J, Rao J, Zhong Y, Wang H (2022) Adaptive dynamic programming for optimal control of discrete-time nonlinear system with state constraints based on control barrier function. Int J Robust Nonlinear Control 32(6):3408–3424
Qin C, Wang J, Zhu H, Zhang J, Hu S, Zhang D (2022) Neural network-based safe optimal robust control for affine nonlinear systems with unmatched disturbances. Neurocomputing 506:228–239
Qin C, Wang J, Zhu H, Xiao Q, Zhang D (2022) Safe adaptive learning algorithm with neural network implementation for \({H}_{ {\infty }}\) control of nonlinear safety-critical system. Int J Robust Nonlinear Control 33(1):372–391
Acknowledgements
This work was supported by science and technology research project of the Henan province (222102240014).
Author information
Authors and Affiliations
Contributions
X.Q. and C.Q. provided methodology, validation, and writing-original draft preparation; J.W. provided conceptualization, writing-review; Z.Z. and Z.S. provided supervision; C.Q. provided funding support. All authors have read and agreed to the published version of the manuscript.
Corresponding author
Ethics declarations
Conflicts of Interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Qiao, X., Qin, C., Wang, J. et al. Adaptive robust control without initial stabilizing for constrained-states nonlinear multiplayer mixed zero-sum game systems with matched input disturbances. Appl Intell 55, 136 (2025). https://doi.org/10.1007/s10489-024-05980-3
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-05980-3