Skip to main content

Advertisement

Adaptive robust control without initial stabilizing for constrained-states nonlinear multiplayer mixed zero-sum game systems with matched input disturbances

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In this paper, for the multiplayer mixed zero-sum game (MZSG) problem of the constrained-states nonlinear systems with matched input disturbances, an adaptive robust control method without initial stabilizing is presented on account of barrier function (BF) transformation. Firstly, the original system with state constraints is converted to a transformed system without state constraints by barrier function transformation. Secondly, to overcome the influence of matched input disturbances, considering the nominal system related to the transformation system, the cost function corresponding to each player is appropriately selected, and the robust regulation scheme with matched input disturbances is converted to the optimal regulation scheme. In addition, a novel weight tuning law is designed for the critic neural network (NN) by combining the experience replay (ER) mechanism and the index function. Then, the corresponding cost function of each player is approximated by the critic NN without requiring initial stabilizing control. Utilizing the Lyapunov stability theory, under the influence of state constraints and matched input disturbances, the critic NN weights and states within the multiplayer system are ensured to be uniformly ultimately bounded (UUB). Ultimately, the validity of the proposed method is verified by two simulation examples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The authors can confirm that all relevant data are included in the article.

References

  1. Nguyen Q, Sreenath K (2022) Robust safety-critical control for dynamic robotics. IEEE Trans Automatic Control 67(3):1073–1088

    MathSciNet  MATH  Google Scholar 

  2. Chen Y, Grizzle J (2018) Obstacle avoidance for low-speed autonomous vehicles with barrier function. IEEE Trans Control Syst Technol 26(1):194–206

    MATH  Google Scholar 

  3. Shaw C, Oetomo D, Manzie C, Choong P (2021) Control barrier functions for mechanical systems: Theory and application to robotic grasping. IEEE Trans Control Syst Technol 29(2):530–545

    Google Scholar 

  4. Tee KP, Ge SS, Tay EH (2009) Barrier lyapunov functions for the control of output-constrained nonlinear systems. Automatica 45(4):918–927

    MathSciNet  MATH  Google Scholar 

  5. Qin C, Zhang Z, Shang Z, Zhang J, Zhang D (2023) Adaptive optimal safety tracking control for multiplayer mixed zero-sum games of continuous-time systems. Appl Intell 53:17460–17475

    MATH  Google Scholar 

  6. Ames AD, Xu X, Grizzle JW, Tabuada P (2017) Barrier lyapunov functions for the control of output-constrained nonlinear systems. IEEE Trans Automatic Control 62(8):3861–3876

    MathSciNet  MATH  Google Scholar 

  7. Lv Y, Ren X (2019) Approximate nash solutions for multiplayer mixed zero-sum game with reinforcement learning. IEEE Trans Syst Man, Cybernetics: Syst 49(12):2739–2750

    MATH  Google Scholar 

  8. Wang D, Zhao M, Ha M, Qiao J (2023) Stability and admissibility analysis for zero-sum games under general value iteration formulation. IEEE Trans Neural Netw Learn Syst 34(11):8707–8718

    MathSciNet  MATH  Google Scholar 

  9. Qin C, Zhang H, Luo Y (2016) Neural network-based online \( {H}_{ {\infty }}\) control for discrete-time affine nonlinear system using adaptive dynamic programming. Neurocomputing 198:91–99

    MATH  Google Scholar 

  10. Zhang H, Cui L, Luo Y (2013) Near-optimal control for nonzero-sum differential games of continuous-time nonlinear systems using single-network adp. IEEE Trans Cybernetics 43(1):206–216

    MATH  Google Scholar 

  11. Liu D, Xue X, Zhao B, Luo B, Wei Q (2021) Adaptive dynamic programming for control: A survey and recent advances. IEEE Trans Syst Man, Cybernetics: Syst 51(1):142–160

    MATH  Google Scholar 

  12. Wan T, Zhang H, Luo Y (2018) Stochastic linear quadratic optimal control for model-free discrete-time systems based on q-learning algorithm. Neurocomputing 312:1–8

    MATH  Google Scholar 

  13. Xue S, Luo B, Liu D (2020) Event-triggered adaptive dynamic programming for zero-sum game of partially unknown continuous-time nonlinear systems. IEEE Trans Neural Netw Learn Syst 50(9):3189–3199

    MATH  Google Scholar 

  14. Huo Y, Wang D, Qiao J, Li M (2023) Adaptive critic design for nonlinear multi-player zero-sum games with unknown dynamics and control constraints. Nonlinear Dynamics 111:11671–11683

  15. Hou J, Wang D, Liu D, Zhao D, Zhang Y (2020) Model-free \( {H}_{ {\infty }}\) optimal tracking control of constrained nonlinear systems via an iterative adaptive learning algorithm. IEEE Trans Syst Man, Cybernetics: Syst 50(11):4097–4108

    MATH  Google Scholar 

  16. Yang X, He H, Liu D (2019) Event-triggered optimal neuro-controller design with reinforcement learning for unknown nonlinear systems. IEEE Trans Syst, Man, Cybernetics 49(9):1866–1878

    MATH  Google Scholar 

  17. Wang K, Mu C (2022) Asynchronous learning for actor–critic neural networks and synchronous triggering for multiplayer system. ISA Trans 129:295–308

    MATH  Google Scholar 

  18. Lin H, Zhao B, Liu D, Liu D, Alippi C (2020) Data-based fault tolerant control for affine nonlinear systems through particle swarm optimized neural networks. IEEE/CAA J Automatica Sinica 7(4):954–964

    MathSciNet  MATH  Google Scholar 

  19. Sun T, Sun X (2021) An adaptive dynamic programming scheme for nonlinear optimal control with unknown dynamics and its application to turbofan engines. IEEE Trans Industrial Inf 17(1):367–376

    MATH  Google Scholar 

  20. El-Sousy FFM, Amin MM, Al-Durra A (2021) Adaptive optimal tracking control via actor-critic-identifier based adaptive dynamic programming for permanent-magnet synchronous motor drive system. IEEE Trans Industry Appl 57(6):6577–6591

    MATH  Google Scholar 

  21. Wang D, Mu C (2018) Adaptive-critic-based robust trajectory tracking of uncertain dynamics and its application to a spring-mass-damper system. IEEE Trans Industrial Electron 65(1):654–663

    MathSciNet  MATH  Google Scholar 

  22. Yang X, He H, Zhong X (2018) Adaptive dynamic programming for robust regulation and its application to power systems. IEEE Trans Industrial Electron 65(7):5722–5732

    MATH  Google Scholar 

  23. Wang D, He H, Zhong X, Liu D (2017) Event-driven nonlinear discounted optimal regulation involving a power system application. IEEE Trans Industrial Electron 64(10):8177–8186

    MATH  Google Scholar 

  24. Yang X, Xu M, Wei Q (2023) Adaptive dynamic programming for nonlinear-constrained \({H}_{ {\infty }}\) control. IEEE Trans Syst, Man, Cybernetics: Syst 53(7):4393–4403

    MATH  Google Scholar 

  25. Wu N, Luo B (2012) Neural network based online simultaneous policy update algorithm for solving the HJI equation in nonlinear \({H}_{ {\infty }}\) control. IEEE Trans Neural Netw Learn Syst 23(12):1884–1895

    MATH  Google Scholar 

  26. Ren H, Jiang B, Ma Y (2024) Zero-sum differential game-based fault-tolerant control for a class of affine nonlinear systems. IEEE Trans Cybernetics 54(2):1272–1282

    MATH  Google Scholar 

  27. Zhao D, Zhang Q, Wang D, Zhu Y (2016) Experience replay for optimal control of nonzero-sum game systems with unknown dynamics. IEEE Trans Cybernetics 46(3):854–865

    MATH  Google Scholar 

  28. Zhang H, Zhang K, Luo Y (2019) Event-triggered adaptive dynamic programming algorithm for non-zero-sum games of unknown nonlinear systems via generalized fuzzy hyperbolic models. IEEE Trans Fuzzy Syst 27(11):2202–2214

    MATH  Google Scholar 

  29. Zhang Q, Zhao D (2019) Data-based reinforcement learning for nonzero-sum games with unknown drift dynamics. IEEE Trans Cybernetics 49(8):2874–2885

    MATH  Google Scholar 

  30. Qiao J, Li M, Wang D (2024) Asymmetric constrained optimal tracking control with critic learning of nonlinear multiplayer zero-sum games. IEEE Trans Neural Netw Learn Syst 35(4):3081–3092

    MathSciNet  MATH  Google Scholar 

  31. Xu S, He B, Zhang X, Luo Y (2023) Robust adaptive fuzzy fault tolerant control of robot manipulators with unknown parameters. IEEE Trans Fuzzy Syst 31(9):190–195

    MATH  Google Scholar 

  32. Yang X, Zhou Y, Gao Z (2023) Reinforcement learning for robust stabilization of nonlinear systems with asymmetric saturating actuators. Neural Netw 158:132–141

    MATH  Google Scholar 

  33. Wei Q, Song R, Yan P (2016) Data-driven zero-sum neuro-optimal control for a class of continuous-time unknown nonlinear systems with disturbance using adp. IEEE Trans Neural Netw Learn Syst 27(2):444–458

    MathSciNet  MATH  Google Scholar 

  34. Vamvoudakis KG, Lewis FL (2011) Multi-player non-zero-sum games: Online adaptive learning solution of coupled hamilton–jacobi equations. Automatica 47(8):1556–1569

    MathSciNet  MATH  Google Scholar 

  35. Qu Q, Zhang H, Luo C, Yu R (2019) Robust control design for multi-player nonlinear systems with input disturbances via adaptive dynamic programming. Neurocomputing 334:1–10

    MATH  Google Scholar 

  36. Zhang Y, Zhao B, Liu D, Zhang S (2023) Adaptive dynamic programming-based event-triggered robust control for multiplayer nonzero-sum games with unknown dynamics. IEEE Trans Cybernetics 53(8):5151–5164

    MATH  Google Scholar 

  37. Cui X, Zhang H, Luo Y, Zu P (2016) Online finite-horizon optimal learning algorithm for nonzero-sum games with partially unknown dynamics and constrained inputs. Neurocomputing 185:37–44

    MATH  Google Scholar 

  38. Yang Y, Vamvoudakis KG, Modares H (2020) Safe reinforcement learning for dynamical games. Int J Robust Nonlinear Control 30(9):3706–3726

    MathSciNet  MATH  Google Scholar 

  39. Yang Y, Ding D, Xiong H, Yin Y, Wunsch DC (2020) Online barrier-actor-critic learning for \({H}_{ {\infty }}\) control with full-state constraints and input saturation. J Franklin Institute 357(6):3316–3344

    MathSciNet  MATH  Google Scholar 

  40. Yang Y, Yin Y, He W, Vamvoudakis KG, Modares H, Wunsch DC (2019) Safety-aware reinforcement learning framework with an actor-critic-barrier structure. American Control Conference (ACC), 2352–2358

  41. Xu J, Wang J, Rao J, Zhong Y, Wang H (2022) Adaptive dynamic programming for optimal control of discrete-time nonlinear system with state constraints based on control barrier function. Int J Robust Nonlinear Control 32(6):3408–3424

    MathSciNet  MATH  Google Scholar 

  42. Qin C, Wang J, Zhu H, Zhang J, Hu S, Zhang D (2022) Neural network-based safe optimal robust control for affine nonlinear systems with unmatched disturbances. Neurocomputing 506:228–239

    MATH  Google Scholar 

  43. Qin C, Wang J, Zhu H, Xiao Q, Zhang D (2022) Safe adaptive learning algorithm with neural network implementation for \({H}_{ {\infty }}\) control of nonlinear safety-critical system. Int J Robust Nonlinear Control 33(1):372–391

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by science and technology research project of the Henan province (222102240014).

Author information

Authors and Affiliations

Authors

Contributions

X.Q. and C.Q. provided methodology, validation, and writing-original draft preparation; J.W. provided conceptualization, writing-review; Z.Z. and Z.S. provided supervision; C.Q. provided funding support. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Chunbin Qin.

Ethics declarations

Conflicts of Interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qiao, X., Qin, C., Wang, J. et al. Adaptive robust control without initial stabilizing for constrained-states nonlinear multiplayer mixed zero-sum game systems with matched input disturbances. Appl Intell 55, 136 (2025). https://doi.org/10.1007/s10489-024-05980-3

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-05980-3

Keywords