Skip to main content
SpringerLink
Log in

Event-triggered adaptive dynamic programming for multi-player zero-sum games with unknown dynamics

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

In this paper, a novel event-triggered optimal control approach is developed to solve zero-sum game problems for continuous-time multi-player nonlinear systems with unknown dynamics. To begin with, a model neural network (NN) is employed to reconstruct the unknown multi-player nonlinear system by measured input and output data. Then, a critic NN is used to solve the event-triggered Hamilton–Jacobi–Isaacs (HJI) equation for multi-player zero-sum game. Meanwhile, the optimal control law and the worst disturbance law are approximated with the help of critic NN only, respectively. Compared with time-triggered method, the developed control law and the disturbance law are updated only when the triggering condition is violated; thus, the computational and communication burden are reduced. The Lyapunov stability analysis shows that the closed-loop system can be guaranteed to be stable. Finally, two simulation examples are provided to validate the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  • Aliyu MDS (2018) An iterative relaxation approach to the solution of the Hamilton-Jacobi–Bellman–Isaacs equation in nonlinear optimal control. IEEE/CAA J Automatica Sinica 5(1):360–366

    Article  MathSciNet  Google Scholar 

  • Dong L, Zhong X, Sun C, He H (2017) Event-triggered adaptive dynamic programming for continuous-time systems with control constraints. IEEE Trans Neural Netw Learn Syst 28(8):1941–1952

    Article  MathSciNet  Google Scholar 

  • Fu X, Chai T (2016) Online solution of two-player zero-sum games for continuous-time nonlinear systems with completely unknown dynamics. IEEE Trans Neural Netw Learn Syst 27(12):2577–2587

    Article  MathSciNet  Google Scholar 

  • Jiang H, Zhang H (2018) Iterative ADP learning algorithms for discrete-time multi-player games. Artif Intell Rev 50(1):75–91

    Article  Google Scholar 

  • Jiang H, Zhang H, Han J (2018) Iterative adaptive dynamic programming methods with neural network implementation for multi-player zero-sum games. Neurocomputing 307:54–60

    Article  Google Scholar 

  • Liu D, Li H, Wang D (2014) Online synchronous approximate optimal learning algorithm for multi-player non-zero-sum games with unknown dynamics. IEEE Trans Syst Man Cybern 44(8):1015–1027

    Article  Google Scholar 

  • Liu D, Wei Q, Wang D (2017) Adaptive dynamic programming with applications in optimal control. Springer, Cham

    Book  Google Scholar 

  • Liu D, Xu Y, Wei Q, Liu X (2018) Residential energy scheduling for variable weather solar energy based on adaptive dynamic programming. IEEE/CAA J Automatica Sinica 5(1):36–46

    Article  Google Scholar 

  • Luo B, Yang Y, Liu D, Wu H (2019) Event-triggered optimal control with performance guarantees using adaptive dynamic programming. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2019.2899594

    Article  Google Scholar 

  • Song R, Wei Q, Song B (2017) Neural-network-based synchronous iteration learning method for multi-player zero-sum games. Neurocomputing 242:73–82

    Article  Google Scholar 

  • Wang D, Mu C, Liu D, Ma H (2018b) On mixed data and event driven design for adaptive-critic-based nonlinear \(H_{\infty }\) control. IEEE Trans Neural Netw Learn Syst 29(4):993–1005

    Article  Google Scholar 

  • Wang B, Zhao D, Cheng J (2019) Adaptive cruise control via adaptive dynamic programming with experience replay. Soft Comput 23(12):4131–4144

    Article  Google Scholar 

  • Wei Q, Liu D, Xu Y (2016) Neuro-optimal tracking control for a class of discrete-time nonlinear systems via generalized value iteration adaptive dynamic programming approach. Soft Comput 20(2):697–706

    Article  Google Scholar 

  • Wei Q, Liu D, Liu Y, Song R (2017a) Optimal constrained self-learning battery sequential management in microgrid via adaptive dynamic programming. IEEE/CAA J Automatica Sinica 4(2):168–176

    Article  MathSciNet  Google Scholar 

  • Xue S, Luo B, Liu D (2018) Event-triggered adaptive dynamic programming for zero-sum game of partially unknown continuous-time nonlinear systems. IEEE Trans Syst Man Cybern. https://doi.org/10.1109/TSMC.2018.2852810

    Article  Google Scholar 

  • Yang X, Liu D, Wang D (2013) Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints. Int J Control 87(3):553–566

    Article  MathSciNet  Google Scholar 

  • Zhang Q, Zhao D (2019) Data-based reinforcement learning for nonzero-sum games with unknown drift dynamics. IEEE Trans Cybern 49(8):2874–2885

    Article  Google Scholar 

  • Zhang H, Su H, Zhang K, Luo Y (2019) Event-triggered adaptive dynamic programming algorithm for non-zero-sum games of unknown nonlinear systems via generalized fuzzy hyperbolic models. IEEE Trans Fuzzy Syst. https://doi.org/10.1109/TFUZZ.2019.2896544

    Article  Google Scholar 

  • Zhao B, Liu D (2019) Event-triggered decentralized tracking control of modular reconfigurable robots through adaptive dynamic programming. IEEE Trans Ind Electron. https://doi.org/10.1109/TIE.2019.2914571

    Article  Google Scholar 

  • Zhao D, Zhang Q, Wang D, Zhu Y (2016) Experience replay for optimal control of nonzero-sum game systems with unknown dynamics. IEEE Trans Cybern 46(3):854–865

    Article  Google Scholar 

  • Zhao B, Jia L, Xia H, Li Y (2018a) Adaptive dynamic programming-based stabilization of nonlinear systems with unknown actuator saturation. Nonlinear Dyn 93(4):2089–2103

    Article  Google Scholar 

  • Zhao B, Wang D, Shi G, Liu D, Li Y (2018b) Decentralized control for large-scale nonlinear systems with unknown mismatched interconnections via policy iteration. IEEE Trans Syst Man Cybern 48(10):1725–1735

    Article  Google Scholar 

  • Zhong X, He H, Wang D, Ni Z (2018) Model-free adaptive control for unknown nonlinear zero-sum differential game. IEEE Trans Cybern 48(5):1633–1646

    Article  Google Scholar 

  • Zhu Y, Zhao D (2015) A data-based online reinforcement learning algorithm satisfying probably approximately correct principle. Neural Comput Appl 26(4):775–787

    Article  Google Scholar 

  • Zhu Y, Zhao D (2018) Comprehensive comparison of online ADP algorithms for continuous-time optimal control. Artif Intell Rev 49(4):531–547

    Article  Google Scholar 

  • Zhu Y, Zhao D, Li X (2017) Iterative adaptive dynamic programming for solving unknown nonlinear zero-sum game based on online data. IEEE Trans Neural Netw Learn Syst 28(3):714–725

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grants 61533017, 61973330, 61773075 and 61603387, in part by the Early Career Development Award of SKLMCCS under Grant 20180201, and in part by the State Key Laboratory of Synthetical Automation for Process Industries under Grant 2019-KF-23-03.

Author information

Authors and Affiliations

  1. School of Automation, Guangdong University of Technology, Guangzhou, 510006, China

    Yongwei Zhang & Derong Liu

  2. School of Systems Science, Beijing Normal University, Beijing, 100875, China

    Bo Zhao

Authors
  1. Yongwei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bo Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Derong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Zhao.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Communicated by V. Loia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Zhao, B. & Liu, D. Event-triggered adaptive dynamic programming for multi-player zero-sum games with unknown dynamics. Soft Comput 25, 2237–2251 (2021). https://doi.org/10.1007/s00500-020-05293-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-020-05293-w

Keywords

Search

Navigation