Skip to main content
SpringerLink
Log in

Adaptive dynamic programming-based optimal regulation on input-constrained nonlinear time-delay systems

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The adaptive dynamic programming (ADP)-based optimal regulation strategy is put forward for input-constrained nonlinear time-delay systems. In the spirit of Lyapunov theories, the stability of the nominal system is investigated in terms of linear matrix inequalities (LMIs), which consequently gives rise to sufficient delay-dependent stability conditions. Afterward, a single neural network (NN) which serves as critic and actor NN simultaneously is employed for the realization of ADP-based optimal regulation. The NN weights are updated in real-time and the weight estimate errors are proved to be convergent. As a result, computational complexity is efficiently decreased together with the storage space. Numerical simulation shows the validation of our approach.Kindly check and confirm the inserted city name is correct. Amend if necessary.CorrectKindly check and confirm the Organization division and Organization name of Affiliation 2.Correcy

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Angelov VG (1988) A converse to a contraction mapping theorem in uniform spaces. Nonlinear Anal Theory Methods Appl 12(10):989–996

    Article  MathSciNet  Google Scholar 

  2. Bensoussan A (1983) Maximum principle and dynamic programming approaches of the optimal control of partially observed diffusions. Stoch Int J Prob Stoch Process 9(3):169–222

    MathSciNet  MATH  Google Scholar 

  3. Branicky MS, Borkar VS, Mitter SK (1998) A unified framework for hybrid control: model and optimal control theory. IEEE Trans Autom Control 43(1):31–45

    Article  MathSciNet  Google Scholar 

  4. Cao YY, Lin Z (2003) Stability analysis of discrete-time systems with actuator saturation by a saturation-dependent lyapunov function. Automatica 39(7):1235–1241

    Article  MathSciNet  Google Scholar 

  5. Chen M, Ge SS, Ren B (2011) Adaptive tracking control of uncertain mimo nonlinear systems with input constraints. Automatica 47(3):452–465

    Article  MathSciNet  Google Scholar 

  6. Gopalsamy K (1992) Stability and oscillations in delay differential equations of population dynamics. Springer, Netherlands

    Book  Google Scholar 

  7. Gu K, Kharitonov V, Chen J (2003) Stability of time-delay systems. Bitkhäuser, Boston

    Book  Google Scholar 

  8. Himmelberg C, Parthasarathy T, VanVleck F (1976) Optimal plans for dynamic programming problems. Math Op Res 1(4):390–394

    Article  MathSciNet  Google Scholar 

  9. Huang Y, Lu WM (1996) Nonlinear optimal control: Alternatives to Hamilton-Jacobi equation. In: Proceedings of 35th IEEE conference on decision and control, vol. 4, pp 3942–3947

  10. Jia Q, Bram AK, Han Z(2021) Synchronization of drive-response networks with event-based pinning control. Neural Comp Appl (to appear)

  11. Kamalapurkar R, Rosenfeld J, Dixon W (2016) Efficient model-based reinforcement learning for approximate online optimal control. Automatica 74:247–258

    Article  MathSciNet  Google Scholar 

  12. Kurtz MJ, Henson MA (1998) Feedback linearizing control of discrete-time nonlinear systems with input constraints. Int J Control 70(4):603–616

    Article  MathSciNet  Google Scholar 

  13. Li C, Huang T (2009) On the stability of nonlinear systems with leakage delay - sciencedirect. J Frankl Inst 346(4):366–377

    Article  Google Scholar 

  14. Li Y (2000) Optimal guaranteed cost control of linear uncertain system: an LMI approach. Control Theory Appl 17(3):423–428

    MathSciNet  MATH  Google Scholar 

  15. Liu D, Wei Q, Wang D, Yang X, Li H (2017) Value iteration ADP for discrete-time nonlinear systems. In: Advances in industrial control] Adaptive dynamic programming with applications in optimal control, pp 37–90. Springer

  16. Liu D, Wei Q, Yang X, Li H, Wang D (2017) Adaptive dynamic programming with applications in optimal control. Springer, Berlin

    Book  Google Scholar 

  17. Manousiouthakis V, Chmielewski DJ (2002) On constrained infinite-time nonlinear optimal control. Chem Eng Sci 57(1):105–114

    Article  Google Scholar 

  18. Rodriguez-Guerrero L, Santos-Sanchez O, Mondie S (2016) A constructive approach for an optimal control applied to a class of nonlinear time delay systems. J Process Control 40:35–49

    Article  Google Scholar 

  19. Ross IM, Karpenko M (2012) A review of pseudospectral optimal control: from theory to flight. Ann Rev Control 36(2):182–197

    Article  Google Scholar 

  20. Saberi A, Lin Z, Teel AR (1996) Control of linear systems with saturating actuators. IEEE Trans Autom Control 41(3):368–378

    Article  MathSciNet  Google Scholar 

  21. Serban I, Sankar C, Pieper M, Pineau J, Bengio Y (2018) The Bottleneck simulator: a model-based deep reinforcement learning approach

  22. Seuret A, Gouaisbaut F, Fridman E (2013) Stability of systems with fast-varying delay using improved wirtinger’s inequality. In: Proceedings of 52nd IEEE Conference on Decision and Control, pp 946–951

  23. Shen J, Lam J (2021) On the algebraic riccati inequality arising in cone-preserving time-delay systems. Automatica (to appear)

  24. Song R, Xiao W, Zhang H (2014) Adaptive dynamic programming for a class of complex-valued nonlinear systems. IEEE Trans Neural Netw Learn Syst 25(9):1733–1739

    Article  Google Scholar 

  25. Vamvoudakis KG, Lewis FL (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5):878–888

    Article  MathSciNet  Google Scholar 

  26. Wei Q, Liu D, Liu D (2013) Numerical adaptive learning control scheme for discrete-time non-linear systems. IET Control Theory Appl 7(11):1472–1486

    Article  MathSciNet  Google Scholar 

  27. Wu Z, Li Q, Wu W, Zhao M (2020) Crowdsourcing model for energy efficiency retrofit and mixed-integer equilibrium analysis. IEEE Trans Ind Inf 16(7):4512–4524

    Article  Google Scholar 

  28. Zhan C, Li B, Zhong X et al (2020) A model for collective behaviour propagation: a case study of video game industry. Neural Comput Appl 32(4):4507–4517

    Article  Google Scholar 

  29. Zhan C, Wu F, Huang Z et al (2020) Analysis of collective action propagation with multiple recurrences. Neural Comput Appl 32(4):13491–13504

    Article  Google Scholar 

  30. Zhu J, Hou Y, Li T (2019) Optimal control of nonlinear systems with time delays: an online adp perspective. IEEE Access 7:145574–145581

    Article  Google Scholar 

  31. Zhu J, Qi T, Ma D, Chen J (2018) Limits of stability and stabilization of time-delay systems: a small gain approach. Springer, Berlin

    Book  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China

    Jing Zhu & Peng Zhang

  2. Key Laboratory of Navigation, Control and Health-Management Technologies of Advanced Aerocraft, Ministry of Industry and information Technology, Nanjing Univ. of Aeronautics and Astronautics, Nanjing, China

    Jing Zhu

  3. State Wuhu Machine Factory, Wuhu, China

    Yijing Hou

Authors
  1. Jing Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yijing Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Zhu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This research was supported in part by the China Postdoctoral Science Foundation under Grant 2019T120427, in part by the Fundamental Research Funds for the Central Universities under Grant NS2020023, and in part by the Macao Young Scholars Program under Grant AM2020006.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, J., Zhang, P. & Hou, Y. Adaptive dynamic programming-based optimal regulation on input-constrained nonlinear time-delay systems. Neural Comput & Applic 33, 13039–13047 (2021). https://doi.org/10.1007/s00521-021-06000-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06000-y

Keywords

Search

Navigation