Abstract
A neural network (NN) optimal control for discrete-time nonlinear dynamic systems with known internal dynamics is designed. The control is described by an algebraic equation with a variable structure. This algebraic equation is derived analytically. A functional block diagram of the controlled system is given and analyzed. Software and hardware implementation aspects of the controller are discussed. The controller does not need any training and has moderate complexity. The discrete-time state variable trajectories of the controlled system are shown to be globally asymptotically stable and convergent to unique steady states. It is proved that these trajectories converge to steady-state neighborhood in a finite number of steps. Sliding mode analysis of controller operation is fulfilled. A correctness of controller operation in the case of disturbances of its nonlinearities is analyzed. Using the controller for a special case of optimal tracking control is discussed. Results of presented computer simulations of optimal control of discrete-time two-dimensional and three-dimensional affine nonlinear systems and optimal tracking control of permanent-magnet motor of linear type applied for accurate positioning and nonlinear cooling continuous stirred tank reactor confirm theoretical statements of the paper and illustrate a performance of the controller.






Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
All data generated and analyzed during this study are included in this published article.
References
Ross IM (2015) A primer on Pontryagin’s principle in optimal control: second edition. Collegiate Publishers.
Lewis FL, Vrabie DL, Syrmos VL (2012) Optimal control. John Wiley & Sons, Hoboken, NJ, USA. https://doi.org/10.1002/9781118122631
Kiumarsi B, Vamvoudakis KG, Modares H, Lewis FL (2018) Optimal and autonomous control using reinforcement learning: a survey. IEEE Trans Neural Netw Learn Syst 29(6):2042–2062. https://doi.org/10.1109/TNNLS.2017.2773458
Schiassi E, D’Ambrosio A, Furfaro R (2022) Bellman neural networks for the class of optimal control problems with integral quadratic cost. IEEE Trans Artif Intell. https://doi.org/10.1109/TAI.2022.3206735
Fairbank M, Li S, Fu X, Alonso E, Wunsch D (2014) An adaptive recurrent neural-network controller using a stabilization matrix and predictive inputs to solve a tracking problem under disturbances. Neural Netw 49:74–86. https://doi.org/10.1016/j.neunet.2013.09.010
Lewis FL, Vrabie D, Vamvoudakis KG (2012) Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control Syst 32(6):76–105. https://doi.org/10.1109/MCS.2012.2214134
Luo B, Wu HN, Huang T, Liu D (2014) Date-based approximate policy iteration for affine nonlinear continuous-time optimal control design. Automatica 50(12):3281–3290. https://doi.org/10.48550/arXiv.1311.0396
Aguilar CO, Krener AJ (2014) Numerical solutions to the Bellman equation of optimal control. J Optim Theory Appl 160:527–552. https://doi.org/10.1007/s10957-013-0403-8
Bauer F, Grune L, Semmler W (2006) Adaptive spline interpolation for Hamilton–Jacobi–Bellman equations. Appl Numer Math 56(9):1196–1210. https://doi.org/10.1016/j.apnum.2006.03.011
Wang Z, Li Y (2021) Nested sparse successive Galerkin approximation for nonlinear optimal control problems. IEEE Control Syst Lett 5(2):511–516. https://doi.org/10.1109/LCSYS.2020.3003732
Tan KK, Zhao S, Hu JX (2007) Online automatic tuning of a proportional integral derivative controller based on an iterative learning control approach. IET Control Theory Appl 1(1):90–96. https://doi.org/10.1049/iet-cta:20050004
Luo Q, Xue S, Liu D (2021) Adaptive critic designs for decentralized robust control of nonlinear interconnected systems via event triggering mechanism. Int J Syst Sci 53(5):1031–1047. https://doi.org/10.1080/00207721.2021.1987578
Lin Z, Duan J, Li SE, Ma H, Li J, Chen J, Cheng B, Ma J (2023) Policy-iteration-based finite-horizon approximate dynamic programming for continuous-time nonlinear optimal control. IEEE Trans Neural Netw Learn Syst 34(9):5255–5267. https://doi.org/10.1109/TNNLS.2022.3225090
Wei Q, Liu D, Song R, Yan P (2016) Data-based self-learning optimal control for continuous-time unknown nonlinear systems with disturbances. In: Proceedings of 2016 Chinese Control Decision Conference, pp 6633–6638. https://doi.org/10.1109/CCDC.2016.7532191
Vamvoudakis KG, Lewis FL (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5):878–888. https://doi.org/10.1016/j.automatica.2010.02.018
Tymoshchuk P (2019) A neural circuit model of adaptive robust tracking control for continuous-time nonlinear systems. In: Proc 28th Intern Conf ICANN, LNCS (vol 11727, pp 819–835). https://doi.org/10.1007/978-3-030-30487-4
Sun C, Saxena A, Sinitsyn NA (2017) Nearly optimal quantum control: an analytical approach. J Phys B At Mol Opt Phys 50:1–8. https://doi.org/10.1088/1361-6455/aa807d
Wu D (2015) Analytic solutions to optimal control problems with constraints. Appl Math 6:2326–2339. https://doi.org/10.4236/am.2015.614205
Liu Y, Zhou Y, Su S, Xun J, Tang T (2021) An analytical optimal control approach for virtually coupled high-speed trains with local and string stability. Transp Res Part C 125:1–19. https://doi.org/10.1016/j.trc.2020.102886
Tymoshchuk P, Lobur M (2020) Principles of artificial neural networks and their applications: manual. Lviv Polytechnic Publishing House, Lviv, Ukraine
Luo B, Liu D, Huang T, Yang X, Ma H (2017) Multi-step heuristic dynamic programming for optimal control of nonlinear discrete-time systems. Inf Sci 411:66–83. https://doi.org/10.1016/j.ins.2017.05.005
Alanis AY, Ornelas-Tellez F, Sanchez EN (2012) Discrete-time inverse optimal neural control for synchronous generators. Eng Appl Artif Intell 26:697–705. https://doi.org/10.1016/j.engappai.2012.07.008
Wei Q, Liu D (2013) A new self-learning optimal control scheme for discrete-time nonlinear systems using police iterative adaptive dynamic programming. IFAC Proc 46:580–585
Kiumarsi B, Lewis FL, Levine DS (2015) Optimal control of nonlinear discrete time-varying systems using a new neural network approximation structure. Neurocomputing 156:157–165. https://doi.org/10.1016/j.neucom.2014.12.067
Postyyan R, Busoniu L, Nesic D, Daafouz J (2017) Stability analysis of discrete-time infinite-horizon optimal control with discounted cost. IEEE Trans Autom Control 62(6):2736–2749. https://doi.org/10.1088/1361-6455/aa807d
Luo H, Yu L (2021) Stability analysis for the discrete-time T-S fuzzy system with stochastic disturbance and state delay. Fuzzy Inf Eng 13(1):17–39. https://doi.org/10.1080/16168658.2021.1932151
Hu NT (2023) Stability analysis for digital redesign of discrete-time switched systems using linear matrix inequality. Mathematics 11:1–22. https://doi.org/10.3390/math11112468
Siegel JW, Xu J (2020) Approximation rates for neural networks with general activation functions. Neural Netw 128:313–321. https://doi.org/10.1016/j.neunet.2020.05.019
Tymoshchuk P (2020) Optimal control for continuous-time scalar nonlinear systems with known dynamics. In: 2020 16th international conference on control, automation, robotics and vision (ICARCV) pp 695–700. https://doi.org/10.1109/ICARCV50220.2020.9305440
Haykin S (2011) Neural networks and learning machines. Pearson, Ontario, Canada.
Singh N, Panda SP (2022) Artificial neural network on graphical processing unit and its emphasis on ground water level prediction. Int J Inf Technol 14:3659–3666. https://doi.org/10.1007/s41870-022-00988-9
Szczeœniak A, Myczuda Z (2010) A method of charge accumulation in the logarithmic analog-to-digital converter with a successive approximation. Electrotech Rev 86(10):336–340
Tymoshchuk P, Shatnyi S (2015) Hardware implementation of discrete-time neural circuit of largest/smallest signal identification. In: The experience of designing and application of CAD systems in microelectronics, pp 226–230. https://doi.org/10.1109/CADSM.2015.7230842.
Tymoshchuk P, Shatnyi S (2019) Hardware implementation design of parallelized fuzzy adaptive resonance theory neural network. In: Perspective technologies and methods in MEMS design, pp 61–66. https://doi.org/10.1109/MEMSTECH.2019.8817406
Kaszkurewicz E, Bhaya A (2005) A generalized Persidskii theorem and its applications to non-smooth gradient dynamical systems. IFAC Proc 38(1):604–609. https://doi.org/10.3182/20050703-6-CZ-1902.00756
Haninger K, Hedrick K (2016) Discrete-time implementations of sliding mode control. In: American Control Conference, pp 6519–6524. https://doi.org/10.1109/ACC.2016.7526696
Dierks T, Jagannthan S (2009) Optimal control of affine nonlinear discrete-time systems. In: 17th Mediterranean conf control automation, pp 1390–1395. https://doi.org/10.1109/MED.2009.5164741
Yan Z, Wang J (2011) Model predictive control of nonlinear affine systems based on the General Projection neural network and its application to a continuous stirred tank reactor. In: International conference on information science and technology, pp 1011–1015. https://doi.org/10.1109/ICARCV50220.2020.9305440.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The author has no relevant financial or non-financial interests to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Tymoshchuk, P. Neural network optimal control for discrete-time nonlinear systems with known internal dynamics. Neural Comput & Applic 36, 4421–4434 (2024). https://doi.org/10.1007/s00521-023-09244-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-023-09244-y