Abstract
In this paper, we investigate the decentralized feedback stabilization and adaptive dynamic programming (ADP)-based optimization for the class of nonlinear systems with matched interconnections. The decentralized control law of the overall system is designed by integrating all controllers of the isolated subsystems, and it satisfies the optimality on the basis of optimal control laws of all the subsystems. For solving the optimal control problems of these isolated subsystems, the policy iteration algorithm is used to approximately solve the Hamilton–Jacobi–Bellman equations in the framework of ADP with the neural network implementation, where a set of critic neural networks is constructed to estimate the optimal cost functions, and the approximate optimal control laws can be obtained after the learning of critic neural networks. The weight estimation errors of the critic networks and the stability of all isolated subsystems are proved based on the Lyapunov theory. Finally, the performance of the proposed decentralized optimal control strategy is verified by simulation results.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Abu-Khalaf M, Lewis FL (2005) Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41:779–791
Bakule L (2008) Decentralized control: an overview. Ann Rev Control 32:87–98
Beard RW, Saridis GN, Wen JT (1997) Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation. Automatica 33:2159–2177
Gu B, Sheng VS, Tay KY, Romano W, Li S (2015a) Incremental support vector learning for ordinal regression. IEEE Trans Neural Netw Learn Syst 26:1403–1416
Gu B, Sheng VS, Wang Z, Ho D, Osman S, Li S (2015b) Incremental learning for \(\nu \)-support vector regression. Neural Netw 67:140–150
Hou ZG, Gupta MM, Nikiforuk PN, Tan M, Cheng L (2007) A recurrent neural network for hierarchical control of interconnected dynamic systems. IEEE Trans Neural Netw 18:466–481
Jiang Y, Jiang ZP (2015) Global adaptive dynamic programming for continuous-time nonlinear systems. IEEE Trans Autom Control 60:2917–2929
Lewis FL, Vrabie D, Vamvoudakis KG (2012) Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control Syst Mag 32:76–105
Liu D, Li H, Wang D (2013) Data-based self-learning optimal control: research progress and prospects. Acta Autom Sin 39:1858–1870
Liu D, Wang D, Li H (2014) Decentralized stabilization for a class of continuous-time nonlinear interconnected systems using online learning optimal control approach. IEEE Trans Neural Netw Learn Syst 25:418–428
Luo B, Wu HN, Huang T, Liu D (2014) Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design. Automatica 50:3281–3290
Mehraeen S, Jagannathan S (2011) Decentralized optimal control of a class of interconnected nonlinear discrete-time systems by using online Hamilton-Jacobi-Bellman formulation. IEEE Trans Neural Netw 22:1757–1769
Modares H, Lewis FL, Naghibi-Sistani MB (2013) Adaptive optimal control of unknown constrained-input systems using policy iteration and neural networks. IEEE Trans Neural Netw Learn Syst 24:1513–1525
Mu C, Ni Z, Sun C, He H (2016) Data-driven tracking control with adaptive dynamic programming for a class of continuous-time nonlinear systems. IEEE Trans Cybern. doi:10.1109/TCYB.2016.2548941 (in press)
Mu C, Ni Z, Sun C, He H (2017a) Air-breathing hypersonic vehicle track ing control based on adaptive dynamic programming. IEEE Trans Neural Netw Learn Syst 28(3):584–598
Mu C, Wang D, He H (2017b) Novel iterative neural dynamic programming for data-based approximate optimal control design. Automatica. doi:10.1016/j.automatica.2017.03.022 (in press)
Na J, Herrmann G (2014) Online adaptive approximate optimal tracking control with simplified dual approximation structure for continuous time unknown nonlinear systems. IEEE/CAA J Autom Sin 1:412–422
Ni Z, He H (2013) Heuristic dynamic programming with internal goal representation. Soft Comput 17:2101–2108
Panagi P, Polycarpou MM (2011) Decentralized fault tolerant control of a class of interconnected nonlinear systems. IEEE Trans Autom Control 56:178–184
Prokhorov DV, Wunsch DC (1997) Adaptive critic designs. IEEE Trans Neural Netw 8:997–1007
Rice JK, Verhaegen M (2009) Distributed control: a sequentially semi-separable approach for spatially heterogeneous linear systems. IEEE Trans Autom Control 54:1270–1283
Saberi A (1998) On optimality of decentralized control for a class of nonlinear interconnected systems. Automatica 24:101–104
Siljak DD (2012) Decentralized control of complex systems. Academic Press, Boston
Sutton RS, Barto AG (1998) Reinforcement learning—an introduction. MIT Press, Cambridge
Vamvoudakis KG, Lewis FL (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5):878–888
Wang FY, Zhang H, Liu D (2009) Adaptive dynamic programming: an introduction. IEEE Comput Intell Mag 4:39–47
Wang D, Liu D, Wei Q, Zhao D, Jin N (2012) Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming. Automatica 48:1825–1832
Wang D, Liu D, Li H (2014a) Policy iteration algorithm for online design of robust control for a class of continuous-time nonlinear systems. IEEE Trans Autom Sci Eng 11:627–632
Wang D, Liu D, Li H, Ma H (2014b) Neural-network-based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming. Inf Sci 282:167–179
Wang D, Liu D, Li H, Ma H, Li C (2016a) A neural-network-based online optimal control approach for nonlinear robust decentralized stabilization. Soft Comput 20:707–716
Wang D, Liu D, Ma H, Mu C (2016b) Decentralized guaranteed cost control of interconnected systems with uncertainties: a learning-based optimal control strategy. Neurocomputing 214:297–306
Wang D, Mu C, Liu D (2017) Data-driven nonlinear near-optimal regulation based on iterative neural dynamic programming. Acta Autom Sin 43:354–363
Werbos PJ (2008) ADP: the key direction for future research in intelligent control and understanding brain intelligence. IEEE Trans Syst Man Cybern B Cybern 38:898–900
Yang X, Liu D, Ma H, Xu Y (2016) Online approximate solution of HJI equation for unknown constrained-input nonlinear continuous-time systems. Inf Sci 328:435–454
Zhang H, Luo Y, Liu D (2009) Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans Neural Netw 20:1490–1503
Zhao D, Zhu Y (2015) MEC-A near-optimal online reinforcement learning algorithm for continuous deterministic systems. IEEE Trans Neural Netw Learn Syst 26:346–356
Acknowledgements
This work was supported by National Natural Science Foundation of China under Grants 61304018, 61304086, 61533017, 61533008, 61520106009, and U1501251, China Postdoctoral Science Foundation under Grant 2014M561559, Tianjin Natural Science Foundation under Grant 14JCQNJC05400, Beijing Natural Science Foundation under Grant 4162065, Tianjin Key Laboratory of Process Measurement and Control under Grant TKLPMC-201612, and the Early Career Development Award of SKLMCCS.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Human and animals rights
This paper does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Communicated by V. Loia.
Rights and permissions
About this article
Cite this article
Mu, C., Sun, C., Wang, D. et al. Decentralized adaptive optimal stabilization of nonlinear systems with matched interconnections. Soft Comput 22, 2705–2715 (2018). https://doi.org/10.1007/s00500-017-2526-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-017-2526-6