Skip to main content
SpringerLink
Log in

A neural-network-based online optimal control approach for nonlinear robust decentralized stabilization

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

In this paper, the robust decentralized stabilization of continuous-time uncertain nonlinear systems with multi control stations is developed using a neural network based online optimal control approach. The novelty lies in that the well-known adaptive dynamic programming method is extended to deal with the nonlinear feedback control problem under uncertain and large-scale environment. Through introducing an appropriate bounded function and defining a modified cost function, it can be observed that the decentralized optimal controller of the nominal system can achieve robust decentralized stabilization of original uncertain system. Then, a critic neural network is constructed for solving the modified Hamilton–Jacobi–Bellman equation corresponding to the nominal system in an online fashion. The weights of the critic network are tuned based on the standard steepest descent algorithm with an additional term provided to guarantee the boundedness of system states. The stability analysis of the closed-loop system is carried out via the Lyapunov approach. At last, two simulation examples are given to verify the effectiveness of the present control approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Abu-Khalaf M, Lewis FL (2005) Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41(5):779–791

    Article  MATH  MathSciNet  Google Scholar 

  • Adhyaru DM, Kar IN, Gopal M (2011) Bounded robust control of nonlinear systems using neural network-based HJB solution. Neural Comput Appl 20(1):91–103

    Article  Google Scholar 

  • Al-Tamimi A, Lewis FL, Abu-Khalaf M (2008) Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof. IEEE Trans Syst Man Cybern Part B Cybern 38(4):943–949

    Article  Google Scholar 

  • Bhasin S, Kamalapurkar R, Johnson M, Vamvoudakis KG, Lewis FL, Dixon WE (2013) A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems. Automatica 49(1):82–92

    Article  MATH  MathSciNet  Google Scholar 

  • Dierks T, Jagannathan S (2012) Online optimal control of affine nonlinear discrete-time systems with unknown internal dynamics by using time-based policy update. IEEE Trans Neural Netw Learn Syst 23(7):1118–1129

    Article  Google Scholar 

  • Dierks T, Jagannathan S (2010) Optimal control of affine nonlinear continuous-time systems. In: Proceedings of the American control conference, Baltimore, MD, USA, June 2010, pp 1568–1573

  • Haddad WM, Chellaboina V, Fausz JL (1998) Robust nonlinear feedback control for uncertain linear systems with nonquadratic performance criteria. Syst Control Lett 33(5):327–338

    Article  MATH  MathSciNet  Google Scholar 

  • Haddad WM, Chellaboina V, Fausz JL (2000) Optimal non-linear robust control for non-linear uncertain systems. Int J Control 73(4):329–342

    Article  MATH  MathSciNet  Google Scholar 

  • Heydari A, Balakrishnan SN (2013) Finite-horizon control-constrained nonlinear optimal control using single network adaptive critics. IEEE Trans Neural Netw Learn Syst 24(1):145–157

    Article  Google Scholar 

  • Jiang ZP, Jiang Y (2013) Robust adaptive dynamic programming for linear and nonlinear systems: an overview. Eur J Control 19(5):417–425

    Article  MATH  MathSciNet  Google Scholar 

  • Lewis FL, Jagannathan S, Yesildirek A (1999) Neural network control of robot manipulators and nonlinear systems. Taylor & Francis, London

    Google Scholar 

  • Lewis FL, Vrabie D, Vamvoudakis KG (2012) Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control Syst Mag 32(6):76–105

    Article  MathSciNet  Google Scholar 

  • Lin F (2000) An optimal control approach to robust control design. Int J Control 73(3):177–186

    Article  MATH  Google Scholar 

  • Liu D, Wang D, Zhao D, Wei Q, Jin N (2012) Neural-network-based optimal control for a class of unknown discrete-time nonlinear systems using globalized dual heuristic programming. IEEE Trans Autom Sci Eng 9(3):628–634

    Article  Google Scholar 

  • Liu D, Wang D, Yang X (2013a) An iterative adaptive dynamic programming algorithm for optimal control of unknown discrete-time nonlinear systems with constrained inputs. Inf Sci 220:331–342

  • Liu D, Li H, Wang D (2013b) Neural-network-based zero-sum game for discrete-time nonlinear systems via iterative adaptive dynamic programming algorithm. Neurocomputing 110:92–100

  • Liu D, Li H, Wang D (2013c) Data-based self-learning optimal control: research progress and prospects. Acta Automatica Sinica 39(11):1858–1870

  • Liu D, Wang D, Li H (2014a) Decentralized stabilization for a class of continuous-time nonlinear interconnected systems using online learning optimal control approach. IEEE Trans Neural Netw Learn Syst 25(2):418–428

  • Liu D, Li H, Wang D (2014b) Online synchronous approximate optimal learning algorithm for multiplayer nonzero-sum games with unknown dynamics. IEEE Trans Syst Man Cybern Syst 44(8):1015–1027

  • Ni Z, He H, Wen J (2013) Adaptive learning in tracking control based on the dual critic network design. IEEE Trans Neural Netw Learn Syst 24(6):913–928

    Article  Google Scholar 

  • Ni Z, He H (2013) Heuristic dynamic programming with internal goal representation. Soft Comput 17(11):2101–2108

    Article  Google Scholar 

  • Nodland D, Zargarzadeh H, Jagannathan S (2013) Neural network-based optimal adaptive output feedback control of a helicopter UAV. IEEE Trans Neural Netw Learn Syst 24(7):1061–1073

  • Vamvoudakis KG, Lewis FL (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5):878–888

  • Wang D, Liu D, Wei Q (2012a) Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach. Neurocomputing 78(1):14–22

  • Wang D, Liu D, Wei Q, Zhao D, Jin N (2012b) Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming. Automatica 48(8):1825–1832

  • Wang D, Liu D, Li H (2014) Policy iteration algorithm for online design of robust control for a class of continuous-time nonlinear systems. IEEE Trans Autom Sci Eng 11(2):627–632

    Article  Google Scholar 

  • Werbos PJ (1992) Approximate dynamic programming for real-time control and neural modeling. In: White DA, Sofge DA (eds) Proceedings of handbook of intelligent control: neural, fuzzy, and adaptive approaches, ch 13, Van Nostrand Reinhold, New York

  • Wu HN, Luo B (2012) Neural network based online simultaneous policy update algorithm for solving the HJI equation in nonlinear \(H_{\infty }\) control. IEEE Trans Neural Netw Learn Syst 23(12):1884–1895

    Article  MathSciNet  Google Scholar 

  • Yang X, Liu D, Wang D (2014) Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints. Int J Control 87(3):553–566

    Article  MathSciNet  Google Scholar 

  • Zhang H, Luo Y, Liu D (2009) Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans Neural Netw 20(9):1490–1503

    Article  Google Scholar 

  • Zhang H, Cui L, Luo Y (2013) Near-optimal control for nonzero-sum differential games of continuous-time nonlinear systems using single-network ADP. IEEE Trans Cybern 43(1):206–216

    Article  Google Scholar 

  • Zhang D, Liu D, Wang D (2014) Approximate optimal solution of the DTHJB equation for a class of nonlinear affine systems with unknown dead-zone constraints. Soft Comput 18(2):349–357

    Article  Google Scholar 

  • Zhao Q, Xu H, Dierks T, Jagannathan S (2013) Finite-horizon neural network-based optimal control design for affine nonlinear continuous-time systems. In: Proceedings of the international joint conference on neural networks, Dallas, TX, USA, Aug 2013, pp 1–6

Download references

Author information

Authors and Affiliations

  1. The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Ding Wang, Derong Liu, Hongliang Li, Hongwen Ma & Chao Li

Authors
  1. Ding Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Derong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongliang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hongwen Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Derong Liu.

Additional information

Communicated by V. Loia.

This work was supported in part by the National Natural Science Foundation of China under Grants 61034002, 61233001, 61273140, 61304086, and 61374105, in part by Beijing Natural Science Foundation under Grant 4132078, and in part by the Early Career Development Award of SKLMCCS.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, D., Liu, D., Li, H. et al. A neural-network-based online optimal control approach for nonlinear robust decentralized stabilization. Soft Comput 20, 707–716 (2016). https://doi.org/10.1007/s00500-014-1534-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-014-1534-z

Keywords

Search

Navigation