Skip to main content
SpringerLink
Log in

Multi-objective optimal control for a class of nonlinear time-delay systems via adaptive dynamic programming

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

A novel multi-objective adaptive dynamic programming (ADP) method is constructed to obtain the optimal controller of a class of nonlinear time-delay systems in this paper. Using the weighted sum technology, the original multi-objective optimal control problem is transformed to the single one. An ADP method is established for nonlinear time-delay systems to solve the optimal control problem. To demonstrate that the presented iterative performance index function sequence is convergent and the closed-loop system is asymptotically stable, the convergence analysis is also given. The neural networks are used to get the approximative control policy and the approximative performance index function, respectively. Two simulation examples are presented to illustrate the performance of the presented optimal control method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Al-Tamimi A, Lewis FL, Abu-Khalaf M (2007) Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infi nity control. Automatica 43:473–481

    Article  MathSciNet  MATH  Google Scholar 

  • Anguelova M, Wennberg B (2008) State elimination and identifiability of the delay parameter for nonlinear time-delay systems. Automatica 44(5):1373–1378

    Article  MathSciNet  Google Scholar 

  • Bellman RE (1957) Dynamic Programming. Princeton University Press, Princeton

    MATH  Google Scholar 

  • Chen B, Liu XP, Liu KF, Lin C (2009) Novel adaptive neural control design for nonlinear MIMO time-delay systems. Automatica 45(6):1554–1560

    Article  MathSciNet  MATH  Google Scholar 

  • Chyung DH (1970) On the controllability of linear systems with delay in control. IEEE Trans Autom Control 15(2):255–257

    Article  MathSciNet  Google Scholar 

  • Chyung DH (1970) Controllability of linear systems with multiple delays in control. IEEE Trans Autom Control 15(6):694–695

    Article  Google Scholar 

  • Enns R, Si J (2003) Helicopter trimming and tracking control using direct neural dynamic programming. IEEE Trans Neural Netw 14(4):929–939

    Article  Google Scholar 

  • Fernández-Navarro F, Hervás-Martínez C, Gutierrez PA (2013) Generalised Gaussian radial basis function neural networks. Soft Comput 17:519–533

    Article  Google Scholar 

  • Fu J, He H, Zhou X (2011) Adaptive learning and control for MIMO system based on adaptive dynamic programming. IEEE Trans Neural Netw 22(7):1133–1148

    Article  Google Scholar 

  • Gyurkovics É, Takács T (2003) Quadratic stabilisation with H\(\infty \)-norm bound of non-linear discrete-time uncertain systems with bounded control. Syst Control Lett 50:277–289

    Article  MATH  Google Scholar 

  • He P, Jagannathan S (2007) Reinforcement learning neural-network-based controller for nonlinear discrete-time systems with input constraints. IEEE Trans Syst Man Cybern Part B: Cybern 37(2):425–436

    Article  Google Scholar 

  • He H, Ni Z, Fu J (2012) A three-network architecture for on-line learning and optimization based on adaptive dynamic programming. Neurocomputing 78(1):3–13

    Google Scholar 

  • Maji K, Pratihar DK, Nath AK (2013) Analysis and synthesis of laser forming process using neural networks and neuro-fuzzy inference system. Soft Comput 17:849–865

    Article  Google Scholar 

  • Murray JJ, Cox CJ, Lendaris GG, Saeks R (2002) Adaptive dynamic programming. IEEE Trans Syst Man Cybernetics Part C Appl Rev 32(2):140–153

    Article  Google Scholar 

  • Powell WB (2009) Approximate Dynamic Programming: Solving the Curses of Dimensionality. Wiley, New York

    Google Scholar 

  • Richard JP (2003) Time-delay systems: an overview of some recent advances and open problems. Automatica 39(10):1667–1694

    Article  MathSciNet  MATH  Google Scholar 

  • Richert D, Masaud K, Macnab CJB (2013) Discrete-time weight updates in neural-adaptive control. Soft Comput 17:431–444

    Article  MATH  Google Scholar 

  • Si J, Wang YT (2001) On-line learning control by association and reinforcement. IEEE Trans Neural Netw 12(2):264–276

    Article  MathSciNet  Google Scholar 

  • Song R, Xiao W, Zhang H (2013) Multi-objective optimal control for a class of unknown nonlinear systems based on finite-approximation-error ADP algorithm. Neurocomputing 119:212–221

    Google Scholar 

  • Vamvoudakis KG, Lewis FL (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5):878–888

    Article  MathSciNet  MATH  Google Scholar 

  • Vrabie D, Pastravanu O, Abu-Khalaf M, Lewis FL (2009) Adaptive optimal control for continuous-time linear systems based on policy iteration. Automatica 45(2):477–484

    Article  MathSciNet  MATH  Google Scholar 

  • Wang FY, Jin N, Liu DR, Wei QL (2011) Adaptive dynamic programming for finite horizon optimal control of discrete-time nonlinear systems with \(\varepsilon \)-error bound. IEEE Trans Neural Netw 22(1):24–36

    Google Scholar 

  • Wang D, Liu D, Wei Q, Zhao D, Jin N (2012) Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming. Automatica 48(8):1825–1832

    Article  MathSciNet  MATH  Google Scholar 

  • Wei Q, Zhang H, Dai J (2009) Model-free multiobjective approximate dynamic programming for discrete-time nonlinear systems with general performance index functions. Neurocomputing 72(7–9):1839–1848

    Google Scholar 

  • Wei Q, Liu D (2012) An iterative \(\epsilon \)-optimal control scheme for a class of discrete-time nonlinear systems with unfixed initial state. Neural Netw 32:236–244

    Article  MATH  Google Scholar 

  • Werbos PJ (1977) Advanced forecasting methods for global crisis warning and models of intelligence. General Syst Yearbook 22:25–38

    Google Scholar 

  • Zhang H, Wei Q, Luo Y (2008) A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm. IEEE Trans Syst Man Cybern Part B Cybern 38(4):937–942

    Article  Google Scholar 

  • Zhang H, Luo Y, Liu D (2009) Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans Neural Netw 20:1490–1503

    Article  Google Scholar 

  • Zhang HG, Song RZ, Zhang TY (2011) Optimal tracking control for a class of nonlinear discrete-time systems with time delays based on heuristic dynamic programming. IEEE Trans Netw 22(12):1851–1862

    Article  Google Scholar 

  • Zhang HG, Wei QL, Liu DR (2011) An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games. Automatica 47(1):207–214

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang H, Wei Q, Liu D (2011) An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games. Automatica 47(1):207–214

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang X, Zhang H, Sun Q, Luo Y (2012) Adaptive dynamic programming-based optimal control of unknown nonaffine nonlinear discrete-time systems with proof of convergence. Neurocomputing 91:48–55

    Google Scholar 

  • Zheng C, Jagannathan S (2008) Generalized Hamilton-Jacobi-Bellman formulation-based neural network control of affine nonlinear discrete-time systems. IEEE Trans Neural Netw 19(1):90–106

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the Open Research Project from SKLMCCS (20120106), the Fundamental Research Funds for the Central Universities (FRF-TP-13-018A), the China Postdoctoral Science Foundation (2013M530527), and the National Natural Science Foundation of China (61304079).

Author information

Authors and Affiliations

  1. School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing, 100083, China

    Ruizhuo Song & Wendong Xiao

  2. The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Qinglai Wei

Authors
  1. Ruizhuo Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wendong Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qinglai Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ruizhuo Song.

Additional information

Communicated by C. Alippi, D. Zaho and D. Liu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Song, R., Xiao, W. & Wei, Q. Multi-objective optimal control for a class of nonlinear time-delay systems via adaptive dynamic programming. Soft Comput 17, 2109–2115 (2013). https://doi.org/10.1007/s00500-013-1111-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-013-1111-x

Keywords

Search

Navigation