Skip to main content
SpringerLink
Log in

The finite-horizon optimal control for a class of time-delay affine nonlinear system

  • ISNN 2011
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In this paper, a new iteration algorithm is proposed to solve the finite-horizon optimal control problem for a class of time-delay affine nonlinear systems with known system dynamic. First, we prove that the algorithm is convergent as the iteration step increases. Then, a theorem is presented to demonstrate that the limit of the iteration performance index function satisfies discrete-time Hamilton–Jacobi–Bellman (DTHJB) equation, and the finite-horizon iteration algorithm is presented with satisfactory accuracy error. At last, two neural networks are used to approximate the iteration performance index function and the corresponding control policy. In simulation part, an example is given to demonstrate the effectiveness of the proposed iteration algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Al-Tamimi A, Abu-Khalaf M, Lewis FL (2007) Adaptive critic designs for discrete-time zero-sum games with application to \(H_{\infty}\) control. IEEE Trans Syst Man Cybern Part B 37:240–247

    Article  Google Scholar 

  2. Bellman RE (1957) Dynamic programming. Princeton University Press, Princeton

    MATH  Google Scholar 

  3. Liu D, Javaherian H, Kovalenko O, Huang T (2008) Adaptive critic learning techniques for engine torque and air-fuel ratio control. IEEE Trans Syst Man Cybern Part B 38(4):988–993

    Article  Google Scholar 

  4. Murray JJ, Cox CJ, Lendaris GG, Saeks R (2002) Adaptive dynamic programming. IEEE Trans Syst Man Cybern Part C 32(2):140–153

    Article  Google Scholar 

  5. Manu MZ, Mohammad J (1987) Time-delay systems analysis, optimization and applications. North-Holland, New York

  6. Prokhorov DV, Wunsch DC (1997) Adaptive critic designs. IEEE Trans Neural Netw 8(5):997–1007

    Article  Google Scholar 

  7. Werbos PJ (1987) Building and understanding adaptive systems: a statistical/numerical approach to factory automation and brain research. IEEE Trans Syst Man Cybern 17(1):7–20

    Article  Google Scholar 

  8. Werbos PJ (1990) A menu of designs for reinforcement learning over time. In: Miller WT, Sutton RS, Werbos PJ (eds) Neural networks for control. MIT Press, Cambridge, pp 67–95

    Google Scholar 

  9. Werbos PJ (1992) Approximate dynamic programming for real-time control and neural modeling. In: White DA, Sofge DA (eds) Handbook of intelligent control: neural, fuzzy and adaptive approaches, Chap 13. Van Nostrand, New York

  10. Werbos PJ (1991) A menu of designs for reinforcement learning over time. In: Miller WT, Sutton RS, Werbos PJ (eds) Neural networks for control. MIT Press, Cambridge, pp 67–95

    Google Scholar 

  11. Wang FY, Jin N, Liu DR, Wei QL (2001) Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with \(\epsilon\)-error bound. IEEE Trans Neural Netw 22:24–36

    Article  Google Scholar 

  12. Zhang HG, Luo YH, Liu DR (2009) Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans Neural Netw 20(9):1490–1503

    Article  Google Scholar 

  13. Zhang HG, Wei QL, Liu DR (2011) An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games. Automatica 47:207–214

    Article  MathSciNet  MATH  Google Scholar 

  14. Zhang HG, Wei QL, Luo YH (2008) A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm. IEEE Trans Syst Man Cybern Part B 38(4):937–942

    Article  Google Scholar 

  15. Wei QL, Zhang HG, Dai J (2009) Model-free multiobjective approximate dynamic programming for discrete-time nonlinear systems with general performance index functions. Neurocomputing 72(7–9):1839–1848

    Article  Google Scholar 

  16. Wang FY, Zhang HG, Liu DR (2009) Adaptive dynamic programming: an introduction. IEEE Comput Intell Mag 4(2):39–47

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China (50977008, 60774048, 61034005), the Fundamental Research Funds for the Central Universities (N100604020).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Northeastern University, Shenyang, Liaoning, 110004, China

    Ruizhuo Song & Huaguang Zhang

  2. Key Laboratory of Integrated Automation of Process Industry, Ministry of Education, Northeastern University, Shenyang, Liaoning, 110004, China

    Huaguang Zhang

Authors
  1. Ruizhuo Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huaguang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huaguang Zhang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Song, R., Zhang, H. The finite-horizon optimal control for a class of time-delay affine nonlinear system. Neural Comput & Applic 22, 229–235 (2013). https://doi.org/10.1007/s00521-011-0706-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-011-0706-3

Keywords

Search

Navigation