Skip to main content
SpringerLink
Log in

Approximate optimal solution of the DTHJB equation for a class of nonlinear affine systems with unknown dead-zone constraints

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

In this paper, an optimal control scheme of a class of unknown discrete-time nonlinear systems with dead-zone control constraints is developed using adaptive dynamic programming (ADP). First, the discrete-time Hamilton–Jacobi–Bellman (DTHJB) equation is derived. Then, an improved iterative ADP algorithm is constructed which can solve the DTHJB equation approximately. Combining with Riemann integral, detailed proofs of existence and uniqueness of the solution are also presented. It is emphasized that this algorithm allows the implementation of optimal control without knowing internal system dynamics. Moreover, the approach removes the requirements of precise parameters of the dead-zone. Finally, simulation studies are given to demonstrate the performance of the present approach using neural networks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Abu-Khalaf M, Lewis FL (2005) Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41(5):779–791

    Article  MATH  MathSciNet  Google Scholar 

  • Al-Tamimi A, Abu-Khalaf M, Lewis FL (2007) Adaptive critic designs for discrete-time zero-sum games with application to \(H_ {\infty}\) control. IEEE Trans Syst Man Cybern B 37(1):240–247

    Article  Google Scholar 

  • Al-Tamimi A, Lewis FL (2008) Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof. IEEE Trans Syst Man Cybern B 38(4):943–949

    Article  Google Scholar 

  • Balakrishnan SN, Ding J, Lewis FL (2008) Issues on stability of ADP feedback controllers for dynamical systems. IEEE Trans Syst Man Cybern B 38(4):913–917

    Article  Google Scholar 

  • Bellman RE (1957) Dynamic programming. Princeton University Press, Princeton

  • Bernstein DS (1995) Optimal nonlinear, but continuous, feedback control of systems with saturating actuators. Int J Control 62(5):1209–1216

    Article  MATH  Google Scholar 

  • Chen Z, Jagannathan S (2008) Generalized Hamilton–Jacobi–Bellman formulation-based neural network control of affine nonlinear discrete-time systems. IEEE Trans Neural Netw 19(1):90–106

    Article  Google Scholar 

  • Gao W, Rastko RS (2006) Neural network control of a class of nonlinear systems with actuator saturation. IEEE Trans Neural Netw 17(1):147–156

    Article  MATH  Google Scholar 

  • Lewis FL, Tim WK, Wang LZ, Li ZX (1999) Deadzone compensation in motion control systems using adaptive fuzzy logic control. IEEE Trans Control Syst Technol 7(6):731–742

    Article  Google Scholar 

  • Liu DR, Xiong XX, Zhang Y (2001) Action-dependent adaptive critic designs. In: Proceeding of the International Joint Conference on Neural Networks, Washington, pp 990–995

  • Liu DR, Zhang Y, Zhang HG (2005) A self-learning call admission control scheme for CDMA cellular networks. IEEE Trans Neural Netw 16(5):1219–1228

    Article  Google Scholar 

  • Liu ZW, Zhang HG, Zhang QL (2010) Novel stability analysis for recurrent neural networks with multiple delays via line integral-type L-K functional. IEEE Trans Neural Netw 21(11):1710–1718

    Article  Google Scholar 

  • Lyshevski SE (1998) Optimal control of nonlinear continuous-time systems: design of bounded controllers via generalized nonquadratic functional. In: Proceedings of American control conference, Philadelphia, pp 205–209

  • Ma HJ, Yang GH (2010) Adaptive output control of uncertain nonlinear systems with non-symmetric dead-zone input. Automatica 46(2):413–420

    Article  MATH  MathSciNet  Google Scholar 

  • Prokhorov DV, Wunsch DC (1997) Adaptive critic designs. IEEE Trans Neural Netw 8(5):997–1007

    Article  Google Scholar 

  • Recker D, Kokotovic P, Rhode D, Winkelman J (1991) Adaptive nonlinear control of systems containing a dead-zone. In: Proceedings of the 30th IEEE conference on decision and control, Brighton, pp 2111–2115

  • Saberi A, Lin Z, Teel A (1996) Control of linear systems with saturating actuators. IEEE Trans Autom Control 41(3):368–378

    Article  MATH  MathSciNet  Google Scholar 

  • Seiffertt J, Sanyal S, Wunsch DC (2001) Hamilton–Jacobi–Bellman equations and approximate dynamic programming on time scales. IEEE Trans Syst Man Cybern B 38(4):918–923

    Article  Google Scholar 

  • Selmic RR, Lewis FL (2000) Deadzone compensation in motion control systems using neural networks. IEEE Trans Autom Control 45(4):602–613

    Article  MATH  MathSciNet  Google Scholar 

  • Si J, Wang Y (2001) Online learning control by association and reinforcement. IEEE Trans Neural Netw 12(2):264–276

    Article  MathSciNet  Google Scholar 

  • Sussmann H, Sontag ED, Yang Y (1994) A general result on the stabilization of linear systems using bounded controls. IEEE Trans Autom Control 39(12):2411–2425

    Article  MATH  MathSciNet  Google Scholar 

  • Tao G, Kokotovic PV (1994) Adaptive control of plants with unknown dead-zones. IEEE Trans Autom Control 39(1):59–68

    Article  MATH  MathSciNet  Google Scholar 

  • Tao G, Kokotovic PV (1995) Adaptive control of systems with unknown output backlash. IEEE Trans Autom Control 40(2):326–330

    Article  MATH  MathSciNet  Google Scholar 

  • Tao G, Kokotovic PV (1995) Continuous-time adaptive control of systems with unknown backlash. IEEE Trans Autom Control 40(6):1083–1087

    Article  MATH  MathSciNet  Google Scholar 

  • Tao G, Kokotovic PV (1995) Discrete-time adaptive control of systems with unknown deadzones. Int J Control 61(1):1–17

    Article  MATH  MathSciNet  Google Scholar 

  • Tao G, Kokotovic PV (1996) Adaptive control of systems with actuator and sensor nonlinearities. John Wiley, New York

  • Wang XS, Su CY, Hong H (2004) Robust adaptive control of a class of nonlinear systems with unknown dead-zone. Automatica 40(3):407–413

    Article  MATH  MathSciNet  Google Scholar 

  • Wei QL, Zhang HG, Dai J (2009) Model-free multiobjective approximate dynamic programming for discrete-time nonlinear systems with general performance index functions. Neurocomputing 72(7–9):1839–1848

    Article  Google Scholar 

  • Werbos PJ (1992) Approximate dynamic programming for real-time control and neural modeling. In: White DA, Sofge DA (eds) Handbook of intelligent control: neural, fuzzy, and adaptive approaches, New York

  • Xu JX, Xu J, Lee TH (2005) Iterative learning control for systems with input deadzone. IEEE Trans Autom Control 50(9):1455–1459

    Article  MathSciNet  Google Scholar 

  • Zhang TP, Ge SS (2007) Adaptive neural control of MIMO nonlinear state time-varying delay systems with unknown dead-zones and gain signs. Automatica 43(6):1021–1033

    Article  MATH  MathSciNet  Google Scholar 

  • Zhang TP, Ge SS (2008) Adaptive dynamic surface control of nonlinear systems with unknown deadzone in pure feedback form. Automatica 44(7):1895–1903

    Article  MATH  MathSciNet  Google Scholar 

  • Zhang H, Luo Y, Liu D (2009) Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans Neural Netw 20(9):1490–1503

    Article  Google Scholar 

  • Zhang HG, Liu ZW, Huang GB, Wang ZS (2010) Novel weighting-delay-based stability criteria for recurrent neural networks with time-varying delay. IEEE Trans Neural Netw 21(1):91–106

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grants 61034002, 61233001, and 61273140.

Author information

Authors and Affiliations

  1. State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Dehua Zhang, Derong Liu & Ding Wang

Authors
  1. Dehua Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Derong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ding Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Derong Liu.

Additional information

Communicated by G. Acampora.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhang, D., Liu, D. & Wang, D. Approximate optimal solution of the DTHJB equation for a class of nonlinear affine systems with unknown dead-zone constraints. Soft Comput 18, 349–357 (2014). https://doi.org/10.1007/s00500-013-1062-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-013-1062-2

Keywords

Search

Navigation