Skip to main content
SpringerLink
Log in

An Analysis of IRL-Based Optimal Tracking Control of Unknown Nonlinear Systems with Constrained Input

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

In this paper, a comparison is addressed between two methods, that is, the optimal tracking control methods of unknown nonlinear systems with and without constrained input. Firstly, the optimal tracking problem for a class of affine nonlinear system is formulated. The tracking cost functions are also defined, both for the two methods. The two methods are proved to be equivalent as the actuator bound is large enough. Integral reinforcement learning (IRL) algorithm is employed to solve the optimal tracking problem by using only system data. To facilitate the implementation of the IRL algorithm, the actor-critic neural network technique and the least squares method are employed in approximating the unknown weights iteratively. In the simulation, a detailed comparison is given to demonstrate the relationship between the two methods in the aspects of control input and tracking cost value.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Modares H, Lewis F (2014) Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning. Automatica 50:1780–1792

    Article  MathSciNet  Google Scholar 

  2. Modares H, Lewis F, Naghibi-Sistani M-B (2014) Integral reinforcement learning and experience replay for adaptive optimal control of partially unknown constrained-input continuous-time systems. Automatica 50:193–202

    Article  MathSciNet  Google Scholar 

  3. Yang X, Liu D, Luo B, Li C (2016) Data-based robust adaptive control for a class of unknown nonlinear constrained-input systems via integral reinforcement learning. Inf Sci 369:731–747

    Article  Google Scholar 

  4. Cui X, Zhang H, Luo Y, Jiang H (2017) Adaptive dynamic programming for H tracking design of uncertain nonlinear systems with disturbances and input constraints. Int J Adapt Control Signal Process 31:1567–1583

    Article  MathSciNet  Google Scholar 

  5. Ding C, Li Y, Zhang L, Zhang J, Yang L, Wei W (2018) Fast-convergent fully connected deep learning model using constrained nodes input. Neural Process Lett pp 1–11. https://doi.org/10.1007/s11063-018-9872-y

    Article  Google Scholar 

  6. Lewis FL, Liu D (2013) Reinforcement learning and approximate dynamic programming for feedback control, vol 17. Wiley, Hoboken

    Google Scholar 

  7. Xiao G, Zhang H, Luo Y, Jiang H (2016) Data-driven optimal tracking control for a class of affine non-linear continuous-time systems with completely unknown dynamics. IET Control Theory Appl 10:700–710

    Article  MathSciNet  Google Scholar 

  8. Liu C, Zhang H, Xiao G, Sun S (2019) Integral reinforcement learning based decentralized optimal tracking control of unknown nonlinear large-scale interconnected systems with constrained-input. Neurocomputing 323:1–11

    Article  Google Scholar 

  9. Devasia S, Degang C, Paden B (1996) Nonlinear inversion-based output tracking. IEEE Trans Autom Control 41:930–942

    Article  MathSciNet  Google Scholar 

  10. Qin C, Zhang H, Luo Y (2013) Online optimal tracking control of continuous-time linear systems with unknown dynamics by using adaptive dynamic programming. Int J Control 87:1000–1009

    Article  MathSciNet  Google Scholar 

  11. Huang HC, Chiang CH (2016) An evolutionary radial basis function neural network with robust genetic-based immunecomputing for online tracking control of autonomous robots. Neural Process Lett 44:19–35

    Article  Google Scholar 

  12. Zhang H, Cui L, Zhang X, Luo Y (2011) Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method. IEEE Trans Neural Netw 22:2226–2236

    Article  Google Scholar 

  13. Lin Q, Wei Q, Liu D (2017) A novel optimal tracking control scheme for a class of discrete-time nonlinear systems using generalised policy iteration adaptive dynamic programming algorithm. Int J Syst Sci 48:525–534

    Article  MathSciNet  Google Scholar 

  14. Wei Q, Liu D, Xu Y (2016) Neuro-optimal tracking control for a class of discrete-time nonlinear systems via generalized value iteration adaptive dynamic programming approach. Soft Comput 20:697–706

    Article  Google Scholar 

  15. Song R, Wei Q, Xiao W (2016) ADP-based optimal sensor scheduling for target tracking in energy harvesting wireless sensor networks. Neural Comput Appl 27:1543–1551

    Article  Google Scholar 

  16. Kiumarsi B, Lewis FL, Modares H, Karimpour A, Naghibi-Sistani MB (2014) Reinforcement -learning for optimal tracking control of linear discrete-time systems with unknown dynamics. Automatica 50:1167–1175

    Article  MathSciNet  Google Scholar 

  17. Han K, Feng J, Cui X (2017) Fault-tolerant optimised tracking control for unknown discrete-time linear systems using a combined reinforcement learning and residual compensation methodology. Int J Syst Sci 48:2811–2825

    Article  MathSciNet  Google Scholar 

  18. Qu Q, Zhang H, Feng T, Jiang H (2017) Decentralized adaptive tracking control scheme for nonlinear large-scale interconnected systems via adaptive dynamic programming. Neurocomputing 225:1–10

    Article  Google Scholar 

  19. Na J, Herrmann G (2014) Online adaptive approximate optimal tracking control with simplified dual approximation structure for continuous-time unknown nonlinear systems. IEEE CAA J Autom Sin 1:412–422

    Article  Google Scholar 

  20. Zhang H, Luo Y, Liu D (2009) Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans Neural Netw 20:1490–1503

    Article  Google Scholar 

  21. Liu Y, Zhang H, Luo Y, Han J (2016) ADP based optimal tracking control for a class of linear discrete-time system with multiple delays. J Frankl Inst 353:2117–2136

    Article  MathSciNet  Google Scholar 

  22. Luo B, Wu H, Huang T, Liu D (2014) Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design. Automatica 50:3281–3290

    Article  MathSciNet  Google Scholar 

  23. Wang D, He H, Liu D (2017) Adaptive critic nonlinear robust control: a survey. IEEE Trans Cybern 47:3429–3451

    Article  Google Scholar 

  24. Lv Y, Na J, Ren X (2017) Online H\(\infty \) control for completely unknown nonlinear systems via an identifier critic-based ADP structure. Int J Control 3:1–28

    MATH  Google Scholar 

  25. Wei Q, Song R, Yan P (2016) Data-driven zero-sum neuro-optimal control for a class of continuous-time unknown nonlinear systems with disturbance using ADP. IEEE Trans Neural Netw Learn Syst 27:444–458

    Article  MathSciNet  Google Scholar 

  26. Yang X, Liu D, Wei Q, Wang D (2016) Guaranteed cost neural tracking control for a class of uncertain nonlinear systems using adaptive dynamic programming. Neurocomputing 198:80–90

    Article  Google Scholar 

  27. Zhang H, Zhang J, Yang G, Luo Y (2015) Leader-based optimal coordination control for the consensus problem of multiagent differential games via fuzzy adaptive dynamic programming. IEEE Trans Fuzzy Syst 23:152–163

    Article  Google Scholar 

  28. Zhang H, Liu D, Luo Y, Wang D (2012) Adaptive dynamic programming for control: algorithms and stability. Springer, London

    Google Scholar 

  29. Wei Q, Zhang H, Dai J (2009) Model-free multiobjective approximate dynamic programming for discrete-time nonlinear systems with general performance index functions. Neurocomputing 72:1839–1848

    Article  Google Scholar 

  30. Wang D, Liu D (2018) Learning and guaranteed cost control with event-based adaptive critic implementation. IEEE Trans Neural Netw Learn Syst 29:6004–6014

    Article  Google Scholar 

  31. Song R, Lewis F, Wei Q (2017) Off-policy integral reinforcement learning method to solve nonlinear continuous-time multiplayer nonzero-sum games. IEEE Trans Neural Netw Learn Syst 28:704–713

    Article  MathSciNet  Google Scholar 

  32. Zhang H, Cui L, Luo Y (2013) Near-optimal control for nonzero-sum differential games of continuous-time nonlinear systems using single-network ADP. IEEE Trans Cybern 43:206–216

    Article  Google Scholar 

  33. Ding C, Sun Y, Zhu Y (2017) A NN-based hybrid intelligent algorithm for a discrete nonlinear uncertain optimal control problem. Neural Process Lett 45:457–473

    Article  Google Scholar 

  34. Wang D, He H, Liu D (2018) Intelligent optimal control with critic learning for a nonlinear overhead crane system. IEEE Trans Ind Inform 14:2932–2940

    Article  Google Scholar 

  35. Faußser S, Schwenker F (2015) Neural network ensembles in reinforcement learning. Neural Process Lett 41:55–69

    Article  Google Scholar 

  36. Pucheta J, Patiño H, Fullana R, Schugurensky C, Kuchen B (2006) A neuro-dynamic programming-based optimal controller for tomato seedling growth in greenhouse systems. Neural Process Lett 24:241–260

    Article  Google Scholar 

  37. Modares H, Lewis F, Jiang Z (2015) H\(\infty \) tracking control of completely unknown continuous-time systems via off-policy reinforcement learning. IEEE Trans Neural Netw Learn Syst 26:2550–2562

    Article  MathSciNet  Google Scholar 

  38. Stewart J (2015) Single variable calculus: early transcendentals. Cengage Learning, Boston

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (6143300461627809, 61621004), and the National High Technology Research and Development Program of China (2012AA040104).

Author information

Authors and Affiliations

  1. State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University, Shenyang, 110004, Liaoning, China

    Huaguang Zhang

  2. College of Information Science and Engineering, Northeastern University, Shenyang, 110004, Liaoning, China

    Chong Liu, Huaguang Zhang, He Ren & Yuling Liang

Authors
  1. Chong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huaguang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. He Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuling Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huaguang Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, C., Zhang, H., Ren, H. et al. An Analysis of IRL-Based Optimal Tracking Control of Unknown Nonlinear Systems with Constrained Input. Neural Process Lett 50, 2681–2700 (2019). https://doi.org/10.1007/s11063-019-10029-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-019-10029-5

Keywords

Search

Navigation