Skip to main content
SpringerLink
Log in

Local Policy Iteration Adaptive Dynamic Programming for Discrete-Time Nonlinear Systems

  • Conference paper
  • First Online:
Advances in Neural Networks - ISNN 2017 (ISNN 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10262))

Included in the following conference series:

  • 2819 Accesses

Abstract

Adaptive dynamic programming is a hot research topic nowadays. Therefore, the paper concerns a new local policy adaptive iterative dynamic programming (ADP) algorithm. Moreover, this algorithm is designed for the discrete-time nonlinear systems, which are used to solve problems concerning infinite horizon optimal control. The new local policy iteration ADP algorithm has the characteristics of updating the iterative control law and value function within one subset of the state space. Morevover, detailed iteration process of the local policy iteration is presented thereafter. The simulation example is listed to show the good performance of the newly developed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Werbos, P.: Advanced forecasting methods for global crisis warning and models of intelligence. Gen. Syst. Yearb. 22, 25–38 (1977)

    Google Scholar 

  2. Fu, Y., Fu, J., Chai, T.: Robust adaptive dynamic programming of two-player zero-sum games for continuous-time linear systems. IEEE Trans. Neural Netw. Learn. Syst. 26, 3314–3319 (2015). doi:10.1109/TNNLS.2015.2461452

    Article  MathSciNet  Google Scholar 

  3. Abouheaf, M., Lewis, F., Vamvoudakis, K., Haesaert, S., Babuska, R.: Multi-agent discrete-time graphical games and reinforcement learning solutions. Automatica 50(12), 3038–3053 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  4. Zargarzadeh, H., Dierks, T., Jagannathan, S.: Optimal control of nonlinear continuous-time systems in strict-feedback form. IEEE Trans. Neural Netw. Learn. Syst. 26(10), 2535–2549 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  5. Wei, Q., Liu, D.: Data-driven neuro-optimal temperature control of water gas shift reaction using stable iterative adaptive dynamic programming. IEEE Trans. Industr. Electron. 61(11), 6399–6408 (2014)

    Article  Google Scholar 

  6. Heydari, A.: Revisiting approximate dynamic programming and its convergence. IEEE Trans. Cybern. 44(12), 2733–2743 (2014)

    Article  Google Scholar 

  7. Lewis, F., Vrabie, D., Vamvoudakis, K.: Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control Syst. 32(6), 76–105 (2012)

    Article  MathSciNet  Google Scholar 

  8. Wei, Q., Liu, D., Lin, H.: Value iteration adaptive dynamic programming for optimal control of discrete-time unknown nonlinear systems with disturbance using ADP. IEEE Trans. Neural Netw. Learn. Syst. 27(2), 444–458 (2016)

    Article  MathSciNet  Google Scholar 

  9. Wei, Q., Liu, D., Yang, X.: Inifinite horizon self-learning optimal control of nonaffine discrete-time nonlinear systems. IEEE Trans. Neural Netw. Learn. Syst. 26(4), 879–886 (2015)

    Article  Google Scholar 

  10. Wei, Q., Song, R., Yan, P.: Data-driven zero-sum neuro-optimal control for a class of continuous-time unknow nonlinear systems with disturbance using ADP. IEEE Trans. Neural Netw. Learn. Syst. 27(2), 444–458 (2016)

    Article  MathSciNet  Google Scholar 

  11. Wei, Q., Wang, F., Liu, D., Yang, X.: Finite-approximation-error based discrete-time iterative adaptive dynamic programming. IEEE Trans. Cybern. 44(12), 2820–2833 (2014)

    Article  Google Scholar 

  12. Wei, Q., Liu, D., Shi, G., Liu, Y.: Optimal multi-battery coordination control for home energy management systems via distributed iterative adaptive dynamic programming. IEEE Trans. Ind. Electron. 42(7), 4203–4214 (2015)

    Article  Google Scholar 

  13. Wei, Q., Liu, D., Shi, G.: A novel dual iterative Q-learning method for optimal battery management in smart residential environments. IEEE Trans. Ind. Electron. 62(4), 2509–2518 (2015)

    Article  Google Scholar 

  14. Wei, Q., Liu, D.: A novel iterative \(\theta \)-adaptive dynamic programming for discrete-time nonlinear systems. IEEE Trans. Autom. Sci. Eng. 11(4), 1176–1190 (2014)

    Article  Google Scholar 

  15. Wei, Q., Liu, D.: Adaptive dynamic programming for optimal tracking control of unknown nonlinear systems with application to coal gasification. IEEE Trans. Autom. Sci. Eng. 11(4), 1020–1036 (2014)

    Article  Google Scholar 

  16. Liu, D., Wei, Q.: Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems. IEEE Trans. Neural Netw. Learn. Syst. 25(3), 621–634 (2014)

    Article  Google Scholar 

  17. Xu, X., Hou, Z., Lian, C., He, H.: Online learning control using adaptive critic designs with sparse kernel machines. IEEE Trans. Neural Netw. Learn. Syst. 24(5), 762–775 (2013)

    Article  Google Scholar 

  18. Liu, D., Yang, X., Wang, D., Wei, Q.: Reinforcement-learning-based robust controller design for continuous-time uncertain nonlinear systems subject to input constraints. IEEE Trans. Cybern. 45(7), 1372–1385 (2015)

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grants 61233001, 61273140, 61374105, and 61304079.

Author information

Authors and Affiliations

  1. The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Beijing, 100190, China

    Qinglai Wei, Yancai Xu & Qiao Lin

  2. The School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing, 100083, China

    Derong Liu & Ruizhuo Song

Authors
  1. Qinglai Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yancai Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiao Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Derong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ruizhuo Song
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qinglai Wei .

Editor information

Editors and Affiliations

  1. Dalian University of Technology, Dalian, China

    Fengyu Cong

  2. City University of Hong Kong, Kowloon Tong, Hong Kong

    Andrew Leung

  3. Chinese Academy of Sciences, Beijing, China

    Qinglai Wei

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Wei, Q., Xu, Y., Lin, Q., Liu, D., Song, R. (2017). Local Policy Iteration Adaptive Dynamic Programming for Discrete-Time Nonlinear Systems. In: Cong, F., Leung, A., Wei, Q. (eds) Advances in Neural Networks - ISNN 2017. ISNN 2017. Lecture Notes in Computer Science(), vol 10262. Springer, Cham. https://doi.org/10.1007/978-3-319-59081-3_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-59081-3_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-59080-6

  • Online ISBN: 978-3-319-59081-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation