Skip to main content
SpringerLink
Log in
Book cover

International Symposium on Neural Networks

ISNN 2017: Advances in Neural Networks - ISNN 2017 pp 197–205Cite as

Hamiltonian-Driven Adaptive Dynamic Programming Based on Extreme Learning Machine

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10261))

Abstract

In this paper, a novel frame work of reinforcement learning for continuous time dynamical system is presented based on the Hamiltonian functional and extreme learning machine. The idea of solution search in the optimization is introduced to find the optimal control policy in the optimal control problem. The optimal control search consists of three steps: evaluation, comparison and improvement of arbitrary admissible policy. The Hamiltonian functional plays an important role in the above framework, under which only one critic is required in the adaptive critic structure. The critic network is implemented by the extreme learning machine. Finally, simulation study is conducted to verify the effectiveness of the presented algorithm.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Abu-Khalaf, M., Lewis, F.L.: Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41(5), 779–791 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  2. Beard, R.W., Saridis, G.N., Wen, J.T.: Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation. Automatica 33(12), 2159–2177 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bertsekas, D.P.: Dynamic Programming and Optimal Control. Athena Scientific, Belmont (1995)

    MATH  Google Scholar 

  4. Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(1), 489–501 (2006)

    Article  Google Scholar 

  5. Jiang, Y., Jiang, Z.P.: Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics. Automatica 48(10), 2699–2704 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  6. Kleinman, D.: On an iterative technique for Riccati equation computations. IEEE Trans. Autom. Control 13(1), 114–115 (1968)

    Article  Google Scholar 

  7. Lewis, F.L., Syrmos, V.L.: Optimal Control. Wiley, New York (1995)

    Google Scholar 

  8. Liu, D., Wei, Q.: Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems. IEEE Trans. Neural Netw. Learn. Syst. 25(3), 621–634 (2014)

    Article  Google Scholar 

  9. Modares, H., Lewis, F.L., Jiang, Z.P.: H\(_\infty \) tracking control of completely unknown continuous-time systems via off-policy reinforcement learning. IEEE Trans. Neural Netw. Learn. Syst. 26(10), 2550–2562 (2015)

    Article  MathSciNet  Google Scholar 

  10. Prokhorov, D.V., Wunsch, D.C.: Adaptive critic designs. IEEE Trans. Neural Netw. 8(5), 997–1007 (1997)

    Article  Google Scholar 

  11. Saridis, G.N., Lee, C.G.: An approximation theory of optimal control for trainable manipulators. IEEE Trans. Syst. Man Cybern. 9(3), 152–159 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  12. Song, R., Xiao, W., Zhang, H., Sun, C.: Adaptive dynamic programming for a class of complex-valued nonlinear systems. IEEE Trans. Neural Netw. Learn. Syst. 25(9), 1733–1739 (2014)

    Article  Google Scholar 

  13. Vamvoudakis, K.G., Lewis, F.L.: Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5), 878–888 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  14. Wang, F.Y., Zhang, H., Liu, D.: Adaptive dynamic programming: an introduction. IEEE Comput. Intell. Mag. 4(2), 39–47 (2009)

    Article  Google Scholar 

  15. Wei, Q., Wang, F.Y., Liu, D., Yang, X.: Finite-approximation-error-based discrete-time iterative adaptive dynamic programming. IEEE Trans. Cybern. 44(12), 2820–2833 (2014)

    Article  Google Scholar 

  16. Yang, Y., Wunsch, D., Yin, Y.: Hamiltonian-driven adaptive dynamic programming for continuous nonlinear dynamical systems. IEEE Trans. Neural Netw. Learn. Syst. (2017, to be published)

    Google Scholar 

  17. Zhang, H., Qin, C., Jiang, B., Luo, Y.: Online adaptive policy learning algorithm for H\(_\infty \) state feedback control of unknown affine nonlinear discrete-time systems. IEEE Trans. Cybern. 44(12), 2706–2718 (2014)

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the Mary K. Finley Missouri Endowment, the Missouri S&T Intelligent Systems Center, the National Science Foundation, the National Natural Science Foundation of China (NSFC Grant No. 61333002) and the China Scholarship Council (CSC No. 201406460057).

Author information

Authors and Affiliations

  1. School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing, 100083, People’s Republic of China

    Yongliang Yang & Yixin Yin

  2. Department of Electrical and Computer Engineering, Missouri University of Science and Technology, Rolla, MO, 65409-0040, USA

    Donald Wunsch

  3. Department of Computer Science, Missouri University of Science and Technology, Rolla, MO, 65409, USA

    Zhishan Guo

Authors
  1. Yongliang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Donald Wunsch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhishan Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yixin Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yongliang Yang .

Editor information

Editors and Affiliations

  1. Dalian University of Technology, Dalian, China

    Fengyu Cong

  2. City University of Hong Kong, Kowloon Tong, Hong Kong

    Andrew Leung

  3. Chinese Academy of Sciences, Beijing, China

    Qinglai Wei

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Yang, Y., Wunsch, D., Guo, Z., Yin, Y. (2017). Hamiltonian-Driven Adaptive Dynamic Programming Based on Extreme Learning Machine. In: Cong, F., Leung, A., Wei, Q. (eds) Advances in Neural Networks - ISNN 2017. ISNN 2017. Lecture Notes in Computer Science(), vol 10261. Springer, Cham. https://doi.org/10.1007/978-3-319-59072-1_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-59072-1_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-59071-4

  • Online ISBN: 978-3-319-59072-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation