Abstract
Dual Heuristic Programming (DHP) is a class of approximate dynamic programming methods using neural networks. Although there have been some successful applications of DHP, its performance and convergence are greatly influenced by the design of the step sizes in the critic module as well as the actor module. In this paper, a Delta-Bar-Delta learning rule is proposed for the DHP algorithm, which helps the two modules adjust learning rate individually and adaptively. Finally, the feasibility and effectiveness of the proposed method are illustrated in the learning control task of an inverted pendulum.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Suykens, J.A.K., de Moor, B., Vandewalle, J.: Stabilizing Neural Controllers: A Case Study for Swing up a Double Inverted Pendulum. In: NOLTA 1993 International Symposium on Nonlinear Theory and Its Application, Hawaii (1993)
Werbos, P.J.: A Menu of Designs for Reinforcement Learning Over Time. In: Mliler, W.T., Sutton, R.S., Werbos, P.J. (eds.) Neural Networks for Control, ch. 3. MIT Press, Cambridge (1990)
Werbos, P.J.: Approximate Dynamic Programming for Real-Time Control and Neural Modeling. In: White, D.A., Sofge, D.A. (eds.) Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, ch.13, Van Nostrand Reinhold, New York (1992)
Liu, D.R.: Approximate Dynamic Programming for Self-Learning Control. Acta Automatica Sinica 31(1), 13–18 (2005)
Prokhorov, D., Wunsch, D.: Adaptive Critic Designs. IEEE Trans. Neural Networks 8, 997–1007 (1997)
Venayagamoorthy, G.K., Harley, R.G., Wunsch, D.C.: Comparison of Heuristic Dynamic Programming and Dual Heuristic Programming Adaptive Critics for Neurocontrol of a Tur-bogenerator. IEEE Transactions on Neural Networks 13(3), 763–764 (2002)
Park, J.W., Harley, R.G., Venayagamoorthc, G.K., et al.: Dual Heuristic Programming Based Nonlinear Optimal Control for a Synchronous Generator. Engineering Applications of Arti-ficial Intelligence 21, 97–105 (2008)
Lin, W.S., Yang, P.C.: Adaptive Critic Motion Control Design of Autonomous Wheeled Mobile Robot by Dual Heuristic Programming. Automatica 44, 2716–2723 (2008)
Zhang, N., Wunsch II, D.C.: Application of Collective Robotic Search Using Neural Network Based Dual Heuristic Programming (DHP). In: Wang, J., Yi, Z., Żurada, J.M., Lu, B.-L., Yin, H. (eds.) ISNN 2006. LNCS, vol. 3972, pp. 1140–1145. Springer, Heidelberg (2006)
Ferrari, S., Stengel, R.F.: Online Adaptive Critic Flight Control. Journal of Guidance, Control, and Dynamics 27(5), 777–786 (2004)
Jacobs, R.A.: Increased Rates of Convergence Through Learning Rate Adaption. Neural Networks 1, 295–307 (1988)
Lendaris, G.G., Shannon, T.T., Schultz, L.J., et al.: Dual Heuristic Programming for Fuzzy Control. In: Proceedings of IFSA / NAFIPS, Vancouver, B.C., vol. 1, pp. 551–556 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wu, J., Xu, X., Lian, C., Huang, Y. (2011). Adaptive Dual Heuristic Programming Based on Delta-Bar-Delta Learning Rule. In: Liu, D., Zhang, H., Polycarpou, M., Alippi, C., He, H. (eds) Advances in Neural Networks – ISNN 2011. ISNN 2011. Lecture Notes in Computer Science, vol 6677. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21111-9_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-21111-9_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21110-2
Online ISBN: 978-3-642-21111-9
eBook Packages: Computer ScienceComputer Science (R0)