Abstract
The mountain car problem is a well-known task, often used for testing reinforcement learning algorithms. It is a problem with real valued state variables, which means that some kind of function approximation is required. In this paper, three reinforcement learning architectures are compared on the mountain car problem. Comparison results are presented, indicating the potentials of the actor-only approach. The function approximation modules used are based on NeuroFAST ( Neuro- Fuzzy ART-Based Structure and Parameter Learning TSK Model). NeuroFAST is a neuro-fuzzy modelling algorithm, with well-proven function approximation capabilities, and features the functional reasoning method (the Takagi-Sugeno-Kang fuzzy model), Fuzzy ART concepts and specific techniques.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Baird, L.C.: Reinforcement learning in continuous time: Advantage updating. In: Proc. of IEEE Intl. Conf. on Neural Networks (ICNN 1994), Orlando, Florida (1994)
Baird, L.C.: Residual algorithms: Reinforcement learning with function approximation. In: Prieditis, A., Russell, S. (eds.) Proc. of 12th Intl. Conf. on Mach. Learn., pp. 30–37. Morgan Kaufmann, San Francisco (1995)
Baxter, J., Bartlett, P.L.: Reinforcement learning in POMDP’s via direct gradient ascent. In: Proc. of 17th Intl. Conf. on Machine Learning, Stanford, CA (2000)
Box, G.E.P., Jenkins, G.M.: Time Series Analysis, Forecasting, and Control. Holden Day, San Francisco (1970)
Boyan, J.A., Moore, A.W.: Generalization in reinforcement learning: Safely approximating the value functions. In: Tesauro, G., Touretzky, D. (eds.) Advances in Neural Information Processing Systems (Proc. of 1994 Conf.), San Mateo, CA, pp. 369–376. Morgan Kaufmann, San Mateo (1995)
Carpenter, G.A., Grossberg, S., Rosen, G.A.: Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system. Neural Networks 4, 759–771 (1991)
Carpenter, G.A., Grossberg, S., Markuzon, N., Reynolds, J.H., Rosen, D.B.: Fuzzy ARTMAP: A neural architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans. on Neural Networks 3(5), 698–712 (1992)
Gordon, G.: Stable function approximation in dynamic programming. In: Proc. of 14th Intl. Conf. on Machine Learning, Nashville, TN (1997)
Kimura, H., Kobayashi, S.: An analysis of actor/critic algorithms using eligibility traces: reinforcement learning with imperfect value functions. In: Proc. of 15th Intl. Conf. on Mach. Learn., Madison, Wisconsin, pp. 278–286 (1998)
Konda, V.R., Tsitsiklis, J.N.: Actor critic algorithms. In: Advances in Neural Information Processing Systems (Proc. of the 1999 conference), Cambridge, MA, vol. 12. MIT Press, Cambridge (1999)
Lin, C.-J., Lee, C.S.G.: Neural Fuzzy Systems: a Neuro-Fuzzy Synergism to Intelligent Systems. Prentice Hall, Englewood Cliffs (1996)
Mackey, M.C., Glass, L.: Oscillation and chaos in physiological control systems. Science 197, 287–289 (1977)
Mamdani, E.H., Assilian, S.: Applications of fuzzy algorithms for control of simple dynamic plant. Proc. Inst. Elec. Eng. 121, 1585–1588 (1974)
Moore, A.W., Atkeson, C.G.: The parti-game algorithm for variable resolution reinforcement learning in multidimensional state-spaces. Machine Learning 21, 1–36 (1995)
Sutton, R.S., Barto, A.G.: Reinforcement learning: An introduction. MIT Press, Cambridge (1998)
Sutton, R.S., McAllester, D., Singh, S., Mansour, Y.: Policy gradient methods for reinforcement learning with function approximation. In: Advances in Neural Information Processing Systems, Cambridge, MA, vol. 12, pp. 1057–1063. MIT Press, Cambridge (1999)
Takagi, T., Sugeno, M.: Fuzzy identification of systems and its application to modeling and control. IEEE Trans on Syst., Man, Cybern. 15, 116–132 (1985)
Tzafestas, S.G., Zikidis, K.C.: NeuroFAST: On-line neuro-fuzzy ART-based structure and parameter learning TSK model. IEEE Trans on Syst., Man, Cybern. 31(5), 797–802 (2001)
Tzafestas, S.G., Zikidis, K.C.: High Accuracy Neuro – Fuzzy Modeling. In: IEEE Int. Conf. on Artificial Intelligence Systems (IEEE ICAIS 2002), Divnomorskoe, Russia (2002)
Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 8, 229–256 (1992)
Zikidis, K.C., Tzafestas, S.G.: Adaptive neuro-fuzzy modeling applied to policy gradient reinforcement learning. In: 5th Hellenic European Conference on Computer Mathematics & its Applications (HERCMA 2001), Athens, Greece (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zikidis, K.C., Tzafestas, S.G. (2003). ART-Based Neuro-fuzzy Modelling Applied to Reinforcement Learning. In: Palade, V., Howlett, R.J., Jain, L. (eds) Knowledge-Based Intelligent Information and Engineering Systems. KES 2003. Lecture Notes in Computer Science(), vol 2774. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45226-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-540-45226-3_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40804-8
Online ISBN: 978-3-540-45226-3
eBook Packages: Springer Book Archive