Abstract
Aimed at achieving multi-step reinforcement learning in continuous spaces, many Learning Classifier Systems have been developed recently to learn fuzzy logic rules. Among these systems, accuracy-based Michigan learning fuzzy classifier systems are gaining increasing research attention. However, in order to learn effectively, existing accuracy-based systems often require the action space to be discrete. Without this restriction, only single-step learning may be supported. In this paper, we will develop a new accuracy-based learning fuzzy classifier system that can perform multi-step reinforcement learning in completely continuous domains. To achieve this goal, a special fuzzy logic system will be introduced in this paper where the output action from the system is modelled through a continuous probability distribution. A natural gradient learning technique will be further exploited to fine-tune the action outputs of individual fuzzy rules. The effectiveness of our learning system has been verified on several benchmark problems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Amari, S.: Natural gradient works efficiently in learning. Neural Comput. 10(2), 251–276 (1998)
Berlanga, F.J., Rivera, A.J., del Jesus, M.J., Herrera, F.: Gp-coach: genetic programming-based learning of compact and accurate fuzzy rule-based classification systems for high-dimensional problems. Inf. Sci. 180(8), 1183–1200 (2010)
Butz, M.V., Goldberg, D.E., Lanzi, P.L.: Gradient descent methods in learning classifier systems: improving XCS performance in multistep problems. IEEE Trans. Evol. Comput. 9, 452–473 (2005)
Casillas, J., Carse, B., Bull, L.: Fuzzy-XCS: a michigan genetic fuzzy system. IEEE Trans. Fuzzy Syst. 15, 536–550 (2007)
Chen, G., Douch, C., Zhang, M.: Using learning classifier systems to learn stochastic decision policies. IEEE Trans. Evol. Comput. (2015, to appear)
Chen, G., Zhang, M., Pang, S., Douch, C.: Stochastic decision making in learning classifier systems through a natural policy gradient method. In: Loo, C.K., Yap, K.S., Wong, K.W., Beng Jin, A.T., Huang, K. (eds.) ICONIP 2014, Part III. LNCS, vol. 8836, pp. 300–307. Springer, Heidelberg (2014)
Gu, D., Hu, H.: Accuracy based fuzzy q-learning for robot behaviours. In: Proceedings of 2004 IEEE International Conference on Fuzzy Systems (2004)
Holland, J.H.: Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor (1975)
Peters, J., Schaal, S.: Natural actor-critic. Neurocomputing 71, 1180–1190 (2008)
Sutton, R.: Generalization in reinforcement learning: successful examples using sparse coarse coding. Adv. Neural Inf. Process. Syst. 8, 1038–1044 (1996)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Wang, X.S., Cheng, Y.H., Yi, J.Q.: A fuzzy actor-critic reinforcement learning network. Inf. Sci. 177, 3764–3781 (2007)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Chen, G., Douch, C., Zhang, M., Pang, S. (2015). Reinforcement Learning in Continuous Spaces by Using Learning Fuzzy Classifier Systems. In: Arik, S., Huang, T., Lai, W., Liu, Q. (eds) Neural Information Processing. ICONIP 2015. Lecture Notes in Computer Science(), vol 9490. Springer, Cham. https://doi.org/10.1007/978-3-319-26535-3_37
Download citation
DOI: https://doi.org/10.1007/978-3-319-26535-3_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26534-6
Online ISBN: 978-3-319-26535-3
eBook Packages: Computer ScienceComputer Science (R0)