Abstract
Learning in non-Markov environments presents difficulties for Learning Classifier Systems. The presence of perceptually aliased situations induces the system to consider some distinct states of the environment as identical. The system, therefore, may not be able to decide the best action in each situation. An alternative, presented by Stolzmann (1999), is to use classifiers with behavioral sequences in the Anticipatory Classifier System (ACS). This method allows ACS to learn latently a non-Markov environment in mobile robot simulations. This paper presents a study of ACS reward and latent learning capacities in some non-Markov environments when using behavioral sequences. An ACS, based on Stolzmann’s work and using some enhancements introduced by Butz, Goldberg and Stolzman (2000), is detailed. This system is tested in several woods environments in order to highlight the learning effectiveness of this method according to the environmental properties.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Stolzmann W.: Anticipatory Classifier Systems. In Koza, John R., Banzhaf, Wolfgang, Chellapilla, Kumar, Deb, Kalyanmoym Dorigo, Marco, Fogel, David B., Garzon, Max H., Goldberg, David E., Iba, Hitoshi, and Riolo, Rick. (editors). Genetic Programming 1998: Proceedings of the Third Annual Conference, July 22-25, 1998, University of Wisconsin, Madison, Wisconsin, 658-664. San Francisco, CA: Morgan Kaufmann (1998)
Hoffmann, J.: Vorhersage und Erkenntnis [Anticipation and Cognition]. Hogrefe, Goettingen (1993)
Booker, L.B., Goldberg, D.E., Holland, J.H.: Classifier systems and genetic algorithms. Artificial Intelligence 40, 235–282 (1989)
Lanzi, P.L.: Adaptive Agents with Reinforcement Learning and Internal Memory. In: Sixth International Conference on the Simulation of Adaptive Behavior (SAB 2000), pp. 333–342. MIT Press, Cambridge (2000)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. The MITPress, Cambridge (1998)
Cliff, D., Ross, S.: Adding Temporary Memory to ZCS. Adaptive Bahavior 3(2), 101–150 (1995)
Lanzi, P.L.: An Analysis of the Memory Mechanism of XCSM. In: Koza, J.R., et al. (eds.) Genetic Programming 1998: Proceedings of the Third Annual Conference, University of Wisconsin, Madison, Wisconsin, July 22-25, pp. 643–651. Morgan Kaufmann, San Francisco (1998)
Stolzmann, W.: Latent Learning in Khepera Robots with Anticipatory Classifier Systems. In: Wu, A.S. (ed.) Proceedings of the 1999 Genetic and Evolutionary Computation Conference Workshop Program, pp. 290–297 (1999)
Butz, M., Goldberg, D.E., Stolzmann, W.: The Anticipatory Classifier System and Genetic Generalization, in the Illigal Report No. 200032 (2000)
Butz, M., Stolzmann, W.: Action-Planning in Anticipatory Classifier Systems. In: Wu, A.S. (ed.) Proceedings of the 1999 Genetic and Evolutionary Computation Conference Workshop Program, pp. 242–249 (1999)
Butz, M.V., Goldberg, D.E., Stolzmann, W.: Introducing a genetic generalization pressure to the Anticipatory Classifier System Part 1: Theoretical approach. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2000 (2000)
Butz, M.V., Goldberg, D.E., Stolzmann, W.: Introducing a genetic generalization pressure to the Anticipatory Classifier System Part 2: Performance Analysis. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2000 (2000)
Butz, M., Goldberg, M., Stolzmann, W.: Probability-Enhanced Predictions in the Anticipatory Classifier System, in the Illigal Report No. 200015 (2000)
Butz, M.V.: Anticipations, Anticipatory Classifier Systems, and Genetic Generalization. A Diploma Thesis from the University of Wuerzburg, German, in the IlliGAL Report No. 2001025 (2001)
Gérard, P., Stolzmann, W., Sigaud, O.: YACS: a new Learning Classifier System using Anticipation. Journal of Soft Computing: Special Issue on Learning Classifier Systems (to appear)
Goldberg, D.E.: Genetic Algorithms in search, optimization and machine learning. Addison-Wesley, Reading (1989)
Butz, M., Goldberg, D.E., Stolzmann, W.:: New Challenges for an Anticipatory Classifier System: Hard Problems and Possible Solutions, in the Illigal Report No. 99019 (1999)
Widrow, B., Hoff, M.: Adaptive switching circuits. Western Electronics Show and Convention 4, 96–104 (1960)
Wilson, S.W.: Classifier fitness based on accuracy. Evolutionary Computation 3(2), 149–175 (1995)
Watkins, C.J.C.H., Dayan, P.: Qlearning. Machine Learning 8(3), 272–292 (1992)
Wilson, S.W.: ZCS: A zeroth level classifier system. Evolutionary Computation 2(1), 1–18 (1994)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Métivier, M., Lattaud, C. (2003). Anticipatory Classifier System Using Behavioral Sequences in Non-Markov Environments. In: Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds) Learning Classifier Systems. IWLCS 2002. Lecture Notes in Computer Science(), vol 2661. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-40029-5_9
Download citation
DOI: https://doi.org/10.1007/978-3-540-40029-5_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-20544-9
Online ISBN: 978-3-540-40029-5
eBook Packages: Springer Book Archive