Skip to main content
Springer Nature Link
Log in

Biasing Exploration in an Anticipatory Learning Classifier System

  • Conference paper
  • First Online:
Advances in Learning Classifier Systems (IWLCS 2001)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 2321))

Included in the following conference series:

  • 334 Accesses

Abstract

The chapter investigates how model and behavioral learning can be improved in an anticipatory learning classifier system by biasing exploration. First, the applied system ACS2 is explained. Next, an overview over the possibilities of applying exploration biases in an anticipatory learning classifier system and specifically ACS2 is provided. In ACS2, a recency bias termed action delay bias as well as an error bias termed knowledge array bias is implemented. The system is applied in a dynamic maze task and an hand-eye coordination task to validate the biases. The experiments exhibit that biased exploration enables ACS2 to evolve and adapt its internal environmental model faster. Also adaptive behavior is improved.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  • Birk, A. (1995). Stimulus Response Lernen [Stimulus response learning]. Doctoral dissertation, University of Saarbrücken, Germany.

    Google Scholar 

  • Butz, M. V. (2001). Anticipatory learning classifier systems. Genetic Algorithms and Evolutionary Computation. Boston, MA: Kluwer Academic Publishers.

    Google Scholar 

  • Butz, M. V., Goldberg, D. E., and Stolzmann, W. (2000). Introducing a genetic generalization pressure to the anticipatory classifier system: Part 2-performance analysis. In Whitely, D., Goldberg, D. E., Cantu-Paz, E., Spector, L., Parmee, I., and Beyer, H.-G. (Eds.), Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2000) pp. 42–49. San Francisco, CA: Morgan Kaufmann.

    Google Scholar 

  • Butz, M. V., Goldberg, D. E., and Stolzmann, W. (2001). Probability-enhanced predictions in the anticipatory classifier system. In Lanzi, P. L., Stolzmann, W., and Wilson, S. W. (Eds.), Advances in Learning Classifier Systems, LNAI 1996 pp. 37–51. Berlin Heidelberg: Springer-Verlag.

    Chapter  Google Scholar 

  • Dayan, P., and Sejnowski, T. J. (1996). Exploration bonus and dual control. Machine Learning, 25(1), 5–22.

    Google Scholar 

  • Gérard, P., and Sigaud, O. (2001). YACS: Combining dynamic programming with generalization in classifier systems. In Lanzi, P. L., Stolzmann, W., and Wilson, S. W. (Eds.), Advances in Learning Classifier Systems, LNAI 1996 pp. 52–69. Berlin Heidelberg: Springer-Verlag.

    Chapter  Google Scholar 

  • Hoffmann, J. (1993). Vorhersage und Erkenntnis [Anticipation and cognition]. Goettingen, Germany: Hogrefe.

    Google Scholar 

  • Kaelbling, L. P. (1993). Learning in embedded systems. Cambridge, MA: MIT Press.

    Google Scholar 

  • Lanzi, P. L. (1999). An analysis of generalization in the XCS classifier system. Evolutionary Computation, 7(2), 125–149.

    Article  Google Scholar 

  • Lanzi, P. L., Stolzmann, W., and Wilson, S. W. (Eds.) (2001). Advances in learning classifier systems, LNAI 1996. Berlin Heidelberg: Springer-Verlag.

    Google Scholar 

  • Moore, A. W., and Atkeson, C. (1993). Memory-based reinforcement learning: Converging with less data and less real time. Machine Learning, 13, 103–130.

    Google Scholar 

  • Stolzmann, W. (1997). Antizipative Classifier Systems [Anticipatory classifier systems]. Aachen, Germany: Shaker Verlag.

    Google Scholar 

  • Stolzmann, W. (2000). An introduction to anticipatory classifier systems. In Lanzi, P. L., Stolzmann, W., and Wilson, S. W. (Eds.), Learning Classifier Systems: From Foundations to Applications, LNAI 1813 pp. 175–194. Berlin Heidelberg: Springer-Verlag.

    Chapter  Google Scholar 

  • Stolzmann, W., and Butz, M. V. (2000). Latent learning and action-planning in robots with anticipatory classifier systems. In Lanzi, P. L., Stolzmann, W., and Wilson, S. W. (Eds.), Learning Classifier Systems: From Foundations to Applications, LNAI 1813 pp. 301–317. Berlin Heidelberg: Springer-Verlag.

    Chapter  Google Scholar 

  • Stolzmann, W., Butz, M. V., Hoffmann, J., and Goldberg, D. E. (2000). First cognitive capabilities in the anticipatory classifier system. In Meyer, J.-A., Berthoz, A., Floreano, D., Roitblat, H., and Wilson, S. W. (Eds.), From Animals to Animats 6: Proceedings of the Sixth International Conference on Simulation of Adaptive Behavior pp. 287–296. Cambridge, MA: MIT Press.

    Google Scholar 

  • Sutton, R. S. (1990). Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In Proceedings of the Seventh International Conference on Machine Learning pp. 216–224. San Mateo, CA: Morgan Kaufmann.

    Google Scholar 

  • Sutton, R. S., and Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.

    Google Scholar 

  • Thrun, S. B. (1992). The role of exploration in learning control. In White, D.A. adn Sofge, D. (Ed.), Handbook of Intelligent Control: Neural, Fuzzy and Adaptive Approaches New York, NY: Van Nostrand Reinhold.

    Google Scholar 

  • Tomlinson, A., and Bull, L. (2000). A corporate XCS. In Lanzi, P. L., Stolzmann, W., and Wilson, S. W. (Eds.), Learning Classifier Systems: From Foundations to Applications, LNAI 1813 pp. 195–208. Berlin Heidelberg: Springer-Verlag.

    Chapter  Google Scholar 

  • Venturini, G. (1994). Adaptation in dynamic environments through a minimal probability of exploration. In Cliff, D., Husbands, P., Meyer, J.-A., and Wilson, S. W. (Eds.), From Animals to Animats 3: Proceedings of the Third International Conference on Simulation of Adaptive Behavior pp. 371–381. Cambridge, MA: MIT Press.

    Google Scholar 

  • Watkins, C. J. C. H. (1989). Learning from delayed rewards. Doctoral dissertation, King’s College, Cambridge, UK.

    Google Scholar 

  • Wilson, S. W. (1995). Classifier fitness based on accuracy. Evolutionary Computation, 3(2), 149–175.

    Article  Google Scholar 

  • Wilson, S. W. (1996). Explore/exploit strategies in autonomy. In Maes, P., Matariac, M. adn Pollak, J., Meyer, J.-A., and Wilson, S. (Eds.), From Animals to Animats 4: Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior pp. 325–332. Cambridge, MA: MIT Press.

    Google Scholar 

  • Wilson, S. W. (1998). Generalization in the XCS classifier system. In Koza, J. R., Banzhaf, W., Chellapilla, K., Deb, K., Dorigo, M., Fogel, D., Grazon, M., Goldberg, D., Iba, H., and Riolo, R. (Eds.), Genetic Programming 1998: Proceedings of the Third Annual Conference pp. 665–674. San Francisco: Morgan Kaufmann.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Cognitive Psychology, University of Würzburg, Röntgenring 11, 97070, Würzburg, Germany

    Martin V. Butz

Authors
  1. Martin V. Butz
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dipartimento di Elettronica e Informazione Artificial Intelligence and Robotics Laboratory, Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133, Milan, Italy

    Pier Luca Lanzi

  2. Research and Technology, Cognition and Robotics, DaimlerChrysler AG, Alt-Moabit 96A, 10559, Berlin, Germany

    Wolfgang Stolzmann

  3. Prediction Dynamics, Concord, MA, 01742, USA

    Stewart W. Wilson

  4. Department of General Engineering, The University of Illinois, Urbana-Champaign, IL, 61801, USA

    Stewart W. Wilson

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Butz, M.V. (2002). Biasing Exploration in an Anticipatory Learning Classifier System. In: Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds) Advances in Learning Classifier Systems. IWLCS 2001. Lecture Notes in Computer Science(), vol 2321. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-48104-4_1

Download citation

  • DOI: https://doi.org/10.1007/3-540-48104-4_1

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-43793-2

  • Online ISBN: 978-3-540-48104-1

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics