skip to main content
10.1145/1276958.1277321acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
Article

Empirical analysis of generalization and learning in XCS with gradient descent

Published: 07 July 2007 Publication History

Abstract

We analyze generalization and learning in XCS with gradient descent. At first, we show that the addition of gradient in XCS may slow down learning because it indirectly decreases the learning rate. However, in contrast to what was suggested elsewhere, gradient descent has no effect on the achieved generalization. We also show that when gradient descent is combined with roulette wheel selection, which is known to be sensitive to small values of the learning rate, the learning speed can slow down dramatically. Previous results reported no difference in the performance of XCS with gradient descent when roulette wheel selection or tournament selection were used. In contrast, we suggest that gradient descent should always be combined with tournament selection, which is not sensitive to the value of the learning rate. When gradient descent is used in combination with tournament selection, the results show that (i) the slowdown in learning is limited and (ii) the generalization capabilities of XCS are not affected.

References

[1]
Alwyn M. Barry. Limits in long path learning with XCS. In Springer-Verlag, editor, Genetic and Evolutionary Computation Conference (GECCO-2003), pages 1832--1843, Chicago, IL, 2003.
[2]
Alwyn M. Barry, John H. Holmes, and Xavier Llorà. Data mining using learning classifier systems. In Bull {5}, pages 15--67.
[3]
Ester Bernadó, Xavier Llorà, and Josep M. Garrell. XCS and GALE: A comparative study of two learning classifier systems on data mining. In Pier Luca Lanzi, Wolfgang Stolzmann, and Stewart W. Wilson, editors, Advances in Learning Classifier Systems, volume 2321 of LNAI, pages 115--132. Springer-Verlag, Berlin, 2002.
[4]
Justin A. Boyan and Andrew W. Moore. Generalization in reinforcement learning: Safely approximating the value function. In G. Tesauro et al., editors, Advances in Neural Information Processing Systems 7, pages 369--376, 1995. The MIT Press.
[5]
Larry Bull, editor. Applications of Learning Classifier Systems. Studies in Fuzziness & Soft Computing. Springer-Verlag, 2004.
[6]
Martin Butz, David G. Goldberg, and Pier Luca Lanzi. Gradient descent methods in learning classifier systems. Technical Report 2003028, Illinois Genetic Algorithms Laboratory - University of Illinois at Urbana-Champaign, 117 Transportation Building, 104 S. Mathews Avenue, Urbana, IL 61801, January 2003.
[7]
Martin V. Butz. Rule-Based Evolutionary Online Learning Systems: A Principled Approach to LCS Analysis and Design. Springer-Verlag, Berlin, 2006
[8]
Martin V. Butz, David E. Goldberg, and Pier Luca Lanzi. Gradient descent methods in learning classifier systems: Improving XCS performance in multistep problems. IEEE Transaction on Evolutionary Computation, 9(5):452--473, October 2005.
[9]
Martin V. Butz, Kumara Sastry, and David E. Goldberg. Strong, stable, and reliable fitness pressure in XCS due to tournament selection. Genetic Programming and Evolvable Machines, 6:53--77, 2005.
[10]
Martin V. Butz and Stewart W. Wilson. An algorithmic description of XCS. Journal of Soft Computing, 6(3-4):144--153, 2002.
[11]
Marco Dorigo. Genetic and non-genetic operators in alecsys. Evolutionary Computation, 1(2):151--164, 1993.
[12]
Jan Drugowitsch and Alwyn M. Barry. A formal framework and extensions for function approximation in learning classifier systems. Technical Report CSBU-2006-02, Department of Computer Science, University of Bath, January 2006.
[13]
Jan Drugowitsch and Alwyn M. Barry. A formal framework for reinforcement learning with function approximation in learning classifier systems. Technical Report CSBU-2006-02, Department of Computer Science, University of Bath, January 2006.
[14]
Geoffrey J. Gordon. Online fitted reinforcement learning from the value function approximation. Workshop on Value Function Approximation held during the 12th International Conference on Machine Learning, 1995.
[15]
Pier Luca Lanzi. An Analysis of Generalization in the XCS Classifier System. Evolutionary Computation Journal, 7(2):125--149, 1999.
[16]
Pier Luca Lanzi. Learning classifier systems from a reinforcement learning perspective. Soft Computing - A Fusion of Foundations, Methodologies and Applications, 6(3):162--170, 2002
[17]
Pier Luca Lanzi and Daniele Loiacono. Standard and averaging reinforcement learning in XCS. In GECCO '06: Proceedings of the 8th annual conference on Genetic and evolutionary computation, pages 1489--1496, New York, NY, USA, 2006. ACM Press.
[18]
Pier Luca Lanzi, Daniele Loiacono, Stewart W. Wilson, and David E. Goldberg. Classifier prediction based on tile coding. In GECCO '06: Proceedings of the 8th annual conference on Genetic and evolutionary computation, pages 1497--1504, New York, NY, USA, 2006. ACM Press.
[19]
Pier Luca Lanzi, Wolfgang Stolzmann, and Stewart W. Wilson, editors. Learning Classifier Systems: From Foundations to Applications, volume 1813 of Lecture Notes in Computer Science. Springer-Verlag, April 2000.
[20]
Stuart Ian Reynolds. Reinforcement Learning with Exploration. PhD thesis, School of Computer Science. The University of Birmingham, Birmingham, B15 2TT, December 2002.
[21]
Richard S. Sutton and Andrew G. Barto. Reinforcement Learning - An Introduction. MIT Press, 1998.
[22]
Atsushi Wada, Keiki Takadama, and Katsumori Shimohara. Learning classifier system equivalent with reinforcement learning with function approximation, 2005. Eighth International Workshop on Learning Classifier Systems (IWLCS-2005).
[23]
Atsushi Wada, Keiki Takadama, Katsumori Shimohara, and Osamu Katai. Learning classifier systems with convergence and generalization. In Larry Bull and Tim Kovacs, editors, Foundations of Learning Classifier Systems, volume 183 of Studies in Fuzziness and Soft Computing, pages 285--304. Springer, 2005.
[24]
Stewart W. Wilson. Classifier Fitness Based on Accuracy. Evolutionary Computation, 3(2):149--175, 1995. http://prediction-dynamics.com/.
[25]
Stewart W. Wilson. Generalization in the XCS classifier system. In Genetic Programming 1998: Proceedings of the Third Annual Conference, pages 665--674. Morgan Kaufmann, 1998.

Cited By

View all

Index Terms

  1. Empirical analysis of generalization and learning in XCS with gradient descent

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      GECCO '07: Proceedings of the 9th annual conference on Genetic and evolutionary computation
      July 2007
      2313 pages
      ISBN:9781595936974
      DOI:10.1145/1276958
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 07 July 2007

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. LCS
      2. RL
      3. XCS
      4. generalization
      5. gradient descent

      Qualifiers

      • Article

      Conference

      GECCO07
      Sponsor:

      Acceptance Rates

      GECCO '07 Paper Acceptance Rate 266 of 577 submissions, 46%;
      Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)3
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 16 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media