Skip to main content
SpringerLink
Log in

A learning classifier system with mutual-information-based fitness

  • Research Paper
  • Published:
Evolutionary Intelligence Aims and scope Submit manuscript

Abstract

This paper introduces a new variety of learning classifier system (LCS), called MILCS, which utilizes mutual information as fitness feedback. Unlike most LCSs, MILCS is specifically designed for supervised learning. We present experimental results, and contrast them to results from XCS, UCS, GAssist, BioHEL, C4.5 and Naïve Bayes. We discuss the explanatory power of the resulting rule sets. MILCS is also shown to promote the discovery of default hierarchies, an important advantage of LCSs. Final comments include future directions for this research, including investigations in neural networks and other systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. The prime of [P′] indicates the population set with one particular rule removed. Same applies to [M′] and [~M′]. However, these sets are not shown in Fig. 3.

References

  1. Bacardit J (2004) Pittsburgh genetics-based machine learning in the data mining era: representations, generalization, and run-time, PhD thesis, Ramon Llull University, Barcelona, Catalonia, Spain

  2. Bacardit J (2007) Personal communication

  3. Bacardit J, Stout M, Hirst JD, Sastry K, Llorà X, Krasnogor N (2007) Automated alphabet reduction method with evolutionary algorithms for protein structure prediction. In: Proceedings of the 9th annual conference on genetic and evolutionary computation (GECCO2007), ACM Press, pp 346–353

  4. Bernadó-Mansilla E (2003) Accuracy-based learning classifier systems: models, analysis and applications to classification tasks. Evol Comput 11(3):209–238

    Article  Google Scholar 

  5. Bull L (2009) Personal communication

  6. Butz MV (2003) Documentation of XCS+TS C-Code 1.2, IlliGAL report 2003023, University of Illinois at Urbana-Champaign, (Source code: ftp://gal2.ge.uiuc.edu/pub/src/XCS/XCS1.2.tar.Z)

  7. Butz M, Wilson SW (2001) An algorithmic description of XCS. In: Lanzi PL, Stolzmann W, Wilson SM (eds) Revised papers from the third international workshop on advances in learning classifier systems. Lecture notes in computer science, 1996. Springer, London, pp 253–272

  8. Fahlman SE, Lebiere C (1990) The cascade-correlation learning algorithm. In: Advances in neural information processing systems, vol 2. Morgan Kaufmann, pp 525–532

  9. Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley, Boston

    MATH  Google Scholar 

  10. Heckerman H (1996) A tutorial on learning with bayesian networks, Technical Report, MSR-TR-95-06

  11. Holland JH (1992) Adaptation in natural and artificial systems, 2nd edn. MIT Press, Cambridge

    Google Scholar 

  12. Holland J, Holyoak KJ, Nisbett RE, Thagard P (1986) Induction: processes of inference learning and discover. MIT Press, Cambridge

    Google Scholar 

  13. Kovacs T (2000) Strength or accuracy? Fitness calculation in learning classifier systems, learning classifier systems: an introduction to contemporary research. Springer, Berlin, pp 143–160

    Google Scholar 

  14. Kovacs T, Kerber M (2001) What makes a problem hard for XCS? In: Lanzi P, Stolzmann W, Wilson S (eds) Advances in learning classifer systems: proceedings of the third international workshop, volume 1996 of lecture notes in artificial intelligence. Springer, Berlin, pp 80–99

  15. Jiang MK (2009) MILCS: a mutual information based learning classifier system, PhD thesis, University College London, UK

  16. Langley P, Iba W, Thompson K (1992) An analysis of bayesian classifiers. In: Proceedings of the tenth national conference on artificial intelligence, AAAI Press, pp 223–228

  17. Lanzi PL, Loiacono D, Wilson SW, Goldberg DE (2005) XCS with computed prediction for the learning of boolean functions. Evol Comput 1:588–595

    Google Scholar 

  18. Orriols-Puig A (2008) Personal communication

  19. Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann, San Mateo

    Google Scholar 

  20. Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27:379–423 (623–656)

    MathSciNet  MATH  Google Scholar 

  21. Shannon CE (1949) Communication in the presence of noise. In: Proceedings of institute of radio engineers, vol 37, no. 1, pp 10–21

  22. Smith RE (1991) Default hierarchy formation and memory exploitation in learning classifier systems, PhD thesis, The University of Alabama, USA

  23. Smith RE, Behzadan B (2008) Mutual information neuro-evolutionary system (MINES), IEEE Congress on Evolutionary Computation (CEC) 2009, pp 1523–1529

  24. Smith RE, Cribbs HB (1994) Is a classifier system a type of neural network? Evol Comput 2(1):19–36

    Article  Google Scholar 

  25. Smith RE, Jiang MK (2007) A learning classifier system with mutual-information-based fitness. In: Proceedings of the 2007 congress on evolutionary computation, pp 2173–2180

  26. Stout M, Bacardit J, Hirst JD, Krasnogor N (2006) From HP lattice models to real proteins: coordination number prediction using learning classifier systems. In: Proceedings of the 4th European workshop on evolutionary computation and machine learning in bioinformatics. Lectures notes in computer science, 3907, Budapest, Hungary, pp 208–220

  27. Stout M, Bacardit J, Hirst JD, Smith RE, Krasnogor N (2008) Prediction of topological contacts in proteins using learning classifier systems. J Soft Comput Fusion Found, Methodol and Appl 13(3):245–258

    Google Scholar 

  28. Stubbs N, Park S (1996) Optimal sensor placement for mode shapes via Shannon.s sampling theorem. Microcomput Civ Eng 11:411–419

    Article  Google Scholar 

  29. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge

    Google Scholar 

  30. Wilson SW (1995) Classifier fitness based on accuracy. Evol Comput 3(2):149–175

    Article  Google Scholar 

  31. Wilson SW (1998) Generalization in the XCS classifier system. In: Koza JR, Banzhaf W, Chellapilla K, Deb K, Dorigo M, Fogel DB, Garzon MH, Goldberg DE, Iba H, Riolo R (eds). Genetic programming 1998: proceedings of the third annual conference, Morgan Kaufmann, pp 665–674

  32. Wilson SW (2001) Function approximation with a classifier system. In: Spector L et al. (eds) Proceedings of the genetic and evolutionary computation conference (GECCO-2001), Morgan Kaufmann, San Francisco, CA, pp 974–981

Download references

Acknowledgments

The authors greatly acknowledge support provided by the UK Engineering and Physical Sciences Research Council (EPSRC) under grant GR/T07541/01 & GR/T07534/01.

Author information

Authors and Affiliations

  1. Department of Computer Science, University College London, London, UK

    Robert Elliott Smith & Max Kun Jiang

  2. School of Computer Science, University of Nottingham, Nottingham, UK

    Jaume Bacardit, Michael Stout & Natalio Krasnogor

  3. School of Chemistry, University of Nottingham, Nottingham, UK

    Jonathan D. Hirst

Authors
  1. Robert Elliott Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Max Kun Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jaume Bacardit
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Stout
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Natalio Krasnogor
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jonathan D. Hirst
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Max Kun Jiang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Smith, R.E., Jiang, M.K., Bacardit, J. et al. A learning classifier system with mutual-information-based fitness. Evol. Intel. 3, 31–50 (2010). https://doi.org/10.1007/s12065-010-0037-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12065-010-0037-9

Keywords

Search

Navigation