Skip to main content
SpringerLink
Log in

Accuracy, Parsimony, and Generality in Evolutionary Learning Systems via Multiobjective Selection

  • Conference paper
Learning Classifier Systems (IWLCS 2002)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 2661))

Included in the following conference series:

Abstract

Evolutionary learning systems (also known as Pittsburgh learningclassifier systems) need to balance accuracy and parsimony for evolving high quality general hypotheses. The learning process used in evolutionary learning systems is based on a set of training instances that sample the target concept to be learned. Thus, the learning process may overfit the learned hypothesis to the given set of training instances. In order to address some of these issues, this paper introduces a multiobjective approach to evolutionary learning systems. Thus, we translate the selection of promising hypotheses into a two-objective problem that looks for: (1) accurate (low error), and (2) compact (low complexity) solutions. Using the proposed multiobjective approach a set of compromise hypotheses are spread along the Pareto front. We also introduce a theory of the impact of noise when sampling the target concept to be learned, as well as the appearance of overfitted hypotheses as the result of perturbations on high quality generalization hypotheses in the Pareto front.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Holland, J.H.: Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence. MIT Press/Bradford Books (1975)

    Google Scholar 

  2. Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley Publishing Company, Inc., Reading (1989)

    Google Scholar 

  3. Wilson, S.W.: Classifier Fitness Based on Accuracy. Evolutionary Computation 3, 149–175 (1995)

    Article  Google Scholar 

  4. Butz, M.V.: Anticipatory learning classifier systems. Genetic Algorithms and Evolutionary Computation. Kluwer Academic Publishers, Boston, MA (2002)

    Google Scholar 

  5. Smith, S.F.: Flexible Learning of Problem Solving Heuristics through Adaptive Search. In: Proceedings of the 8th International Joint Conference on Artificial Intelligence, pp. 422–425 (1983)

    Google Scholar 

  6. De Jong, K.A., Spears, W.M.: Learning Concept Classification Rules Using Genetic Algorithms. In: Proceedings of the International Joint Conference on Artificial Intelligence, Sidney, Australia, pp. 651–656 (1991)

    Google Scholar 

  7. Janikow, C.: Inductive Learning of Decision Rules in Attribute-Based Examples: a Knowledge-Intensive Genetic Algorithm Approach. PhD thesis, University of North Carolina at Chapel Hill (1991)

    Google Scholar 

  8. Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs. Springer, Heidelberg (1992)

    MATH  Google Scholar 

  9. Llorá, X.: Genetic Based Machine Learning using Fine-grained Parallelism for Data Mining. PhD thesis, Enginyeria i Arquitectura La Salle. Ramon Llull University, Barcelona, Catalonia, European Union (February 2002)

    Google Scholar 

  10. Mitchell, T.M.: Machine Learning. McGraw-Hill, New York (1997)

    MATH  Google Scholar 

  11. Koza, J.R.: Genetic Programing: On the Programing of Computers by Means of Natural Selection (Complex Adaptive Systems). MIT Press, Cambridge (1992)

    Google Scholar 

  12. Altenberg, L.: Emergent phenomena in genetic programming. In: Proceedings of the Third Annual Conference on Evolutionary Programming, pp. 233–241 (1994)

    Google Scholar 

  13. Blickle, T., Thiele, L.: Genetic programming and redundancy. In: Genetic Algorithms within the Framework of Evolutionary Computation: Proceedings of the KI 1994 Workshop, pp. 33–38 (1994)

    Google Scholar 

  14. Blickle, T.: Evolving compact solutions in genetic programming: A case study. In: Ebeling, W., Rechenberg, I., Voigt, H.-M., Schwefel, H.-P. (eds.) PPSN 1996. LNCS, vol. 1141, pp. 564–573. Springer, Heidelberg (1996)

    Chapter  Google Scholar 

  15. Angeline, P.J.: Subtree crossover causes bloat. Genetic Programming 98, 745–752 (1998)

    Google Scholar 

  16. Langdon, W.B., Poli, R.: Fitness causes bloat: Mutation. In: Genetic Programming: First European Conference, pp. 37–48 (1998)

    Google Scholar 

  17. Soule, T., Foster, J.A.: Effects of code growth and parsimony pressure on populations in genetic programming. Evolutionary Computation 6, 293–309 (1998)

    Article  Google Scholar 

  18. Langdon, W.B.: Quadratic bloat in genetic programming. In: Proceedings of the Genetic and Evolutionary Computation Conference 2000, pp. 451–458 (2000)

    Google Scholar 

  19. Podgorelec, V., Kokol, P.: Fighting program bloat with the fractal complexity measure. In: Genetic Programming: Third European Conference, pp. 326–337 (2000)

    Google Scholar 

  20. Bleuler, S., Brack, M., Thiele, L., Zitzler, E.: Multiobjective genetic programming: Reducing bloat using SPEA2. In: Proceedings of the 2001 Congress on Evolutionary Computation CEC2001, COEX, World Trade Center, 159 Samseong-dong, Gangnam-gu, Seoul, Korea, pp. 536–543. IEEE Press, Los Alamitos (2001)

    Google Scholar 

  21. Banzhaf, W., Langdon, W.B.: Some Considerations on the Reason for Bloat. Genetic Programming and Evolvable Hardware 3, 81–91 (2002)

    Article  MATH  Google Scholar 

  22. Soule, T.: Exons and code growth in genetic programming. In: Foster, J.A., Lutton, E., Miller, J., Ryan, C., Tettamanzi, A.G.B. (eds.) EuroGP 2002. LNCS, vol. 2278, pp. 143–152. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  23. Garrell, J.M., Golobardes, E., Bernadó, E., Llorà, X.: Automatic Diagnosis with Genetic Algorithms and Case-Based Reasoning. AIENG 13, 367–372 (1999)

    Google Scholar 

  24. Bassett, J.K., De Jong, K.A.: Evolving Behaviors for Cooperating Agents. In: Ohsuga, S., Raś, Z.W. (eds.) ISMIS 2000. LNCS (LNAI), vol. 1932, pp. 157–165. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  25. Bacardit, J., Garrell, J.M.: Métodos de generalizatión para sistemas clasificadores de Pittsburgh. In: Primer Congreso Espaol de Algoritmos Evolutivos y Bioinspirados (AEB 2002), pp. 486–493 (2002)

    Google Scholar 

  26. Pareto, V.: Cours d’Economie Politique, vol. I &II. F. Rouge, Lausanne (1896)

    Google Scholar 

  27. Van Veldhuizen, D.A., Lamont, G.B.: Evolutionary computation and convergence to apareto front. In: Koza, J.R. (ed.) Late Breaking Papers at the Genetic Programming 1998 Conference, pp. 221–228. Omni Press, Madison (1998)

    Google Scholar 

  28. Coello-Coello, C.A.: An updated survey of GA-Based Multiobjective Optimization Techniques. Technical report lania-rd-09-08, Laboratorio Nacional de Informática Avanzada (LA-NIA), Xalapa, Veracruz, México (December 1998)

    Google Scholar 

  29. Van Veldhuizen, D.A., Lamont, G.B.: Multiobjective evolutionary algorithms: Analyzing the state-of-the-art. Evolutionary Computation 8, 125–147 (2000)

    Article  Google Scholar 

  30. Tackett, W.A.: Recombination, selection, and the genetic construction of computer programs. Unpublished doctoral dissertation. University of Southern California (1994)

    Google Scholar 

  31. Zitzler, E.: Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications. PhD thesis, Swiss Federal Institute of Technology (ETH), Zurich (1999)

    Google Scholar 

  32. Zitzler, E., Deb, K., Thiele, L.: Comparison of Multiobjective Evolutionary Algorithms: Empirical Results. Evolutionary Computation 8, 173–195 (2000)

    Article  Google Scholar 

  33. Zitzler, E.: SPEA2: Improving the Strength Pareto Evolutionary Algorithm. Technical report 103, Swiss Federal Institute of Technology (ETH) Zurich, Gloriastrasse 35, CH-8092 Zurich (May 2001)

    Google Scholar 

  34. Nordin, P., Banzhaf, W.: Complexity Compression and Evolution. In: Proceedings of the Sixth International Conference on Genetic Algorithms (1995)

    Google Scholar 

  35. Bernadó, E., Mekaouche, A., Garrell, J.M.: A Study of a Genetic Classifier System Based on the Pittsburgh Approach on a Medical Domain. In: Imam, I., Kodratoff, Y., El-Dessouki, A., Ali, M. (eds.) IEA/AIE 1999. LNCS (LNAI), vol. 1611, pp. 175–184. Springer, Heidelberg (1999)

    Google Scholar 

  36. Gómez-Skarmeta, A.F., Jiménez, F., Ibáez, J.: Pareto-optimality in fuzzy modeling. In: 6th European Congress on Intelligent Techniques and Soft Computing (EUFIT 1998), pp. 694–700 (1998)

    Google Scholar 

  37. Jiménez, F., Gómez-Skarmeta, A.F., Roubos, H., Robert, B.: Accurate, transparent, and compact fuzzy models for function approximation and dynamic modelling through multi-objective evolutionary optimization. In: Zitzler, E., Deb, K., Thiele, L., Coello Coello, C.A., Corne, D.W. (eds.) EMO 2001. LNCS, vol. 1993, pp. 653–667. Springer, Heidelberg (2001)

    Google Scholar 

  38. Breiman, L.: Bagging predictors. Machine Learning 24, 123–140 (1996)

    Google Scholar 

  39. Llorà, X., Garrell, J.M.: Automatic Classification and Artificial Life Models. In: Proceedings of Learning 2000 Workshop, IEEE and Univesidad Carlos III (2000)

    Google Scholar 

  40. Traus, I., Bernadó, E.: Sistema Classificador Pittsburgh basat en Estratégies Evolutives. Technical Report TR-ISRG-2002/0001, Enginyeria i Arquitectura La Salle, Universitat Ramon Llull, Barcelona, European Union (2002)

    Google Scholar 

  41. Llorà, X., Garrell, J.M.: Evolving Partially-Defined Instances with Evolutionary Algorithms. In: Proceedings of the 18th International Conference on Machine Learning (ICML 2001), pp. 337–344. Morgan Kaufmann Publishers, San Francisco (2001)

    Google Scholar 

  42. Llora, X., Garrell, J.M.: Knowledge-Independent Data Mining with Fine-Grained Parallel Evolutionary Algorithms. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2001), pp. 461–468. Morgan Kaufmann Publishers, San Francisco (2001)

    Google Scholar 

  43. Oei, C.K., Goldberg, D.E., Chang, S.J.: Tournament selection, niching, and the preservation of diversity. IlliGAL Report No. 91011, University of Illinois at Urbana-Champaign, Urbana, IL (1991)

    Google Scholar 

  44. Bäck, T.: Generalized convergence models for tournament- and (μ, λ )-selection. In: Proceedings of the Sixth International Conference on Genetic Algorithms, pp. 2–8 (1995)

    Google Scholar 

  45. Miller, B.L., Goldberg, D.E.: Genetic algorithms, tournament selection, and the effects of noise. Complex Systems 9, 193–212 (1995)

    Google Scholar 

  46. Schwefel, H.P.: Kybernetische Evolution als Strategie der experimentellen Forschung in der Strömungstechnik. Technical report, Diplomarbeit, Technische Universität Berlin (1965)

    Google Scholar 

  47. Schwefel, H.P.: Numerische Optimierung von Computer-Modellen mittels der Evolution-sstrategie. In: Interdisciplinary Systems Research, vol. 26, Birkhäuser, Basel (1977)

    Google Scholar 

  48. Bäck, T.: Evolutionary algorithms in theory and practice. Oxford University Press, New York (1996)

    Google Scholar 

  49. Wilson, S.W.: Get real! XCS with continuous-valued intpus. In: Booker, L., Forrest, S., Mitchell, M., Riolo, R.L. (eds.) Festschrift in Honor of John H. Holland, Center for the Study of Complex Systems, pp. 11–121 (1999)

    Google Scholar 

  50. Srinivas, N., Deb, K.: Multiobjective optimization using nondominated sorting in genetic algorithms (1994) (submitted to EC)

    Google Scholar 

  51. Srinivas, N., Deb, K.: Multiobjective optimization using nondominated sorting in genetic algorithms. Evolutionary Computation 2, 221–248 (1995)

    Article  Google Scholar 

  52. Deb, K., Agrawal, S., Pratab, A., Meyarivan, T.: A Fast Elitist Non-Dominated Sorting Genetic Algorithm for Multi-Objective Optimization: NSGA-II. KanGAL report 200001, Indian Institute of Technology (2000)

    Google Scholar 

  53. Goldberg, D.E., Richardson, J.: Genetic algorithms with sharing for multimodal function optimization. In: Proceedings of the Second International Conference on Genetic Algorithms, pp. 41–49 (1987)

    Google Scholar 

  54. Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees. Wadsworth International Group, Belmont (1984)

    MATH  Google Scholar 

  55. Merz, C.J., Murphy, P.M.: UCI Repository for Machine Learning Data-Bases. University of California, Department of Information and Computer Science, Irvine, CA (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html

  56. Bernadó, E., Llorà, X., Garrell, J.M.: XCS and GALE: a Comparative Study of Two Learning Classifier Systems with Six Other Learning Algorithms on Classification Tasks. In: Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds.) IWLCS 2001. LNCS (LNAI), vol. 2321, p. 115. Springer, Heidelberg (2002) (to appear)

    Google Scholar 

  57. Aha, D., Kibler, D.: Instance-based learning algorithms. Machine Learning 6, 37–66 (1991)

    Google Scholar 

  58. Quinlan, R.: Induction of decision trees. Machine Learning 1, 81–106 (1986)

    Google Scholar 

  59. Quinlan, R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, San Francisco (1993)

    Google Scholar 

  60. Frank, E., Witten, I.H.: Generating Accurate Rule Sets Without Global Optimization. In: Shavlik, J. (ed.) Machine Learning: Proceedings of the Fifteenth International Conference, pp. 144–151. Morgan Kaufmann, San Francisco (1998)

    Google Scholar 

  61. Witten, I.H., Eibe, F.: Data Mining. Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Francisco (2000)

    Google Scholar 

  62. Llorà, X., Goldberg, D.E.: Minimal Achievable Error in the LED problem. IlliGAL Report No. 2002015, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL (2002)

    Google Scholar 

  63. Llorà, X., Goldberg, D.E.: Bounding the effect of noise in Multiobjective Learning Classifier Systems. Evolutionary Computation (2003) (in press)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Illinois Genetic Algorithms Laboratory (IlliGAL), National Center for Supercomputing Applications, University of Illinois at Urbana-Champaign, 104 S. Mathews Ave, Urbana, IL, 61801, USA

    Xavier Llorà & David E. Goldberg

  2. Enginyeria i Arquitectura La Salle, Universitat Ramon Llull, Psg. Bonanova 8, 08022, Barcelona, Catalonia, Spain, European Union

    Ivan Traus & Ester Bernadó

Authors
  1. Xavier Llorà
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David E. Goldberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ivan Traus
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ester Bernadó
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dipartimento di Elettronica e Informazione, Politecnico di Milano, Milano, Italy

    Pier Luca Lanzi

  2. Daimler Chrysler AG, 71059, Sindelfingen, Germany

    Wolfgang Stolzmann

  3. Prediction Dynamics, Concord MA 01742 USA, Department of General Engineering, The University of Illinois at Urbana-Champaign, IL 61801, USA

    Stewart W. Wilson

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Llorà, X., Goldberg, D.E., Traus, I., Bernadó, E. (2003). Accuracy, Parsimony, and Generality in Evolutionary Learning Systems via Multiobjective Selection. In: Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds) Learning Classifier Systems. IWLCS 2002. Lecture Notes in Computer Science(), vol 2661. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-40029-5_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-40029-5_8

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-20544-9

  • Online ISBN: 978-3-540-40029-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation