skip to main content
10.1145/1276958.1277319acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
Article

Do not match, inherit: fitness surrogates for genetics-based machine learning techniques

Published: 07 July 2007 Publication History

Abstract

A byproduct benefit of using probabilistic model-building genetic algorithms is the creation of cheap and accurate surrogate models. Learning classifier systems -- and genetics-based machine learning in general -- can greatly benefit from such surrogates which may replace the costly matching procedure of a rule against large data sets. In this paper we investigate the accuracy of such surrogate fitness functions when coupled with the probabilistic models evolved by the $\chi$-ary extended compact classifier system ($\chi$eCCS). To achieve such a goal, we show the need that the probabilistic models should be able to represent all the accurate basis functions required for creating an accurate surrogate. We also introduce a procedure to transform populations of rules based into dependency structure matrices (DSMs) which allows building accurate models of overlapping building blocks -- a necessary condition to accurately estimate the fitness of the evolved rules.

References

[1]
J.-F.M. Barthelemy and R. T. Haftka. Approximation concepts for optimum structural design-a review. Structural Optimization, 5:129--144, 1993.
[2]
E. Bernadó-Mansilla and J. M. Garrell-Guiu. MOLeCS: A MultiObjective Learning Classifier System. Proceedings of the 2000 Conference on Genetic and Evolutionary Computation, 1:390, 2000.
[3]
E. Bernadó-Mansilla, XLlorà, and I. Traus. MultiObjective Machine Learning, chapter MultiObjective Learning Classifier System, pages 261--288. Springer, 2005.
[4]
A. A. Björk. Numerical method for least squares problems. SIAM, Philadelphia, PA, 1996.
[5]
G. Box, J. Hunter, W.G.and Hunter, and W. Hunter. Statistics for Experimenters: Design, Innovation, and Discovery. Wiley, 2005.
[6]
M. V. Butz, M. Pelikan, XLlorà, and DE. Goldberg. Extracted global structure makes local building block processing effective in XCS. Proceedings of the 2005 Conference on Genetic and Evolutionary Computation, 1:655--662, 2005.
[7]
L. de la Ossa, K. Sastry, and F. G. Lobo. Extended compact genetic algorithm in C++: Version 1.1. IlliGAL Report No. 2006013, University of Illinois at Urbana--Champaign, Urbana, IL, March 2006.
[8]
N. R. Draper and H. Smith. Applied Regression Analysis. John Wiley & Sons, New York, USA, 1966.
[9]
C. Fernandez. Integration Analysis of Product Architecture to Support Effective Team Co-location. Master thesis, Massachusetts Institute of Technology, 1998.
[10]
D. E. Goldberg. Genetic algorithms and Walsh functions: Part I, a gentle introduction. Complex Systems, 3:129----152, 1989. (Also IllIGAL Report No. 88006).
[11]
D. E. Goldberg. The Design of Innovation: Lessons from and for Competent Genetic Algorithms. Kluwer Academic Publishers, Norwell, MA, 2002.
[12]
D. E. Goldberg, K. Deb, and J. H. Clark. Genetic algorithms, noise, and the sizing of populations. Complex Systems, 6:333--362, 1992. (Also IlliGAL Report No. 91010).
[13]
D. E. Goldberg, B. Korb, and K. Deb. Messy genetic algorithms: Motivation, analysis, and first results. Complex Systems, 3(5):493--530, 1989.
[14]
J. J. Grefenstette and J. M. Fitzpatrick. Genetic search with approximate function evaluations. Proceedings of the International Conference on Genetic Algorithms and Their Applications, pages 112--120, 1985.
[15]
G. R. Harik. Finding multimodal solutions using restricted tournament selection. Proceedings of the Sixth International Conference on Genetic Algorithms, pages 24--31, 1995. (Also IlliGAL Report No. 94002).
[16]
G. R. Harik, F. G. Lobo, and K. Sastry. Linkage learning via probabilistic modeling in the ECGA. In MPelikan, KSastry, and ECantú--Paz, editors, Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications, chapter~3. Springer, Berlin, {in press}. (Also IlliGAL Report No. 99010).
[17]
S. Haykin. Adaptive Filter Theory. Prentice Hall, 1996.
[18]
Y. Jin. A comprehensive survey of fitness approximation in evolutionary computation. Soft Computing Journal, 9(1):3--12, 2005.
[19]
T. Kailath, A. H. Sayed, and B. Hassibi. Linear estimation. Prentice-Hall, Upper Saddle River, NJ, 2000.
[20]
X. Llorà and K. Sastry. Fast rule matching for learning classifier systems via vector instructions. In Proceedings of the 2006 Genetic and Evolutionary Computation Conference (GECCO 2006)}, page 1513--1520, 2006. (Also IlliGAL Report No. 2006001).
[21]
X. Llorà, K. Sastry, D. E. Goldberg, and L. de la Ossa. The $\chi$--ary extended compact classifier system: Linkage learning in pittsburgh lcs. In Proceedings of the 2006 Genetic and Evolutionary Computation Conference (GECCO 2006) Workshops: International Workshop on Learning Classifier Systems, 2006. (Also IlliGAL Report No. 2006015).
[22]
X. Llorà, K. Sastry, D. E. Goldberg, A. Gupta, and L. Lakshmi. Combating user fatigue in iGAs: Partial ordering, support vector machines, and synthetic fitness. In GECCO 2005: Proceedings of the 2005 conference on Genetic and evolutionary computation, volume 2, pages 1363--1370, Washington DC, USA, 25--29 June 2005. ACM Press.
[23]
X. Llorà, K. Sastry, D. E. Goldberg, A. Gupta, and L. Lakshmi. Combating user fatigue in iGAs: Partial ordering, support vector machines, and synthetic fitness. Proceedings of the Genetic and Evolutionary Computation Conference, pages 1363--1370, 2005. (Also IlliGAL Report No. 2005009).
[24]
S. W. Mahfoud. Population size and genetic drift in fitness sharing. Foundations of Genetic Algorithms, 3:185--224, 1994. (Also IlliGAL Report No. 94005).
[25]
M. Pelikan. Hierarchical Bayesian Optimization Algorithm: Toward a New Generation of Evolutionary Algorithm. Springer Verlag, Berlin, 2005.
[26]
M. Pelikan and K. Sastry. Fitness inheritance in the bayesian optimization algorithm. Proceedings of the Genetic and Evolutionary Computation Conference, 2:48--59, 2004. (Also IlliGAL Report No. 2004009).
[27]
CR. Rao and HToutenburg. Linear models: Least squares and alternatives. Springer, Berlin, 1999.
[28]
J. Rissinen. Modeling by shortest data description. Automatica, 14:465--471, 1978.
[29]
K. Sastry. Evaluation-relaxation schemes for genetic and evolutionary algorithms. Master's thesis, University of Illinois at Urbana-Champaign, General Engineering Department, Urbana, IL, 2001. (Also IlliGAL Report No. 2002004).
[30]
K. Sastry, H. A. Abbass, D. E. Goldberg, and D. D. Johnson. Sub-structural niching in estimation of distribution algorithms. Proceedings of the Genetic and Evolutionary Computation Conference, pages 671--678, 2005. (Also IlliGAL Report No. 2005003).
[31]
K. Sastry and D. E. Goldberg. Probabilistic model building and competent genetic programming. In RL. Riolo and BWorzel, editors, Genetic Programming Theory and Practise, chapter 13, pages 205--220. Kluwer, 2003.
[32]
K. Sastry and D. E. Goldberg. Designing competent mutation operators via probabilistic model building of neighborhoods. Proceedings of the Genetic and Evolutionary Computation Conference, 2:114--125, 2004. Also IlliGAL Report No. 2004006.
[33]
K. Sastry, C. F. Lima, and D. E. Goldberg. Evaluation relaxation using substructural information and linear estimation. In Proceedings of the 2006 Genetic and Evolutionary Computation Conference (GECCO 2006), pages 419--426, 2006. (Also IlliGAL Report No. 2006003).
[34]
K. Sastry, M. Pelikan, and D. E. Goldberg. Efficiency enhancement of genetic algorithms via building-block-wise fitness estimation. Proceedings of the {IEEE} International Conference on Evolutionary Computation}, pages 720--727, 2004. Also IlliGAL Report No. 2004010.
[35]
D Sharman, A Yassine, and P Carlile. Characterizing modular architectures. ASME 14th International Conference, pages DTM--34024, Sept. 2002.
[36]
R. Smith, B. Dike, and S. Stegmann. Fitness inheritance in genetic algorithms. In Proceedings of the ACM Symposium on Applied Computing, pages 345--350, New York, NY, USA, 1995. ACM.
[37]
D. V. Steward. The design structure system: {A} method for managing the design of complex systems. IEEE Transactions on Engineering Managment, 28:77--74, 1981.
[38]
S. Wilson. Classifier fitness based on accuracy. Evolutionary Computation, 3(2):149--175, 1995.
[39]
A. Yassine, D. R. Falkenburg, and K. Chelst. Engineering design management: An informatoin structure approach. International Journal of production research, 37(13):2957--2975, 1999.
[40]
T.-L. Yu and DE. Goldberg. Conquering hierarchical difficulty by explicit chunking: Substructural chromosome compression. In Proceedings of the 2006 Genetic and Evolutionary Computation Conference (GECCO 2006) Workshops: International Workshop on Learning Classifier Systems, pages 1385--1392, 2006. (Also IlliGAL Report No. 2006007).
[41]
T.-L. Yu, DE. Goldberg, AYassine, and Y.-P. Chen. A genetic algorithm design inspired by organizational theory: Pilot study of a dependency structure matrix driven genetic algorithm. Artificial Neural Networks in Engineering}, pages 327--332, 2003. (Also IlliGAL Report No. 2003007).

Cited By

View all
  • (2019)Smart adaptive run parameterization (SArP): enhancement of user manual selection of running parameters in fluid dynamic simulations using bio-inspired and machine-learning techniquesSoft Computing10.1007/s00500-019-03761-6Online publication date: 19-Jan-2019
  • (2016)Design Structure Matrix Extensions and Innovations: A Survey and New OpportunitiesIEEE Transactions on Engineering Management10.1109/TEM.2015.249128363:1(27-52)Online publication date: Feb-2016
  • (2015)Ant Colony and Surrogate Tree-Structured Models for Orderings-Based Bayesian Network LearningProceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation10.1145/2739480.2754806(543-550)Online publication date: 11-Jul-2015
  • Show More Cited By

Index Terms

  1. Do not match, inherit: fitness surrogates for genetics-based machine learning techniques

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    GECCO '07: Proceedings of the 9th annual conference on Genetic and evolutionary computation
    July 2007
    2313 pages
    ISBN:9781595936974
    DOI:10.1145/1276958
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 07 July 2007

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. DSMGA
    2. EDA
    3. fitness estimation
    4. genetics-based machine learning
    5. learning classifier system
    6. surrogate fitness
    7. xeCCS

    Qualifiers

    • Article

    Conference

    GECCO07
    Sponsor:

    Acceptance Rates

    GECCO '07 Paper Acceptance Rate 266 of 577 submissions, 46%;
    Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)2
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 16 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2019)Smart adaptive run parameterization (SArP): enhancement of user manual selection of running parameters in fluid dynamic simulations using bio-inspired and machine-learning techniquesSoft Computing10.1007/s00500-019-03761-6Online publication date: 19-Jan-2019
    • (2016)Design Structure Matrix Extensions and Innovations: A Survey and New OpportunitiesIEEE Transactions on Engineering Management10.1109/TEM.2015.249128363:1(27-52)Online publication date: Feb-2016
    • (2015)Ant Colony and Surrogate Tree-Structured Models for Orderings-Based Bayesian Network LearningProceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation10.1145/2739480.2754806(543-550)Online publication date: 11-Jul-2015
    • (2013)Large scale data mining using genetics-based machine learningProceedings of the 15th annual conference companion on Genetic and evolutionary computation10.1145/2464576.2480807(741-764)Online publication date: 6-Jul-2013
    • (2013)An application of a GA with Markov network surrogate to feature selectionInternational Journal of Systems Science10.1080/00207721.2012.68444944:11(2039-2056)Online publication date: 1-Nov-2013
    • (2013)GAssist vs. BioHELSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-013-1016-817:6(953-981)Online publication date: 1-Jun-2013
    • (2013)Large-scale data mining using genetics-based machine learningWiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery10.1002/widm.10783:1(37-61)Online publication date: 1-Jan-2013
    • (2012)Large scale data mining using genetics-based machine learningProceedings of the 14th annual conference companion on Genetic and evolutionary computation10.1145/2330784.2330936(1171-1196)Online publication date: 7-Jul-2012
    • (2011)Large scale data mining using genetics-based machine learningProceedings of the 13th annual conference companion on Genetic and evolutionary computation10.1145/2001858.2002137(1285-1310)Online publication date: 12-Jul-2011
    • (2010)Using a Markov network as a surrogate fitness function in a genetic algorithmIEEE Congress on Evolutionary Computation10.1109/CEC.2010.5586548(1-8)Online publication date: Jul-2010
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media