skip to main content
10.1145/1276958.1277315acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
Article

Heuristic speciation for evolving neural network ensemble

Published: 07 July 2007 Publication History

Abstract

Speciation is an important concept in evolutionary computation. It refers to an enhancements of evolutionary algorithms to generate a set ofdiverse solutions. The concept is studied intensively in the evolutionary design of neural network ensembles. Thediversity and cooperation of individual networks are among the essential criteria of the design.This paper proposes a speciation framework for ensemble design which integratesa collection of new techniques. Its characteristic features are:(a) the population of networks are speciated as such thatthe mutual information between the networks' outputs and genotypic representations is preserved. (b) The ensemble is designed incrementally,upon discovery of a species of networks which enhances the ensembleperformance. (c) Multiple species are evolved andindividual networks are evaluated according to therole of their respective species in the ensemble.This framework provides an implementation of evolutionary algorithm which performs simultaneous single-objective optimizations.The new algorithm is evaluated with a series of classification benchmarks andshows an improvement over other evolutionary training strategiesand a statistical algorithm.

References

[1]
S. Ando, J. Sakuma, and S. Kobayashi. Adaptiveisolation model using data clustering for multimodal function optimization. In GECCO '05: Proceedings of the 2005 conference on Genetic and evolutionary computation, pp. 1417--1424, New York, NY, USA, 2005. ACM Press.
[2]
S. Ando and E. Suzuki. An information theoretic approach to detection of minority subsets in database. In ICDM '06: Proceedings of the Sixth International Conference on Data Mining, pp. 11--20, Washington, DC, USA, 2006. IEEE Computer Society.
[3]
D. Beasley, D. R. Bull, and R. R. Martin. A sequential niche technique for multimodal function optimization. Evolutionary Computation, 1(2):101--125, 1993.
[4]
L. Breiman. Stacked regressions. Machine Learning, 24(1):49--64, 7 1996.
[5]
A. D. Cioppa, C. D. Stefano, and A. Marcelli. On the role of population size and niche radius in fitness sharing. IEEE Trans. Evolutionary Computation, 8(6):580--592, 2004.
[6]
P. J. Darwen and X. Yao. Speciation as automatic categorical modularization. IEEE Trans. Evolutionary Computation, 1(2):101--108, 1997.
[7]
K. A. De Jong. An analysis of behavior of a class of genetic adaptive systems. Ph.D. thesis,University of Michigan, 1975.
[8]
K. DEB, A. Anand, and D. Joshi. A computationally efficient evolutionary algorithm for real--parameter optimization. Evolutionary Computation, 10:371--395, 2002.
[9]
K. Deb and D. E. Goldberg. An investigation of niche and species formation in genetic function optimization. In J. D. Schaffer, editor, Proceedings of the Third International Conference on Genetic Algorithms, pp. 42--50, San Mateo, California, 1989. Morgan Kaufmann.
[10]
Y. Freund and R. E. Schapire. Experiments with a new boosting algorithm. In ICML, pp. 148--156, 1996.
[11]
N. García-Pedrajas, C. Hervás-Martíez, and D. Ortiz-Boyer. Cooperative coevolution of artificial neural network ensembles for pattern classification. IEEE Trans. Evolutionary Computation, 9(3):271--302, 2005.
[12]
D. E. Goldberg and J. Richardson. Genetic algorithms with sharing for multimodal function optimization. In Proceedings of the Second International Conference on Genetic Algorithms, pp. 41--49, Hillsdale, New Jersey, 1987.
[13]
V. Hanagandi and M. Nikolaou. A hybrid approach to global optimization using a clustering algorithm in a genetic search framework. Computers and Chemical Engineering, 22(12):1913--1925, November 1998.
[14]
T. Higuchi, S. Tsutsui, and M. Yamamura. Theoretical analysis of simplex crossover for real--coded genetic algorithms. In PPSN VI: Proceedings of the 6th International Conference on Parallel Problem Solving from Nature, pp. 365--374, London, UK, 2000. Springer--Verlag.
[15]
K. Ikeda and S. Kobayashi. GA based on the UV--structure hypothesis and its application to JSP. In Parallel Problem Solving from Nature(PPSN VI), volume 1281 of Lecture Notes in Computer Science, pp. 273--282. Springer--Verlag, March 2000.
[16]
M. M. Islam, X. Yao, and K. Murase. A constructive algorithm for training cooperative neural network ensembles. IEEE Transactions on Neural Networks, pp. 820--834, 2003.
[17]
J. Li, M. Balazs, G. Parks, and P. Clarkson. A genetic algorithm using species conservation for multimodal function optimization. Evolutionary Computation, 10(3):207--234, 2002.
[18]
Y. Liu and X. Yao. Simultaneous training of negatively correlated neural networks in an ensemble. IEEE Transactions on Systems, Man and Cybernetics, Part B, 29(6):716--725, December 1999.
[19]
Y. Liu, X. Yao, and T. Higuchi. Evolutionary ensembles with negative correlation learning. IEEE Trans. Evolutionary Computation, 4(4):380--387, 2000.
[20]
S. W. Mahfoud. Crowding and preselection revisited. In R. Männer and B. Manderick, editors, Parallel problem solving from nature 2, pp. 27--36, Amsterdam, North--Holland, 1992.
[21]
A. Martinez-Estudillo, C. Hervas-Martinez, F. Martinez-Estudillo, and N. Garcia--Pedrajas. Hybridization of evolutionary algorithms and local search by means of a clustering method. IEEE Transactions on Systems, Man and Cybernetics, Part B, 36(3):534--545, 2006.
[22]
O. Mengshoel and D. Goldberg. Probabilistic crowding: Deterministic crowding with probabilistic replacement. In W. B. et al., editor, Proceedings of the Genetic and Evolutionary Computation Conference, pp. 409--416. Morgan KauRmann, 1999.
[23]
M. A. Potter and K. A. D. Jong. Cooperative coevolution: An architecture for evolving coadapted subcomponents. Evol. Comput., 8(1):1.29, 2000.
[24]
G. Singh and D. Kalyanmoy Deb. Comparison of multi--modal optimization algorithms based on evolutionary algorithms. In GECCO '06: Proceedings of the 8th annual conference on Genetic and evolutionary computation, pp. 1305--1312, New York, NY, USA, 2006. ACM Press.
[25]
F. Streichert, G. Stein, H. Ulmer, and A. Zell. A clustering based niching ea for multimodal search spaces. In Proceedings of Evolution Artificielle (LNCS 2935), pp. 293--304. Springer--Verlag, 2003.
[26]
N. Tishby, F. C. Pereira, and W. Bialek. The information bottleneck method. Computing Research Repository(CoRR), physics/0004057, 2000.
[27]
R. K. Ursem. Multinational gas: Multimodal optimization techniques in dynamic environments. In Proceedings of the Genetic and Evolutionary Computation Conference, pp. 19--26. Morgan Kaufmann, 2000.
[28]
X. Yin and N. Germay. A fast genetic algorithm with sharing scheme using cluster methods in multimodal function optimization. In R. F. Albrecht, C. R. Reeves, and N. C. Steele, editors, Proceedings of the International Conference on Artificial Neural Nets and Genetic Algorithms, pp. 450--457. Springer--Verlag, 1993.

Cited By

View all
  • (2010)Creating Rule Ensembles from Automatically-Evolved Rule Induction AlgorithmsAdvances in Machine Learning I10.1007/978-3-642-05177-7_13(257-273)Online publication date: 2010

Index Terms

  1. Heuristic speciation for evolving neural network ensemble

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    GECCO '07: Proceedings of the 9th annual conference on Genetic and evolutionary computation
    July 2007
    2313 pages
    ISBN:9781595936974
    DOI:10.1145/1276958
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 07 July 2007

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. evolutionary design
    2. neural network
    3. niching
    4. pattern recognition

    Qualifiers

    • Article

    Conference

    GECCO07
    Sponsor:

    Acceptance Rates

    GECCO '07 Paper Acceptance Rate 266 of 577 submissions, 46%;
    Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)2
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 16 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2010)Creating Rule Ensembles from Automatically-Evolved Rule Induction AlgorithmsAdvances in Machine Learning I10.1007/978-3-642-05177-7_13(257-273)Online publication date: 2010

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media