research-article

CMA-TWEANN: efficient optimization of neural networks via self-adaptation and seamless augmentation

Authors:

Hirotaka Moriguchi,

Shinichi HonidenAuthors Info & Claims

GECCO '12: Proceedings of the 14th annual conference on Genetic and evolutionary computation

Pages 903 - 910

https://doi.org/10.1145/2330163.2330288

Published: 07 July 2012 Publication History

Abstract

Neuroevolutionary algorithms are successful methods for optimizing neural networks, especially for learning a neural policy (controller) in reinforcement learning tasks. Their significant advantage over gradient-based algorithms is the capability to search network topology as well as connection weights. However, state-of-the-art topology evolving methods are known to be inefficient compared to weight evolving methods with an appropriately hand-tuned topology. This paper introduces a novel efficient algorithm called CMA-TWEANN for evolving both topology and weights. Its high efficiency is achieved by introducing efficient topological mutation operators and integrating a state-of-the-art function optimization algorithm for weight optimization. Experiments on benchmark reinforcement learning tasks demonstrate that CMA-TWEANN solves tasks significantly faster than existing topology evolving methods. Furthermore, it outperforms weight evolving techniques even when they are equipped with a hand-tuned topology. Additional experiments reveal how and why CMA-TWEANN is the best performing weight evolving method.

References

[1]

D. Floreano, P. Dürr, and C. Mattiussi. Neuroevolution: from architectures to learning. Evolutionary Intelligence, 1(1):47--62, 2008.

[2]

F. Gomez, J. Schmidhuber, and R. Miikkulainen. Accelerated neural evolution through cooperatively coevolved synapses. The Journal of Machine Learning Research, 9:937--965, 2008.

Digital Library

[3]

N. Hansen, A. Auger, R. Ros, S. Finck, and P. Pošík. Comparing results of 31 algorithms from the black-box optimization benchmarking BBOB-2009. In Proceedings of GECCO '10. ACM, 2010.

Digital Library

[4]

N. Hansen, S. D. Müller, and P. Koumoutsakos. Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES). Evolutionary Computation, 11(1):1--18, 2003.

Digital Library

[5]

N. Hansen and A. Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evolutionary computation, 9(2):159--195, 2001.

Digital Library

[6]

V. Heidrich-Meisner and C. Igel. Neuroevolution strategies for episodic reinforcement learning. Journal of Algorithms-Cognition Informatics and Logic, 64(4):152--168, 2009.

Digital Library

[7]

C. Igel. Neuroevolution for reinforcement learning using evolution strategies. In Proceedings of CEC '03, pages 2588--2595. IEEE, 2003.

[8]

J. Metzen, M. Edgington, Y. Kassahun, and F. Kirchner. Analysis of an evolutionary reinforcement learning method in a multiagent domain. In Proceedings of AAMAS '08, pages 291--298, 2008.

Digital Library

[9]

J. Nelder and R. Mead. A simplex method for function minimization. The Computer Journal, 7(4):308, 1965.

[10]

S. Nolfi and D. Floreano. Evolutionary Robotics. The Biology, Intelligence, and Technology of Self-Organizing Machines. The MIT Press, Cambridge, MA, 2004.

Digital Library

[11]

K. Stanley and R. Miikkulainen. Evolving neural networks through augmenting topologies. Evolutionary Computation, 10(2):99--127, 2002.

Digital Library

[12]

K. Stanley and R. Miikkulainen. Efficient evolution of neural networks through complexification. The University of Texas at Austin, 2004.

[13]

R. S. Sutton and A. G. Barto. Reinforcement learning. The MIT Press, Cambridge, MA, 1998.

Digital Library

[14]

M. E. Taylor, S. Whiteson, and P. Stone. Comparing evolutionary and temporal difference methods in a reinforcement learning domain. In Proceedings of GECCO '06. ACM, 2006.

Digital Library

[15]

S. Whiteson and P. Stone. Evolutionary Function Approximation for Reinforcement Learning. The Journal of Machine Learning Research, 7:877--917, 2006.

Digital Library

[16]

X. Yao. Evolving artificial neural networks. Proceedings of the IEEE, 87(9):1423--1447, 1999.

Cited By

Bai HCheng RJin Y(2023)Evolutionary Reinforcement Learning: A SurveyIntelligent Computing10.34133/icomputing.00252Online publication date: 10-May-2023
https://doi.org/10.34133/icomputing.0025
Ohashi KFujiyoshi NSakamoto NAkimoto YTakadama K(2018)Model parameter adaptive instance-based policy optimization for episodic control tasks of nonholonomic systemsProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3205651.3208295(1426-1433)Online publication date: 6-Jul-2018
https://dl.acm.org/doi/10.1145/3205651.3208295
Fernández-Navarro FCruz MGutiérrez PCastaño AHervás-Martínez C(2018)Time series forecasting by recurrent product unit neural networksNeural Computing and Applications10.1007/s00521-016-2494-229:3(779-791)Online publication date: 1-Feb-2018
https://dl.acm.org/doi/10.1007/s00521-016-2494-2
Show More Cited By

Index Terms

CMA-TWEANN: efficient optimization of neural networks via self-adaptation and seamless augmentation
1. Computer systems organization
  1. Embedded and cyber-physical systems
    1. Robotics
2. Computing methodologies
  1. Artificial intelligence
    1. Control methods
      1. Robotic planning
    2. Planning and scheduling
      1. Robotic planning
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks

Recommendations

Evolving parsimonious networks by mixing activation functions
GECCO '17: Proceedings of the Genetic and Evolutionary Computation Conference

Neuroevolution methods evolve the weights of a neural network, and in some cases the topology, but little work has been done to analyze the effect of evolving the activation functions of individual nodes on network size, an important factor when ...
Topology and weight evolving artificial neural networks in cooperative transport by a robotic swarm
Abstract
This paper applies Topology and Weight Evolving Artificial Neural Networks (TWEANNs) to design controllers for a robotic swarm. A typical method of designing controllers by an evolutionary robotics approach uses neural networks as the robot ...
Evolving neural networks through augmenting topologies

An important question in neuroevolution is how to gain an advantage from evolving neural network topologies along with weights. We present a method, NeuroEvolution of Augmenting Topologies (NEAT), which outperforms the best fixed-topology method on a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '12: Proceedings of the 14th annual conference on Genetic and evolutionary computation

July 2012

1396 pages

ISBN:9781450311779

DOI:10.1145/2330163

Editor:
Terence Soule
University of Idaho, USA
,
General Chair:
Jason H. Moore
Dartmouth College, USA

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GECCO '12

Sponsor:

SIGEVO

GECCO '12: Genetic and Evolutionary Computation Conference

July 7 - 11, 2012

Pennsylvania, Philadelphia, USA

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
312
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)5

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bai HCheng RJin Y(2023)Evolutionary Reinforcement Learning: A SurveyIntelligent Computing10.34133/icomputing.00252Online publication date: 10-May-2023
https://doi.org/10.34133/icomputing.0025
Ohashi KFujiyoshi NSakamoto NAkimoto YTakadama K(2018)Model parameter adaptive instance-based policy optimization for episodic control tasks of nonholonomic systemsProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3205651.3208295(1426-1433)Online publication date: 6-Jul-2018
https://dl.acm.org/doi/10.1145/3205651.3208295
Fernández-Navarro FCruz MGutiérrez PCastaño AHervás-Martínez C(2018)Time series forecasting by recurrent product unit neural networksNeural Computing and Applications10.1007/s00521-016-2494-229:3(779-791)Online publication date: 1-Feb-2018
https://dl.acm.org/doi/10.1007/s00521-016-2494-2
Rodzin SRodzina ORodzina L(2016)Neuroevolution: Problems, algorithms, and experiments2016 IEEE 10th International Conference on Application of Information and Communication Technologies (AICT)10.1109/ICAICT.2016.7991745(1-4)Online publication date: Oct-2016
https://doi.org/10.1109/ICAICT.2016.7991745
Czajkowski MKretowski M(2016)The role of decision tree representation in regression problems - An evolutionary perspectiveApplied Soft Computing10.1016/j.asoc.2016.07.00748:C(458-475)Online publication date: 1-Nov-2016
https://dl.acm.org/doi/10.1016/j.asoc.2016.07.007
Müller AKaymaz AGabernet GPosselt GWessler SHiss JSchneider G(2016)Sparse Neural Network Models of Antimicrobial Peptide-Activity RelationshipsMolecular Informatics10.1002/minf.20160002935:11-12(606-614)Online publication date: 11-Jul-2016
https://doi.org/10.1002/minf.201600029
Kazawa TMiyamoto DGoto APark HFukuda TKanzaki R(2015)Toward the Whole Brain Simulation of Insect BrainThe Brain & Neural Networks10.3902/jnns.22.8922:3(89-102)Online publication date: 2015
https://doi.org/10.3902/jnns.22.89
Xu SInoue YInamura TMoriguchi HHoniden SBazzan AHuhns MLomuscio AScerri P(2014)Sample efficiency improvement on neuroevolution via estimation-based elimination strategyProceedings of the 2014 international conference on Autonomous agents and multi-agent systems10.5555/2615731.2616050(1537-1538)Online publication date: 5-May-2014
https://dl.acm.org/doi/10.5555/2615731.2616050
Moshaiov AAbramovich O(2014)Is MO-CMA-ES superior to NSGA-II for the evolution of multi-objective neuro-controllers?2014 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC.2014.6900433(2809-2816)Online publication date: Jul-2014
https://doi.org/10.1109/CEC.2014.6900433
Yu TYasuda TOhkura KMatsumura YGoka M(2013)Cooperative Transport by a Swarm Robotic System Based on CMA-NeuroES ApproachJournal of Advanced Computational Intelligence and Intelligent Informatics10.20965/jaciii.2013.p093217:6(932-942)Online publication date: 20-Nov-2013
https://doi.org/10.20965/jaciii.2013.p0932
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten