technical-note

Selecting evolutionary operators using reinforcement learning: initial explorations

Authors:

GECCO Comp '14: Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary Computation

Pages 1033 - 1036

https://doi.org/10.1145/2598394.2605681

Published: 12 July 2014 Publication History

Get Access

Abstract

In evolutionary optimization, it is important to use efficient evolutionary operators, such as mutation and crossover. But it is often difficult to decide, which operator should be used when solving a specific optimization problem. So an automatic approach is needed. We propose an adaptive method of selecting evolutionary operators, which takes a set of possible operators as input and learns what operators are efficient for the considered problem. One evolutionary algorithm run should be enough for both learning and obtaining suitable performance. The proposed EA+RL(O) method is based on reinforcement learning. We test it by solving H-IFF and Travelling Salesman optimization problems. The obtained results show that the proposed method significantly outperforms random selection, since it manages to select efficient evolutionary operators and ignore inefficient ones.

References

[1]

M. Buzdalov, A. Buzdalova, and I. Petrova. Generation of Tests for Programming Challenge Tasks Using Multi-Objective Optimization. In C. Blum and E. Alba, editors, GECCO (Companion), pages 1655--1658. ACM, 2013.

Digital Library

Google Scholar

[2]

M. Buzdalov, A. Buzdalova, and A. Shalyto. A First Step towards the Runtime Analysis of Evolutionary Algorithm Adjusted with Reinforcement Learning. In Proceedings of the International Conference on Machine Learning and Applications, volume 1, pages 203--208. IEEE Computer Society, 2013.

Digital Library

Google Scholar

[3]

A. E. Eiben, M. Horvath, W. Kowalczyk, and M. C. Schut. Reinforcement learning for online control of evolutionary algorithms. In Proceedings of the 4th international conference on Engineering self-organising systems ESOA'06, pages 151--160. Springer-Verlag, Berlin, Heidelberg, 2006.

Digital Library

Google Scholar

[4]

A. E. Eiben, Z. Michalewicz, M. Schoenauer, and J. E. Smith. Parameter control in evolutionary algorithms. In Parameter Setting in Evolutionary Algorithms, pages 19--46. 2007.

Crossref

Google Scholar

[5]

L. P. Kaelbling, M. L. Littman, and A. W. Moore. Reinforcement Learning: A Survey. Journal of Artificial Intelligence Research, 4:237--285, 1996.

Crossref

Google Scholar

[6]

J. D. Knowles, R. A. Watson, and D. Corne. Reducing local optima in single-objective problems by multi-objectivization. In Proceedings of the First International Conference on Evolutionary Multi-Criterion Optimization, EMO '01, pages 269--283, London, UK, 2001. Springer-Verlag.

Digital Library

Google Scholar

[7]

P. Larrañaga, C. M. H. Kuijpers, R. H. Murga, I. Inza, and S. Dizdarevic. Genetic algorithms for the travelling salesman problem: A review of representations and operators. Artif. Intell. Rev., 13(2):129--170, Apr. 1999.

Digital Library

Google Scholar

[8]

M. Mitchell. An Introduction to Genetic Algorithms. MIT Press, Cambridge, MA, 1996.

Crossref

Google Scholar

[9]

S. Müller, N. N. Schraudolph, and P. D. Koumoutsakos. Step size adaptation in evolution strategies using reinforcement learning. In Proceedings of the Congress on Evolutionary Computation, pages 151--156. IEEE, 2002.

Crossref

Google Scholar

[10]

J. E. Pettinger and R. M. Everson. Controlling genetic algorithms with reinforcement learning. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '02, page 692, San Francisco, CA, USA, 2002. Morgan Kaufmann Publishers Inc.

Digital Library

Google Scholar

[11]

Y. Sakurai, K. Takada, T. Kawabe, and S. Tsuruta. A method to control parameters of evolutionary algorithms by using reinforcement learning. In Signal-Image Technology and Internet-Based Systems (SITIS), 2010 Sixth International Conference on, pages 74--79, 2010.

Digital Library

Google Scholar

[12]

D. S. Soper. Analysis of variance (anova) calculator - one-way anova from summary data {software}, 2014.

Google Scholar

[13]

R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, USA, 1998.

Digital Library

Google Scholar

[14]

R. A. Watson and J. B. Pollack. Analysis of recombinative algorithms on a hierarchical building-block problem. in foundations of genetic algorithms. Foundations of Genetic Algorithms (FOGA), 2000.

Google Scholar

Cited By

View all

Liu YLi JZou JHou ZYang SZheng J(2024)Continuous variation operator configuration for decomposition-based evolutionary multi-objective optimizationSwarm and Evolutionary Computation10.1016/j.swevo.2024.10164489(101644)Online publication date: Aug-2024
https://doi.org/10.1016/j.swevo.2024.101644
Song YWu YGuo YYan RSuganthan PZhang YPedrycz WDas SMallipeddi RAjani OFeng Q(2024)Reinforcement learning-assisted evolutionary algorithm: A survey and research opportunitiesSwarm and Evolutionary Computation10.1016/j.swevo.2024.10151786(101517)Online publication date: Apr-2024
https://doi.org/10.1016/j.swevo.2024.101517
Selmi AZerarka MCheriet A(2024)Innovative Clustering-Driven Techniques for Enhancing Initial Solutions in Euclidean Traveling Salesman Problems with Machine Learning IntegrationArabian Journal for Science and Engineering10.1007/s13369-024-09094-3Online publication date: 18-May-2024
https://doi.org/10.1007/s13369-024-09094-3
Show More Cited By

Index Terms

Selecting evolutionary operators using reinforcement learning: initial explorations
1. Computing methodologies
  1. Machine learning

Recommendations

Generic parameter control with reinforcement learning
GECCO '14: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation

Parameter control in Evolutionary Computing stands for an approach to parameter setting that changes the parameters of an Evolutionary Algorithm (EA) on-the-fly during the run. In this paper we address the issue of a generic and parameter-independent ...
To explore or to exploit: An entropy-driven approach for evolutionary algorithms

An evolutionary algorithm is an optimization process comprising two important aspects: exploration discovers potential offspring in new search regions; and exploitation utilizes promising solutions already identified. Intelligent balance between these ...
Self-adjusting Population Sizes for Non-elitist Evolutionary Algorithms: Why Success Rates Matter
Abstract
Evolutionary algorithms (EAs) are general-purpose optimisers that come with several parameters like the sizes of parent and offspring populations or the mutation rate. It is well known that the performance of EAs may depend drastically on these ...

Comments

Information & Contributors

Information

Published In

GECCO Comp '14: Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary Computation

July 2014

1524 pages

ISBN:9781450328814

DOI:10.1145/2598394

Editor-in-chief:
Christian Igel
Ruhr University of Bochum, University of Copenhagen
,
General Chair:
Dirk V. Arnold
Dalhousie University

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 July 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Technical-note

Funding Sources

Government of Russian Federation

Conference

GECCO '14

Sponsor:

SIGEVO

GECCO '14: Genetic and Evolutionary Computation Conference

July 12 - 16, 2014

BC, Vancouver, Canada

Acceptance Rates

GECCO Comp '14 Paper Acceptance Rate 180 of 544 submissions, 33%;

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
200
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Liu YLi JZou JHou ZYang SZheng J(2024)Continuous variation operator configuration for decomposition-based evolutionary multi-objective optimizationSwarm and Evolutionary Computation10.1016/j.swevo.2024.10164489(101644)Online publication date: Aug-2024
https://doi.org/10.1016/j.swevo.2024.101644
Song YWu YGuo YYan RSuganthan PZhang YPedrycz WDas SMallipeddi RAjani OFeng Q(2024)Reinforcement learning-assisted evolutionary algorithm: A survey and research opportunitiesSwarm and Evolutionary Computation10.1016/j.swevo.2024.10151786(101517)Online publication date: Apr-2024
https://doi.org/10.1016/j.swevo.2024.101517
Selmi AZerarka MCheriet A(2024)Innovative Clustering-Driven Techniques for Enhancing Initial Solutions in Euclidean Traveling Salesman Problems with Machine Learning IntegrationArabian Journal for Science and Engineering10.1007/s13369-024-09094-3Online publication date: 18-May-2024
https://doi.org/10.1007/s13369-024-09094-3
Yi WQu RJiao LNiu B(2023)Automated Design of Metaheuristics Using Reinforcement Learning Within a Novel General Search FrameworkIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.319729827:4(1072-1084)Online publication date: Aug-2023
https://doi.org/10.1109/TEVC.2022.3197298
Dantas APozo A(2022)The Impact of State Representation on Approximate Q-Learning for a Selection Hyper-heuristicIntelligent Systems10.1007/978-3-031-21686-2_4(45-60)Online publication date: 19-Nov-2022
https://doi.org/10.1007/978-3-031-21686-2_4
Sun JLiu XBack TXu Z(2021)Learning Adaptive Differential Evolution Algorithm From Optimization Experiences by Policy GradientIEEE Transactions on Evolutionary Computation10.1109/TEVC.2021.306081125:4(666-680)Online publication date: Aug-2021
https://doi.org/10.1109/TEVC.2021.3060811
Dantas APozo A(2021)Online Selection of Heuristic Operators with Deep Q-Network: A Study on the HyFlex FrameworkIntelligent Systems10.1007/978-3-030-91702-9_19(280-294)Online publication date: 28-Nov-2021
https://doi.org/10.1007/978-3-030-91702-9_19

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Generic parameter control with reinforcement learning

To explore or to exploit: An entropy-driven approach for evolutionary algorithms

Self-adjusting Population Sizes for Non-elitist Evolutionary Algorithms: Why Success Rates Matter

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations