research-article

Population-based coordinate descent algorithm with majority voting

Authors:

Davood Zaman Farsa,

Azam Asilian Bidgoli,

Ehsan Rokhsat-Yazdi,

Shahryar RahnamayanAuthors Info & Claims

GECCO '21: Proceedings of the Genetic and Evolutionary Computation Conference Companion

Pages 1283 - 1289

https://doi.org/10.1145/3449726.3463186

Published: 08 July 2021 Publication History

Abstract

Many real-world optimization problems belong to the class of expensive problems and the costly process of computing fitness value or gradient of the objective function may cause the failure of various optimization algorithms to solve them quickly. Because of the low computation and memory requirements of Coordinate Descent (CD) search methods they are suitable algorithms to optimize these problems. Despite the efficiency of CD methods, searching a large-scale search space just by using one candidate solution decreases the exploration capability of the algorithm. In this paper, a novel population-based version of the CD algorithm called Population-Based Coordinate Descent (PBCD) is proposed which is an efficacious method for tackling such problems using the collective intelligence and collaboration of the population. It takes advantage of three phases of locating the region of interest, folding the search space, and communication among the population members with majority voting to find more promising regions in the search space. As it shrinks the search space swiftly, it needs a low computational budget for finding the optimal value per coordinate and ultimately in overall. To investigate its performance, we benchmarked it on CEC-2017 test suite consisting of 29 low-scale problems with dimensions of 30, 50, and 100.

References

[1]

Léon Bottou, Frank E Curtis, and Jorge Nocedal. 2018. Optimization methods for large-scale machine learning. Siam Review 60, 2 (2018), 223--311.

[2]

Jui-Fang Chang, John Francis Roddick, Jeng-Shyang Pan, and Shu-Chuan Chu. 2005. A parallel particle swarm optimization algorithm with communication strategies. (2005).

[3]

Marco Dorigo, Mauro Birattari, and Thomas Stutzle. 2006. Ant colony optimization. IEEE computational intelligence magazine 1, 4 (2006), 28--39.

Digital Library

[4]

Davood Zaman Farsa and Shahryar Rahnamayan. 2020. Discrete Coordinate Descent (DCD). In 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 184--190.

Digital Library

[5]

Tobias Glasmachers and Urun Dogan. 2013. Accelerated coordinate descent with adaptive coordinate frequencies. In Asian Conference on Machine Learning. 72--86.

[6]

Nikolaus Hansen. 2008. Adaptive encoding: How to render search coordinate system invariant. In International Conference on Parallel Problem Solving from Nature. Springer, 205--214.

[7]

John H Holland. 1992. Genetic algorithms. Scientific american 267, 1 (1992), 66--73.

[8]

James Kennedy and Russell Eberhart. 1995. Particle swarm optimization. In Proceedings of ICNN'95-International Conference on Neural Networks, Vol. 4. IEEE, 1942--1948.

[9]

Ilya Loshchilov, Marc Schoenauer, and Michèle Sebag. 2011. Adaptive coordinate descent. In Proceedings of the 13th annual conference on Genetic and evolutionary computation. 885--892.

Digital Library

[10]

Heinz Mühlenbein. 1989. Parallel genetic algorithms, population genetics and combinatorial optimization. In Workshop on Parallel Processing: Logic, Organization, and Technology. Springer, 398--406.

Digital Library

[11]

Yu Nesterov. 2012. Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM Journal on Optimization 22, 2 (2012), 341--362.

[12]

G. E. Noether. 1992. Introduction to Wilcoxon (1945) Individual Comparisons by Ranking Methods. Springer Series in Statistics Breakthroughs in Statistics (1992), 191--195.

[13]

Shahryar Rahnamayan and Seyed Jalaleddin Mousavirad. 2020. Towards Solving Large-scale Expensive Optimization Problems Efficiently Using Coordinate Descent Algorithm. In 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2506--2513.

[14]

Herbert Robbins and Sutton Monro. 1951. A stochastic approximation method. The annals of mathematical statistics (1951), 400--407.

[15]

Ana Maria AC Rocha, M Fernanda P Costa, and Edite MGP Fernandes. 2019. A Population-Based Stochastic Coordinate Descent Method. In World Congress on Global Optimization. Springer, 16--25.

[16]

Sebastian Ruder. 2016. An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747 (2016).

[17]

Thomas Stützle. 1998. Parallelization strategies for ant colony optimization. In International Conference on Parallel Problem Solving from Nature. Springer, 722--731.

[18]

Stephen J Wright. 2015. Coordinate descent algorithms. Mathematical Programming 151, 1 (2015), 3--34.

Digital Library

[19]

Guohua Wu, Rammohan Mallipeddi, and Ponnuthurai Suganthan. 2016. Problem Definitions and Evaluation Criteria for the CEC 2017 Competition and Special Session on Constrained Single Objective Real-Parameter Optimization.

Cited By

Miyandoab SRahnamayan SBidgoli A(2023)Multi-Objective Binary Coordinate Search for Feature Selection2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC53992.2023.10394067(4176-4183)Online publication date: 1-Oct-2023
https://doi.org/10.1109/SMC53992.2023.10394067
Nikbakhtsarvestani FBidgoli AEbrahimi MRahnamayan S(2023)Multi-Objective Coordinate Search Optimization2023 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC53210.2023.10254106(1-9)Online publication date: 1-Jul-2023
https://doi.org/10.1109/CEC53210.2023.10254106

Index Terms

Population-based coordinate descent algorithm with majority voting
1. Theory of computation
  1. Design and analysis of algorithms
    1. Algorithm design techniques
      1. Divide and conquer

Recommendations

Majority voting for discrete population-based optimization algorithms

Population-based metaheuristic algorithms have been extensively applied to solve discrete optimization problems. Generally speaking, they work with a set of candidate solutions in the population which evolve during generations using variant reproduction ...
Adaptive coordinate descent
GECCO '11: Proceedings of the 13th annual conference on Genetic and evolutionary computation

Independence from the coordinate system is one source of efficiency and robustness for the Covariance Matrix Adaptation Evolution Strategy (CMA-ES). The recently proposed Adaptive Encoding (AE) procedure generalizes CMA-ES adaptive mechanism, and can be ...
Bi-population CMA-ES agorithms with surrogate models and line searches
GECCO '13 Companion: Proceedings of the 15th annual conference companion on Genetic and evolutionary computation

In this paper, three extensions of the BI-population Covariance Matrix Adaptation Evolution Strategy with weighted active covariance matrix update (BIPOP-aCMA-ES) are investigated. First, to address expensive optimization, we benchmark a recently ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '21: Proceedings of the Genetic and Evolutionary Computation Conference Companion

July 2021

2047 pages

ISBN:9781450383516

DOI:10.1145/3449726

Editor:
Francisco Chicano
University of Malaga
,
General Chair:
Krzysztof Krawiec
Poznan University of Technology

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 July 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GECCO '21

Sponsor:

SIGEVO

GECCO '21: Genetic and Evolutionary Computation Conference

July 10 - 14, 2021

Lille, France

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
65
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Miyandoab SRahnamayan SBidgoli A(2023)Multi-Objective Binary Coordinate Search for Feature Selection2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC53992.2023.10394067(4176-4183)Online publication date: 1-Oct-2023
https://doi.org/10.1109/SMC53992.2023.10394067
Nikbakhtsarvestani FBidgoli AEbrahimi MRahnamayan S(2023)Multi-Objective Coordinate Search Optimization2023 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC53210.2023.10254106(1-9)Online publication date: 1-Jul-2023
https://doi.org/10.1109/CEC53210.2023.10254106

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten