skip to main content
10.1145/1276958.1277077acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
Article

Towards billion-bit optimization via a parallel estimation of distribution algorithm

Published: 07 July 2007 Publication History

Abstract

This paper presents a highly efficient, fully parallelized implementation of the compact genetic algorithm (cGA) to solve very large scale problems with millions to billions of variables. The paper presents principled results demonstrating the scalable solution of a difficult test function on instances over a billion variables using a parallel implementation of cGA. The problem addressed is a noisy, blind problem over a vector of binary decision variables. Noise is added equaling up to a tenth of the deterministic objective function variance of the problem, thereby making it difficult for simple hillclimbers to find the optimal solution. The compact GA, on the other hand, is able to find the optimum in the presence of noise quickly, reliably, and accurately, and the solution scalability follows known convergence theories. These results on noisy problem together with other results on problems involving varying modularity, hierarchy, and overlap foreshadow routine solution of billion-variable problems across the landscape of search problems.

References

[1]
S. Baluja. Population-based incremental learning: A method of integrating genetic search based function optimization and competitive learning. Technical Report CMU-CS-94-163, Carnegie Mellon University, 1994.
[2]
S. J. Benson, L. C. McInnes, and J. J. More. A case study in the performance and scalability of optimization algorithms. ACM Transactions on Mathematical Software 27(3):361--376, 2001.
[3]
B. Carter and K. Park. Scalability problems of genetic search. Proceedings of the 1994 IEEE International Conference on Systems, Man, and Cybernetics 2:1591--1596, 2004.
[4]
K. Deb and K. Pal. Efficiently solving: A large-scale integer linear program using a customized genetic algorithm. Proceedings of the 2004 Genetic and Evolutionary Computation Conference pages 1054--1065, 2004.
[5]
K. Deb, A. R. Reddy, and G. Singh. Optimal scheduling of casting sequence using genetic algorithms. Materials and Manufacturing Processes 18(3):409--432, 2003.
[6]
D. E. Goldberg. Genetic algorithms in search optimization and machine learning Addison-Wesley, Reading, MA, 1989.
[7]
D. E. Goldberg. Design of innovation: Lessons from and for competent genetic algorithms Kluwer Academic Publishers, Boston, MA, 2002.
[8]
D. E. Goldberg, K. Deb, and J. H. Clark. Genetic algorithms, noise, and the sizing of populations. Complex Systems 6:333--362, 1992. (Also IlliGAL Report No. 91010).
[9]
D. E. Goldberg, B. Korb, and K. Deb. Messy genetic algorithms: Motivation, analysis, and first results. Complex Systems 3(5):493--530, 1989. (Also IlliGAL Report No. 89003).
[10]
J. Gondzio and A. Grothey. Direct solution of linear systems of size 10 9 arising in optimization with interior point methods. Proceedings of the Paral lel Processing and Applied Mathematics (PPAM 2005) pages 513--525, 2006.
[11]
G. Harik, E. Cantú-Paz, D. E. Goldberg, and B. L. Miller. The gambler's ruin problem, genetic algorithms, and the sizing of populations. Evolutionary Computation 7(3):231--253, 1999. (Also IlliGAL Report No. 96004).
[12]
G. Harik, F. Lobo, and D. E. Goldberg. The compact genetic algorithm. Proceedings of the IEEE International Conference on Evolutionary Computation pages 523--528, 1998. (Also IlliGAL Report No. 97006).
[13]
J. H. Holland. Adaptation in Natural and Artificial Systems University of Michigan Press, Ann Arbor, MI, 1975.
[14]
L.-D. Lang and L. T. Biegler. Large-scale nonlinear programming with cape-open compliant interface. Chemical Engineering Research and Design 83(A6):718--723, 2005.
[15]
M. Matsumoto and T. Nishimura. Mersenne twister: A 623-dimensionally equidistributed uniform pseudorandom number generator. ACM Transactions on Modeling and Computer Simulation 8(1):3--30, 1998. http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/emt.html
[16]
B. L. Miller and D. E. Goldberg. Genetic algorithms, tournament selection, and the effects of noise. Complex Systems 9(3):193--212, 1995. (Also IlliGAL Report No. 95006).
[17]
H. Mühlenbein. How genetic algorithms really work: Mutation and hillclimbing. Parallel Problem Solving from Nature II pages 15--26, 1992.
[18]
H. Mühlenbein and G. Paaβ. From recombination of genes to the estimation of distributions I. Binary parameters. Parallel Problem Solving from Nature 4:178--187, 1996.
[19]
S. S. Nielsen and S. A. Zenios. Scalable parallel benders decomposition for stochastic linear programming. Parallel Computing 23:1069--1088, 1997.
[20]
S. Oh and S. Y. Shin. A parallel algorithm for large-scale linear programs with a special structure. Proceedings of IEEE Scalable High Performance Computing Conference pages 749--755, 1994.
[21]
K. Sastry. Evaluation-relaxation schemes for genetic and evolutionary algorithms. Master's thesis, University of Illinois at Urbana-Champaign, Urbana, IL, 2001. (Also IlliGAL Report No. 2002004).
[22]
K. Sastry and D. E. Goldberg. Let's get ready to rumble: Crossover versus mutation head to head. Proceedings of the 2004 Genetic and Evolutionary Computation Conference 2:126--137, 2004. Also IlliGAL Report No. 2004005.
[23]
K. Sastry, P. Winward, D. E. Goldberg, and C. F. Lima. Fluctuating crosstalk as a source of deterministic noise and its effects on ga scalability. Applications of Evolutionary Computing EvoWorkshops2006: EvoBIO, EvoCOMNET, EvoHOT, EvoIASP, EvoInteraction, EvoMUSART, EvoSTOCK pages 740--751, 2006. (Also IlliGAL Report No. 2005025).
[24]
Y. Semet and M. Schoenauer. An efficient memetic, permutation-based evolutionary algorithm for real-world train timetabling. Proceedings of the 2005 Congress on Evolutionary Computation pages 661--667, 2005.
[25]
D. Yang and S. A. Zenios. A scalable parallel interior point algorithm for stochastic linear programming and robust optimization. Computational Optimization and Applications 7:143--158, 1997.
[26]
T.-L. Yu. A matrix approach for finding extrema: Problems with modularity, hierarchy, and overlap PhDthesis, University of Illinois at Urbana-Champaign, Urbana, IL, 2006.

Cited By

View all
  • (2025)Evolutionary Sequential Transfer Learning for Multi-Objective Feature Selection in ClassificationIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2024.34517099:1(1019-1033)Online publication date: Feb-2025
  • (2025)Transfer Optimization for Heterogeneous Drone Delivery and Pickup ProblemIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2024.34066899:1(347-364)Online publication date: Feb-2025
  • (2024)A Subspace Gradient Descent Method for High-Dimensional Simulation-Based Optimization2024 China Automation Congress (CAC)10.1109/CAC63892.2024.10865606(6865-6869)Online publication date: 1-Nov-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
GECCO '07: Proceedings of the 9th annual conference on Genetic and evolutionary computation
July 2007
2313 pages
ISBN:9781595936974
DOI:10.1145/1276958
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2007

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. billion-variable optimization
  2. compact genetic algorithm
  3. convergence time
  4. efficiency enhancement
  5. large-scale optimization
  6. parallelization
  7. population sizing
  8. scalability analysis
  9. vectorization

Qualifiers

  • Article

Conference

GECCO07
Sponsor:

Acceptance Rates

GECCO '07 Paper Acceptance Rate 266 of 577 submissions, 46%;
Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)8
  • Downloads (Last 6 weeks)0
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Evolutionary Sequential Transfer Learning for Multi-Objective Feature Selection in ClassificationIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2024.34517099:1(1019-1033)Online publication date: Feb-2025
  • (2025)Transfer Optimization for Heterogeneous Drone Delivery and Pickup ProblemIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2024.34066899:1(347-364)Online publication date: Feb-2025
  • (2024)A Subspace Gradient Descent Method for High-Dimensional Simulation-Based Optimization2024 China Automation Congress (CAC)10.1109/CAC63892.2024.10865606(6865-6869)Online publication date: 1-Nov-2024
  • (2023)Metaheuristics in the Balance: A Survey on Memory-Saving Approaches for Platforms with Seriously Limited ResourcesInternational Journal of Intelligent Systems10.1155/2023/57080852023(1-32)Online publication date: 4-Nov-2023
  • (2022)A Review of Population-Based Metaheuristics for Large-Scale Black-Box Global Optimization—Part IIEEE Transactions on Evolutionary Computation10.1109/TEVC.2021.313083826:5(802-822)Online publication date: Oct-2022
  • (2020)A novel encoding for separable large-scale multi-objective problems and its application to the optimisation of housing stock improvementsApplied Soft Computing10.1016/j.asoc.2020.10665096:COnline publication date: 1-Nov-2020
  • (2019)Curbing Negative Influences Online for Seamless Transfer Evolutionary OptimizationIEEE Transactions on Cybernetics10.1109/TCYB.2018.286434549:12(4365-4378)Online publication date: Dec-2019
  • (2018)Insights on Transfer Optimization: Because Experience is the Best TeacherIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2017.27691042:1(51-64)Online publication date: Feb-2018
  • (2016)EDAs cannot be Balanced and StableProceedings of the Genetic and Evolutionary Computation Conference 201610.1145/2908812.2908895(1139-1146)Online publication date: 20-Jul-2016
  • (2015)Genetic Algorithm Using MapReduce-A Critical Reviewi-manager's Journal on Cloud Computing10.26634/jcc.2.4.49072:4(35-42)Online publication date: 15-Oct-2015
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media