skip to main content
10.1145/3071178.3071321acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
research-article

Time complexity reduction in efficient global optimization using cluster kriging

Published: 01 July 2017 Publication History

Abstract

Efficient Global Optimization (EGO) is an effective method to optimize expensive black-box functions and utilizes Kriging models (or Gaussian process regression) trained on a relatively small design data set. In real-world applications, such as experimental optimization, where a large data set is available, the EGO algorithm becomes computationally infeasible due to the time and space complexity of Kriging. Recently, the so-called Cluster Kriging methods have been proposed to reduce such complexities for the big data, where data sets are clustered and Kriging models are built on each cluster. Furthermore, Kriging models are combined in an optimal way for the prediction. In addition, we analyze the Cluster Kriging landscape to adopt the existing infill-criteria, e.g., the expected improvement. The approach is tested on selected global optimization problems. It is shown by the empirical studies that this approach significantly reduces the CPU time of the EGO algorithm while maintaining the convergence rate of the algorithm.

References

[1]
Ioannis Andrianakis and Peter G Challenor. 2012. The effect of the nugget on Gaussian process emulators of computer models. Computational Statistics & Data Analysis 56, 12 (2012), 4215--4228.
[2]
Thomas Bartz-Beielstein and Sandor Markon. 2004. Tuning search algorithms for real-world applications: A regression tree based approach. In Evolutionary Computation, 2004. CEC2004. Congress on, Vol. 1. IEEE, 1111--1118.
[3]
Anirban Basudhar, Christoph Dribusch, Sylvain Lacaze, and Samy Missoum. 2012. Constrained efficient global optimization with support vector machines. Structural and Multidisciplinary Optimization 46, 2 (2012), 201--221.
[4]
Leo Breiman, Jerome Friedman, Charles J Stone, and Richard A Olshen. 1984. Classification and regression trees. CRC press.
[5]
JE Dennis and Virginia Torczon. 1997. Managing approximation models in optimization. Multidisciplinary design optimization: State-of-the-art (1997), 330--347.
[6]
David Ginsbourger, Rodolphe Le Riche, and Laurent Carraro. 2010. Kriging is well-suited to parallelize optimization. In Computational Intelligence in Expensive Optimization Problems. Springer, 131--162.
[7]
Nikolaus Hansen. 2006. The CMA evolution strategy: a comparing review. In Towards a new evolutionary computation. Springer, 75--102.
[8]
Nikolaus Hansen and Andreas Ostermeier. 2001. Completely Derandomized Self-Adaptation in Evolution Strategies. Evolutionary computation 9, 2 (2001), 159--195.
[9]
D. Huang, T. T. Allen, W. I. Notz, and N. Zeng. 2006. Global Optimization of Stochastic Black-Box Systems via Sequential Kriging Meta-Models. Journal of Global Optimization 34, 3 (2006), 441--466.
[10]
Donald R Jones. 2001. A taxonomy of global optimization methods based on response surfaces. Journal of global optimization 21, 4 (2001), 345--383.
[11]
Donald R Jones, Matthias Schonlau, and William J Welch. 1998. Efficient global optimization of expensive black-box functions. Journal of Global optimization 13, 4 (1998), 455--492.
[12]
Jack P. C. Kleijnen, Wim van Beers, and Inneke van Nieuwenhuyse. 2012. Expected improvement in efficient global optimization through bootstrapped kriging. Journal of Global Optimization 54, 1 (2012), 59--73.
[13]
Daniel G. Krige. 1951. A Statistical Approach to Some Basic Mine Valuation Problems on the Witwatersrand. Journal of the Chemical, Metallurgical and Mining Society of South Africa 52, 6 (Dec. 1951), 119--139.
[14]
J Močkus. 1975. On Bayesian methods for seeking the extremum. In Optimization Techniques IFIP Technical Conference. Springer, 400--404.
[15]
Jonas Mockus. 1994. Application of Bayesian approach to numerical methods of global and stochastic optimization. Journal of Global Optimization 4, 4 (1994), 347--365.
[16]
Jonas Mockus. 2012. Bayesian approach to global optimization: theory and applications. Vol. 37. Springer Science & Business Media.
[17]
Douglas C Montgomery. 1991. Design and analysis of experiments. (1991).
[18]
Andrew Naish-Guzman and Sean Holden. 2007. The generalized FITC approximation. In Advances in Neural Information Processing Systems. 1057--1064.
[19]
Joaquin Quinonero-Candela, Carl Edward Rasmussen, and Christopher KI Williams. 2007. Approximation methods for gaussian process regression. Large-scale kernel machines (2007), 203--224.
[20]
C.E. Rasmussen and C.K.I. Williams. 2006. Gaussian Processes for Machine Learning. University Press Group Limited.
[21]
Olivier Roustant, David Ginsbourger, and Yves Deville. 2012. DiceKriging, DiceOptim: Two R packages for the analysis of computer experiments by kriging-based metamodeling and optimization. (2012).
[22]
Jerome Sacks, William J. Welch, Toby J. Mitchell, and Henry P. Wynn. 1989. Design and Analysis of Computer Experiments. Statist. Sci. 4, 4 (1989), 409--423.
[23]
T.J. Santner, B.J. Williams, and W. Notz. 2003. The Design and Analysis of Computer Experiments. Springer.
[24]
Edward Snelson and Zoubin Ghahramani. 2005. Sparse Gaussian processes using pseudo-inputs. In Advances in neural information processing systems. 1257--1264.
[25]
A. Sóbester, S.J. Leary, and A.J. Keane. 2004. A parallel updating scheme for approximating and optimizing high fidelity computer simulations. Structural and Multidisciplinary Optimization 27, 5 (2004), 371--383.
[26]
V Tresp. 2000. A Bayesian committee machine. Neural computation 12, 11 (2000), 2719--2741.
[27]
Bas van Stein, Hao Wang, Wojtek Kowalczyk, Thomas Bäck, and Michael Emmerich. 2015. Optimally Weighted Cluster Kriging for Big Data Regression. Springer International Publishing, Cham, 310--321.
[28]
Bas van Stein, Hao Wang, Wojtek Kowalczyk, Michael Emmerich, and Thomas Bäck. 2016. Fuzzy clustering for Optimally Weighted Cluster Kriging. In Fuzzy Systems (FUZZ-IEEE), 2016 IEEE International Conference on. IEEE, 939--945.
[29]
Bas van Stein, Hao Wang, Wojtek Kowalczyk, Michael Emmerich, and Thomas Bäck. 2017. Cluster-based Kriging Approximation Algorithms for Complexity Reduction. arXiv preprint arXiv:1702.01313 (2017).
[30]
Felipe A. C. Viana, Raphael T. Haftka, and Layne T. Watson. 2013. Efficient global optimization algorithm assisted by multiple surrogate techniques. Journal of Global Optimization 56, 2 (2013), 669--689.
[31]
Antanas Žilinskas. 1992. A review of statistical models for global optimization. Journal of Global Optimization 2, 2 (1992), 145--153.

Cited By

View all
  • (2024)Towards reliable uncertainty quantification via deep ensemble in multi-output regression taskEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.107871132(107871)Online publication date: Jun-2024
  • (2024)GeoRF: a geospatial random forestData Mining and Knowledge Discovery10.1007/s10618-024-01046-738:6(3414-3448)Online publication date: 1-Nov-2024
  • (2023)Treed Gaussian Process Regression for Solving Offline Data-Driven Continuous Multiobjective Optimization ProblemsEvolutionary Computation10.1162/evco_a_0032931:4(375-399)Online publication date: 1-Dec-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
GECCO '17: Proceedings of the Genetic and Evolutionary Computation Conference
July 2017
1427 pages
ISBN:9781450349208
DOI:10.1145/3071178
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2017

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. big-data
  2. efficient global optimization
  3. kriging
  4. surrogate-assisted optimization

Qualifiers

  • Research-article

Conference

GECCO '17
Sponsor:

Acceptance Rates

GECCO '17 Paper Acceptance Rate 178 of 462 submissions, 39%;
Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)10
  • Downloads (Last 6 weeks)0
Reflects downloads up to 19 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Towards reliable uncertainty quantification via deep ensemble in multi-output regression taskEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.107871132(107871)Online publication date: Jun-2024
  • (2024)GeoRF: a geospatial random forestData Mining and Knowledge Discovery10.1007/s10618-024-01046-738:6(3414-3448)Online publication date: 1-Nov-2024
  • (2023)Treed Gaussian Process Regression for Solving Offline Data-Driven Continuous Multiobjective Optimization ProblemsEvolutionary Computation10.1162/evco_a_0032931:4(375-399)Online publication date: 1-Dec-2023
  • (2023)The first AI4TSP competitionArtificial Intelligence10.1016/j.artint.2023.103918319:COnline publication date: 1-Jun-2023
  • (2023)Methods/ContributionsEnhancing Surrogate-Based Optimization Through Parallelization10.1007/978-3-031-30609-9_3(29-94)Online publication date: 30-May-2023
  • (2022)A Three-Level Radial Basis Function Method for Expensive OptimizationIEEE Transactions on Cybernetics10.1109/TCYB.2021.306142052:7(5720-5731)Online publication date: Jul-2022
  • (2022)A sparse multi-fidelity surrogate-based optimization method with computational awarenessEngineering with Computers10.1007/s00366-022-01766-839:5(3473-3489)Online publication date: 1-Dec-2022
  • (2020)Expected improvement versus predicted value in surrogate-based optimizationProceedings of the 2020 Genetic and Evolutionary Computation Conference10.1145/3377930.3389816(868-876)Online publication date: 25-Jun-2020
  • (2020)Gaussian Process Regression Ensemble Model for Network Traffic PredictionIEEE Access10.1109/ACCESS.2020.30263378(176540-176554)Online publication date: 2020
  • (2020)Prediction for Manufacturing Factors in a Steel Plate Rolling Smart Factory Using Data Clustering-Based Machine LearningIEEE Access10.1109/ACCESS.2020.29831888(60890-60905)Online publication date: 2020
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media