Skip to main content
SpringerLink
Log in

Surrogate-Based Methods

  • Chapter
Computational Optimization, Methods and Algorithms

Part of the book series: Studies in Computational Intelligence ((SCI,volume 356))

Abstract

Objective functions that appear in engineering practice may come from measurements of physical systems and, more often, from computer simulations. In many cases, optimization of such objectives in a straightforward way, i.e., by applying optimization routines directly to these functions, is impractical. One reason is that simulation-based objective functions are often analytically intractable (discontinuous, non-differentiable, and inherently noisy). Also, sensitivity information is usually unavailable, or too expensive to compute. Another, and in many cases even more important, reason is the high computational cost of measurement/simulations. Simulation times of several hours, days or even weeks per objective function evaluation are not uncommon in contemporary engineering, despite the increase of available computing power. Feasible handling of these unmanageable functions can be accomplished using surrogate models: the optimization of the original objective is replaced by iterative re-optimization and updating of the analytically tractable and computationally cheap surrogate. This chapter briefly describes the basics of surrogate-based optimization, various ways of creating surrogate models, as well as several examples of surrogate-based optimization techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Bandler, J.W., Cheng, Q.S., Dakroury, S.A., Mohamed, A.S., Bakr, M.H., Madsen, K., Søndergaard, J.: Space mapping: the state of the art. IEEE Trans. Microwave Theory Tech. 52, 337–361 (2004)

    Article  Google Scholar 

  • Pironneau, O.: On optimum design in fluid mechanics. J. Fluid Mech. 64, 97–110 (1974)

    Article  MATH  MathSciNet  Google Scholar 

  • Queipo, N.V., Haftka, R.T., Shyy, W., Goel, T., Vaidynathan, R., Tucker, P.K.: Surrogate-based analysis and optimization. Progress in Aerospace Sciences 41, 1–28 (2005)

    Article  Google Scholar 

  • Forrester, A.I.J., Keane, A.J.: Recent advances in surrogate-based optimization. Prog. Aerospace Sciences 45, 50–79 (2009)

    Article  Google Scholar 

  • Koziel, S., Bandler, J.W., Madsen, K.: A space mapping framework for engineering optimization: theory and implementation. IEEE Trans. Microwave Theory Tech. 54, 3721–3730 (2006)

    Article  Google Scholar 

  • Koziel, S., Cheng, Q.S., Bandler, J.W.: Space mapping. IEEE Microwave Magazine 9, 105–122 (2008)

    Article  Google Scholar 

  • Alexandrov, N.M., Lewis, R.M.: An overview of first-order model management for engineering optimization. Optimization and Engineering 2, 413–430 (2001)

    Article  MATH  Google Scholar 

  • Echeverria, D., Hemker, P.W.: Space mapping and defect correction. CMAM Int. Mathematical Journal Computational Methods in Applied Mathematics 5, 107–136 (2005)

    MATH  MathSciNet  Google Scholar 

  • Booker, A.J., Dennis, J.E., Frank, P.D., Serafini, D.B., Torczon, V., Trosset, M.W.: A rigorous framework for optimization of expensive functions by surrogates. Structural Optimization 17, 1–13 (1999)

    Article  Google Scholar 

  • Simpson, T.W., Peplinski, J., Koch, P.N., Allen, J.K.: Metamodels for computer-based engineering design: survey and recommendations. Engineering with Computers 17, 129–150 (2001)

    Article  MATH  Google Scholar 

  • Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust Region Methods. MPS-SIAM Series on Optimization (2000)

    Google Scholar 

  • Alexandrov, N.M., Dennis, J.E., Lewis, R.M., Torczon, V.: A trust region framework for managing use of approximation models in optimization. Struct. Multidisciplinary Optim. 15, 16–23 (1998)

    Google Scholar 

  • Echeverría, D., Hemker, P.W.: Manifold mapping: a two-level optimization technique. Computing and Visualization in Science 11, 193–206 (2008)

    Article  MathSciNet  Google Scholar 

  • Koziel, S., Bandler, J.W., Madsen, K.: Quality assessment of coarse models and surrogates for space mapping optimization. Optimization Eng. 9, 375–391 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  • Koziel, S., Bandler, J.W.: Coarse and surrogate model assessment for engineering design optimization with space mapping. In: IEEE MTT-S Int. Microwave Symp. Dig, Honolulu, HI, pp. 107–110 (2007)

    Google Scholar 

  • Koziel, S., Bandler, J.W.: Space-mapping optimization with adaptive surrogate model. IEEE Trans. Microwave Theory Tech. 55, 541–547 (2007)

    Article  Google Scholar 

  • Alexandrov, N.M., Nielsen, E.J., Lewis, R.M., Anderson, W.K.: First-order model management with variable-fidelity physics applied to multi-element airfoil optimization. In: AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Design and Optimization, Long Beach, CA, AIAA Paper 2000-4886 (2000)

    Google Scholar 

  • Wu, K.-L., Zhao, Y.-J., Wang, J., Cheng, M.K.K.: An effective dynamic coarse model for optimization design of LTCC RF circuits with aggressive space mapping. IEEE Trans. Microwave Theory Tech. 52, 393–402 (2004)

    Article  Google Scholar 

  • Robinson, T.D., Eldred, M.S., Willcox, K.E., Haimes, R.: Surrogate-based optimization using multifidelity models with variable parameterization and corrected space mapping. AIAA Journal 46, 2814–2822 (2008)

    Article  Google Scholar 

  • Søndergaard, J.: Optimization using surrogate models – by the space mapping technique. Ph.D. Thesis, Informatics and Mathematical Modelling, Technical University of Denmark, Lyngby (2003)

    Google Scholar 

  • Kleijnen, J.P.C.: Kriging metamodeling in simulation: a review. European Journal of Operational Research 192, 707–716 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  • Rayas-Sanchez, J.E.: EM-based optimization of microwave circuits using artificial neural networks: the state-of-the-art. IEEE Trans. Microwave Theory Tech. 52, 420–435 (2004)

    Article  Google Scholar 

  • Giunta, A.A., Wojtkiewicz, S.F., Eldred, M.S.: Overview of modern design of experiments methods for computational simulations. American Institute of Aeronautics and Astronautics, paper AIAA 2003–0649 (2003)

    Google Scholar 

  • Santner, T.J., Williams, B., Notz, W.: The Design and Analysis of Computer Experiments. Springer, Heidelberg (2003)

    MATH  Google Scholar 

  • Koehler, J.R., Owen, A.B.: Computer experiments. In: Ghosh, S., Rao, C.R. (eds.) Handbook of Statistics, vol. 13, pp. 261–308. Elsevier Science B.V., Amsterdam (1996)

    Google Scholar 

  • Cheng, Q.S., Koziel, S., Bandler, J.W.: Simplified space mapping approach to enhancement of microwave device models. Int. J. RF and Microwave Computer-Aided Eng. 16, 518–535 (2006)

    Article  Google Scholar 

  • McKay, M., Conover, W., Beckman, R.: A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21, 239–245 (1979)

    Article  MATH  MathSciNet  Google Scholar 

  • Beachkofski, B., Grandhi, R.: Improved distributed hypercube sampling. American Institute of Aeronautics and Astronautics, Paper AIAA 2002–1274 (2002)

    Google Scholar 

  • Leary, S., Bhaskar, A., Keane, A.: Optimal orthogonal-array-based latin hypercubes. Journal of Applied Statistics 30, 585–598 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  • Ye, K.Q.: Orthogonal column latin hypercubes and their application in computer experiments. Journal of the American Statistical Association 93, 1430–1439 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  • Palmer, K., Tsui, K.-L.: A minimum bias latin hypercube design. IIE Transactions 33, 793–808 (2001)

    Google Scholar 

  • Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. The Johns Hopkins University Press, Baltimore (1996)

    MATH  Google Scholar 

  • Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. MPS-SIAM Series on Optimization, MPS-SIAM (2009)

    Google Scholar 

  • Wild, S.M., Regis, R.G., Shoemaker, C.A.: ORBIT: Optimization by radial basis function interpolation in trust-regions. SIAM J. Sci. Comput. 30, 3197–3219 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  • Journel, A.G., Huijbregts, C.J.: Mining Geostatistics. Academic Press, London (1981)

    Google Scholar 

  • O’Hagan, A.: Curve fitting and optimal design for predictions. Journal of the Royal Statistical Society B 40, 1–42 (1978)

    MATH  MathSciNet  Google Scholar 

  • Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)

    MATH  Google Scholar 

  • Jones, D., Schonlau, M., Welch, W.: Efficient global optimization of expensive black-box functions. Journal of Global Optimization 13, 455–492 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  • Haykin, S.: Neural Networks: A Comprehensive Foundation, 2nd edn. Prentice-Hall, Englewood Cliffs (1998)

    Google Scholar 

  • Minsky, M.I., Papert, S.A.: Perceptrons: An Introduction to Computational Geometry. MIT Press, Cambridge (1969)

    MATH  Google Scholar 

  • Gunn, S.R.: Support vector machines for classification and regression. Technical Report. School of Electronics and Computer Science, University of Southampton (1998)

    Google Scholar 

  • Angiulli, G., Cacciola, M., Versaci, M.: Microwave devices and antennas modeling by support vector regression machines. IEEE Trans. Magn. 43, 1589–1592 (2007)

    Article  Google Scholar 

  • Smola, A.J., Schölkopf, B.: A tutorial on support vector regression. Statistics and Computing 14, 199–222 (2004)

    Article  MathSciNet  Google Scholar 

  • Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)

    MATH  Google Scholar 

  • Levin, D.: The approximation power of moving least-squares. Mathematics of Computation 67, 1517–1531 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  • Aitken, A.C.: On least squares and linear combinations of observations. Proceedings of the Royal Society of Edinburgh 55, 42–48 (1935)

    Google Scholar 

  • Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Massachussets (2006)

    Google Scholar 

  • Geisser, S.: Predictive Inference. Chapman and Hall, Boca Raton (1993)

    MATH  Google Scholar 

  • Koziel, S., Cheng, Q.S., Bandler, J.W.: Implicit space mapping with adaptive selection of preassigned parameters. IET Microwaves, Antennas & Propagation 4, 361–373 (2010)

    Article  Google Scholar 

  • Alexandrov, N.M., Lewis, R.M., Gumbert, C.R., Green, L.L., Newman, P.A.: Approximation and model management in aerodynamic optimization with variable-fidelity models. AIAA Journal of Aircraft 38, 1093–1101 (2001)

    Article  Google Scholar 

  • Moré, J.J.: Recent developments in algorithms and software for trust region methods. In: Bachem, A., Grötschel, M., Korte, B. (eds.) Mathematical Programming. The State of Art, pp. 258–287. Springer, Heidelberg (1983)

    Google Scholar 

  • Leary, S.J., Bhaskar, A., Keane, A.J.: A constraint mapping approach to the structural optimization of an expensive model using surrogates. Optimization and Engineering 2, 385–398 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  • Redhe, M., Nilsson, L.: Optimization of the new Saab 9-3 exposed to impact load using a space mapping technique. Structural and Multidisciplinary Optimization 27, 411–420 (2004)

    Google Scholar 

  • Koziel, S., Bandler, J.W., Cheng, Q.S.: Robust trust-region space-mapping algorithms for microwave design optimization. IEEE Trans. Microwave Theory and Tech. 58, 2166–2174 (2010)

    Article  Google Scholar 

  • Koziel, S., Bandler, J.W., Cheng, Q.S.: Adaptively constrained parameter extraction for robust space mapping optimization of microwave circuits. IEEE MTT-S Int. Microwave Symp. Dig., 205–208 (2010)

    Google Scholar 

  • Echeverría, D.: Multi-Level optimization: space mapping and manifold mapping. Ph.D. Thesis, Faculty of Science, University of Amsterdam (2007)

    Google Scholar 

  • Koziel, S., Echeverría Ciaurri, D.: Reliable simulation-driven design optimization of microwave structures using manifold mapping. Progress in Electromagnetics Research B 26, 361–382 (2010)

    Article  Google Scholar 

  • Hemker, P.W., Echeverría, D.: A trust-region strategy for manifold mapping optimization. JCP Journal of Computational Physics 224, 464–475 (2007)

    Article  MATH  Google Scholar 

  • Echeverría, D.: Two new variants of the manifold-mapping technique. COMPEL The International Journal for Computation and Mathematics in Electrical Engineering 26, 334–344 (2007)

    Article  MATH  Google Scholar 

  • Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Review 45, 385–482 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  • Marsden, A.L., Wang, M., Dennis, J.E., Moin, P.: Optimal aeroacoustic shape design using the surrogate management framework. Optimization and Engineering 5, 235–262 (2004)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Engineering Optimization & Modeling Center, School of Science and Engineering, Reykjavik University, Menntavegur 1, 101, Reykjavik, Iceland

    Slawomir Koziel & Leifur Leifsson

  2. Department of Energy Resources Engineering, Stanford University, Stanford, CA, 94305-2220, USA

    David Echeverría Ciaurri

Authors
  1. Slawomir Koziel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Echeverría Ciaurri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Leifur Leifsson
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Science and Engineering Engineering Optimization & Modeling Center, Reykjavik University, Menntavegur 1, 101, Reykjavik, Iceland

    Slawomir Koziel

  2. Mathematics and Scientific Computing, National Physical Laboratory, TW11 0LW, Teddington, UK

    Xin-She Yang

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Koziel, S., Ciaurri, D.E., Leifsson, L. (2011). Surrogate-Based Methods. In: Koziel, S., Yang, XS. (eds) Computational Optimization, Methods and Algorithms. Studies in Computational Intelligence, vol 356. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-20859-1_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-20859-1_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-20858-4

  • Online ISBN: 978-3-642-20859-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation