Skip to main content
SpringerLink
Log in

Integrated experimental design and nonlinear optimization to handle computationally expensive models under resource constraints

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

In many real-world applications of optimization, the underlying descriptive system model is defined by computationally expensive functions: simulation modules, numerical models and other “black box” model components are typical examples. In such cases, the model development and optimization team often has to rely on optimization carried out under severe resource constraints. To address this important issue, recently a Regularly Spaced Sampling (RSS) module has been added to the Lipschitz Global Optimizer (LGO) solver suite. RSS generates non-collapsing space filling designs, and produces corresponding solution estimates: this information is passed along to LGO for refinement within the given resource (function evaluation and/or runtime) limitations. Obviously, the quality of the solution obtained will essentially depend both on model instance difficulty and on the admissible computational effort. In spite of this general caveat, our results based on solving a selection of non-trivial global optimization test problems suggest that even a moderate amount of well-placed sampling effort enhanced by limited optimization can lead at least to reasonable or even to high quality results. Our numerical tests also indicate that LGO’s overall efficiency is often increased by using RSS as a presolver, both in resource-constrained and in completed LGO runs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ali M.M., Khompatraporn C., Zabinsky Z.B.: A numerical evaluation of several stochastic algorithms on selected continuous global optimization test problems. J. Glob. Optim. 31, 635–672 (2005)

    Article  Google Scholar 

  • Audze P., Eglais V.: New approach for planning out of experiments. Probl. Dyn. Strengths 35, 104–107 (1977)

    Google Scholar 

  • Bailey R.A.: Design of Comparative Experiments. Cambridge University Press, Cambridge (2008)

    Book  Google Scholar 

  • Barthelemy J.F.M., Haftka R.T.: Approximation concepts for optimum structural design—a review. Struct. Optim. 5(3), 129–144 (1993)

    Article  Google Scholar 

  • Biles, W.E., Kleijnen, J.P.C., van Beers, W.C.M., van Nieuwenhuyse, I.: Kriging metamodeling in constrained simulation optimization: an explorative study. In: Henderson, S.G., Biller, B., Hsieh, M.H., Shortle, J., Tew, J.D., Barton, R.R. (eds.) Proceedings of the 2007 Winter Simulation Conference, pp. 355–362 (2007)

  • Booker A.J., Dennis J.E., Frank P.D., Serafini D.B., Torczon V.J., Trosset M.W.: A rigorous framework for optimization of expensive functions by surrogates. Struct. Optim. 17, 1–13 (1999)

    Article  Google Scholar 

  • Bursztyn D., Steinberg D.: Comparison of designs for computer experiments. J. Stat. Plan. Inference 136, 1103–1119 (2006)

    Article  Google Scholar 

  • Caflisch R.E.: Monte Carlo and quasi-Monte Carlo methods. Acta Numerica 7, 1–49 (1998)

    Article  Google Scholar 

  • Çağlayan, M.O., Pintér, J.D.: Development and calibration of a currency trading strategy using global optimization. J. Glob. Optim. (to appear) (2011). doi:10.1007/s10898-012-9879-2

  • Castillo I., Kampas F.J., Pintér J.D.: Solving circle packing problems by global optimization: Numerical results and industrial applications. Eur. J. Oper. Res. 191, 786–802 (2008)

    Article  Google Scholar 

  • den Hertog D., Stehouwer H.P.: Optimizing color picture tubes by high-cost nonlinear programming. Eur. J. Oper. Res. 140(2), 197–211 (2002)

    Article  Google Scholar 

  • Devroye L.: Non-uniform Random Variate Generation. Springer, New York (1986)

    Google Scholar 

  • Dolan E.D., Moré J.J.: Benchmarking optimization software with performance profiles. Math. Program. Ser. A 91, 201–213 (2002)

    Article  Google Scholar 

  • Doye J.P.K.: Physical perspectives on the global optimization of atomic clusters. In: Pintér, J.D. (eds) Global Optimization: Scientific and Engineering Case Studies, pp. 103–139. Springer Science + Business Media, New York (2006)

    Chapter  Google Scholar 

  • Edgar T.F., Himmelblau D.M., Lasdon L.S.: Optimization of Chemical Processes, 2nd edn. McGraw-Hill, New York (2001)

    Google Scholar 

  • Fang K.T., Li R., Sudjianto A.: Design and Modeling for Computer Experiments. CRC Press, New York (2006)

    Google Scholar 

  • Fiacco A.V., McCormick G.P.: Nonlinear Programming: Sequential Unconstrained Minimization Techniques. Wiley, New York (1968)

    Google Scholar 

  • Fletcher R.: Penalty functions. In: Bachem, A., Grötschel, M., Korte, B. (eds) Mathematical Programming: The State of the Art, pp. 87–114. Springer, Berlin (1983)

    Chapter  Google Scholar 

  • Floudas C.A., Gounaris C.E.: A review of recent advances in global optimization. J. Glob. Optim. 45, 3–38 (2009)

    Article  Google Scholar 

  • Füsti-Molnár L., Merz K.M.: An efficient and accurate molecular alignment and docking technique using ab initio quality scoring. J. Chem. Phys. 129, 025102 (2008)

    Article  Google Scholar 

  • Georgieva A., Jordanov I.: A hybrid meta-heuristic for global optimisation using low-discrepancy sequences of points. Comput. Oper. Res. 37, 456–469 (2010)

    Article  Google Scholar 

  • Goossens, P., McPhee, J., Schmitke, C., Pintér, J.D., Stahl, H.: Driving innovation: how mathematical modeling and optimization increase efficiency and productivity in vehicle design. In: Technical Memorandum, Maplesoft, Waterloo (2007)

  • Grosso A., Jamali A.R.M.J.U., Locatelli M.: Finding maximin Latin hypercube designs by iterated local search heuristics. Eur. J. Oper. Res. 197(2), 541–547 (2009)

    Article  Google Scholar 

  • Gutmann H.M.: A radial basis function method for global optimization. J. Glob. Optim. 19, 201–227 (2001)

    Article  Google Scholar 

  • Holmström K.: An adaptive radial basis algorithm (ARBF) for expensive black-box global optimization. J. Glob. Optim. 41, 447–464 (2008)

    Article  Google Scholar 

  • Horst, R., Pardalos, P.M. (eds): Handbook of Global Optimization, vol. 1. Kluwer, Dordrecht (1995)

    Google Scholar 

  • Horváth, A., Horváth, Z.: Optimal shape design of diesel intake ports with evolutionary algorithm. In: Proceedings of ENUMATH 2003. Springer, New York (2004)

  • Horváth, Z., Morauszki, T., Tóth, K.: Automated CAD-based CFD-optimization and applications in diesel engine design. Proceedings of the 3rd European Automotive CFD Conference. Frankfurt, Germany (2007a)

  • Horváth, Z., Morauszki, T., Tóth, K.: CAD-based optimization and applications in automotive engineering. In: Proceedings of the 6th EUROSIM Congress on Modelling and Simulation. Ljubljana, Slovenia (2007b)

  • Husslage, B.M.G., Rennen, G., van Dam, E.R., den Hertog, D.: Space-filling Latin hypercube designs for computer experiments. CentER Discussion Paper 2006-18, Center for Economic Research. Tilburg University, Netherlands (2006)

  • Iman R.L., Helton J.C, Campbell J.E.: An approach to sensitivity analysis of computer models, Part 1. Introduction, input variable selection and preliminary variable assessment. J. Qual. Technol. 13(3), 174–183 (1981)

    Google Scholar 

  • Isenor G., Pintér J.D., Cada M.: A global optimization approach to laser design. Optim. Eng. 4, 177–196 (2003)

    Article  Google Scholar 

  • Jakobsson S., Patriksson M., Rudholm J., Wojciechowski A.: A method for simulation based optimization using radial basis functions. Optim. Eng. 11, 501–532 (2010)

    Article  Google Scholar 

  • Johnson M.E., Moore L.M., Ylvisaker D.: Minimax and maximin distance design. J. Stat. Plan. Inference 26, 131–148 (1990)

    Article  Google Scholar 

  • Johnson, R.T., Montgomery, D.C., Jones, B., Fowler, J.W.: Comparing designs for computer simulation experiments. In: Mason, S., Hill, R., Mönch, L., Rose, O. (eds.) In: Proceedings of the 40th Conference on Winter Simulation, pp. 463–470. Miami, FL, USA, December 07–10 (2008)

  • Jones D.R.: A taxonomy of global optimization methods based on response surfaces. J. Glob. Optim. 21, 345–383 (2001)

    Article  Google Scholar 

  • Jones D.R., Schonlau M., Welch W.J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13, 455–492 (1998)

    Article  Google Scholar 

  • Jourdan A., Franco J.: Optimal Latin hypercube designs for the Kullback–Leibler criterion. Adv. Stat. Anal. 94, 341–351 (2010)

    Article  Google Scholar 

  • Kampas F.J., Pintér J.D.: Configuration analysis and design by using optimization tools in Mathematica. Math. J. 10(1), 128–154 (2006)

    Google Scholar 

  • Khompatraporn Ch., Pintér J.D., Zabinsky Z.B.: Comparative assessment of algorithms and software for global optimization. J. Glob. Optim. 31, 613–633 (2005)

    Article  Google Scholar 

  • Kleijnen J.P.C.: Design and Analysis of Simulation Experiments. Springer, New York (2009)

    Google Scholar 

  • Kleiner, Y., Rajani, B.: Prioritising individual water mains for renewal. Research Report NRCC-50450, Institute for Research in Construction, National Research Council Canada, Ottawa (2008)

  • Kucherenko S., Sytsko Y.: Application of deterministic low-discrepancy sequences in global optimization. Comput. Optim. Appl. 30, 297–318 (2005)

    Article  Google Scholar 

  • Lahey Computer Systems: Lahey/Fujitsu FORTRAN 95 (Release 5.70f) (2004)

  • Liberti L.: Writing global optimization software. In: Liberti, Li., Maculan, N. (eds) Global Optimization: From Theory to Implementation, pp. 211–262. Springer Science + Business Media, New York (2006)

    Chapter  Google Scholar 

  • Liefvendahl M., Stocki R.: A study on algorithms for optimization of Latin hypercubes. J. Stat. Plan. Inference 136, 3231–3247 (2006)

    Article  Google Scholar 

  • Locatelli M., Schoen F.: Fast global optimization of difficult Lennard-Jones clusters. Comput. Optim. Appl. 21, 55–70 (2002)

    Article  Google Scholar 

  • Locatelli M., Schoen F.: Efficient algorithms for large scale global optimization: Lennard-Jones clusters. Comput. Optim. Appl. 26, 173–190 (2003)

    Article  Google Scholar 

  • Mason T.L., Emelle C., van Berkel J., Bagirov A.M., Kampas F.J., Pintér J.D.: Integrated production system optimization using the Lipschitz global optimizer and the discrete gradient method. J. Ind. Manag. Optim. 3(2), 257–277 (2007)

    Article  Google Scholar 

  • McKay M.D., Beckman R.J., Conover W.J.: A comparison of three methods for selecting input values in the analysis of output from a computer code. Technometrics 21(2), 239–245 (1979)

    Google Scholar 

  • Michalewicz Z.: Genetic Algorithms + Data Structures = Evolution Programs. Springer, Heidelberg (1996)

    Book  Google Scholar 

  • Montgomery D.C.: Design and Analysis of Experiments, 2nd edn. Wiley, New York (1984)

    Google Scholar 

  • Morris M., Mitchell T.: Exploratory designs for computational experiments. J. Stat. Plan. Inference 43, 381–402 (1995)

    Article  Google Scholar 

  • Myers R.H.: Response surface methodology—current status and future directions. J. Qual. Technol. 31, 30–44 (1999)

    Google Scholar 

  • Neumaier A.: Molecular modeling of proteins and mathematical prediction of protein structure. SIAM Rev. 39, 407–460 (1997)

    Article  Google Scholar 

  • Neumaier A.: Complete search in continuous global optimization and constraint satisfaction. In: Iserles, A. (eds) Acta Numerica, pp. 271–369. Cambridge University Press, Cambridge (2004)

    Google Scholar 

  • Neumaier A., Shcherbina O., Huyer W., Vinkó T.: A comparison of complete global optimization solvers. Math. Program. Ser. B 103, 335–356 (2005)

    Article  Google Scholar 

  • Niederreiter H.: Random Number Generation and Quasi-Monte Carlo Methods. SIAM, Philadelphia (1992)

    Book  Google Scholar 

  • G.: Quasi Monte Carlo methods in option pricing. Math. Educ. Res. 8(3-4), 52– (1999)

    Google Scholar 

  • Pantoleontos G., Basinas P., Skodras G., Grammelis P., Pintér J.D., Topis S., Sakellaropoulos G.P.: A global optimization study on the devolatilisation kinetics of coal, biomass and waste fuels. Fuel Process. Technol. 90, 762–769 (2009)

    Article  Google Scholar 

  • Pardalos, P.M., Romeijn, H.E. (eds): Handbook of Global Optimization, vol. 2. Kluwer, Dordrecht (2002)

    Google Scholar 

  • Pinchon D., Hoggan P.E.: Gaussian approximation of exponential type orbitals based on B functions. Int. J. Quantum Chem. 109, 135–144 (2009)

    Article  Google Scholar 

  • Pintér J.D.: Global Optimization in Action. Kluwer, Dordrecht (1996)

    Book  Google Scholar 

  • Pintér J.D.: LGO—a program system for continuous and Lipschitz optimization. In: Bomze, I.M., Csendes, T., Horst, R., Pardalos, P.M. (eds) Developments in Global Optimization, pp. 183–197. Kluwer, Dordrecht (1997)

    Chapter  Google Scholar 

  • Pintér J.D.: Globally optimized spherical point arrangements: model variants and illustrative results. Ann. Oper. Res. 104, 213–230 (2001)

    Article  Google Scholar 

  • Pintér J.D.: Global optimization: software, test problems, and applications. In: Pardalos, P.M., Romeijn, H.E. (eds) Handbook of Global Optimization, vol. 2, pp. 515–569. Kluwer, Dordrecht (2002)

    Chapter  Google Scholar 

  • Pintér J.D.: Nonlinear optimization in modeling environments: software implementations for compilers, spreadsheets, modeling languages, and integrated computing systems. In: Jeyakumar, V., Rubinov, A.M. (eds) Continuous Optimization: Current Trends and Modern Applications, pp. 147–173. Springer, New York (2005)

    Chapter  Google Scholar 

  • Pintér J.D.: Nonlinear optimization with GAMS/LGO. J. Glob. Optim. 38, 79–101 (2007)

    Article  Google Scholar 

  • Pintér J.D.: Software development for global optimization. In: Pardalos, P.M., Coleman, T.F. (eds) Global Optimization: Methods and Applications. Fields Institute Communications, vol. 55, pp. 183–204. American Mathematical Society, Providence (2009)

    Google Scholar 

  • Pintér, J.D.: LGO—A Model Development and Solver System for Global-Local Nonlinear Optimization. User’s Guide (Current version). Distributed by Pintér Consulting Services, Inc., Canada (2011)

  • Pintér J.D.: Calibrating artificial neural networks by global optimization. Expert Syst. Appl. 39, 25–32 (2012)

    Article  Google Scholar 

  • Pintér J.D., Kampas F.J.: Nonlinear optimization in mathematica with mathoptimizer professional. Math. Educ. Res. 10(2), 1–18 (2005)

    Google Scholar 

  • Pintér J.D., Kampas F.J.: Mathoptimizer professional: key features and illustrative applications. In: Liberti, L., Maculan, N. (eds) Global Optimization: From Theory to Implementation, pp. 263–279. Springer Science + Business Media, New York (2006)

    Chapter  Google Scholar 

  • Pintér, J.D., Kampas, F.J.: Benchmarking nonlinear optimization software in technical computing environments—global optimization. In: Mathematica with Mathoptimizer Professional. TOP, Published online August 17 (2011). doi:10.1007/s11750-011-0209-5

  • Pintér J.D., Linder D., Chin P.: Global optimization toolbox for maple: an introduction with illustrative applications. Optim. Methods Softw. 21, 565–582 (2006)

    Article  Google Scholar 

  • Press W.H., Flannery B.P., Teukolsky S.A., Vetterling W.T.: Numerical Recipes in FORTRAN: The Art of Scientific Computing, 2nd edn. Cambridge University Press, Cambridge (1992)

    Google Scholar 

  • Pronzato, L., Müller, W.G.: Design of computer experiments: space filling and beyond. In: Statistics and Computing, pp. 1–21. Published online (2011). doi:10.1007/s11222-011-9242-3

  • Regis R.G., Shoemaker C.A.: Improved strategies for radial basis function methods for global optimization. J. Glob. Optim. 37, 113–135 (2007)

    Article  Google Scholar 

  • Rikards R., Auzins J.: Response surface method for solution of structural identification problems. Inverse Probl. Eng. 12(1), 59–70 (2004)

    Article  Google Scholar 

  • Rios, L.M., Sahinidis, N.V.: Derivative-free optimization: A review of algorithms and comparison of software implementations. Technical Report, Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh, PA (2009). (Submitted for publication)

  • Sacks J.W., Welch J., Mitchell T.J., Wynn H.P.: Design and analysis of computer experiments. Stat. Sci. 4, 409–423 (1989)

    Article  Google Scholar 

  • Saff E.B., Kuijlaars A.B.J.: Distributing many points on a sphere. Math. Intell. 19(1), 5–11 (1997)

    Article  Google Scholar 

  • Santner Th.J., Williams B.J., Notz W.I.: The Design and Analysis of Computer Experiments. Springer, New York (2003)

    Book  Google Scholar 

  • Shewry M.C., Wynn H.P.: Maximum entropy sampling. J. Appl. Stat. 14, 165–170 (1987)

    Article  Google Scholar 

  • Simpson T.W., Booker A.J., Ghosh D., Giunta A.A., Koch P.N., Yang R.J.: Approximation methods in multidisciplinary analysis and optimization: a panel discussion. Struct. Multidiscip. Optim. 27(5), 302–313 (2004)

    Article  Google Scholar 

  • Space Filling Designs: Maintained by researchers from the Center for Economic Research, Tilburg University, Netherlands, http://www.spacefillingdesigns.nl/ (2011)

  • Stortelder W.J.H., de Swart J.J.B., Pintér J.D.: Finding elliptic Fekete points sets: two numerical solution approaches. J. Comput. Appl. Math. 130, 205–216 (2001)

    Article  Google Scholar 

  • Tervo J., Kolmonen P., Lyyra-Laitinen T., Pintér J.D., Lahtinen T.: An optimization-based approach to the multiple static delivery technique in radiation therapy. Ann. Oper. Res. 119, 205–227 (2003)

    Article  Google Scholar 

  • Trefethen N.L.: The hundred-dollar, hundred-digit challenge problems. SIAM News 1, 3 (2002)

    Google Scholar 

  • van Dam E.R., Husslage B.G.M., den Hertog D.: One-dimensional nested maximin designs. J. Glob. Optim. 46, 287–306 (2010)

    Article  Google Scholar 

  • van Dam E.R., Husslage B.G.M., den Hertog D., Melissen J.B.M.: Maximin Latin hypercube designs in two dimensions. Oper. Res. 57(3), 595–608 (2007)

    Google Scholar 

  • van Dam E.R., Rennen G., Husslage B.G.M.: Bounds for maximin Latin hypercube designs. Oper. Res. 55(1), 158–169 (2009)

    Google Scholar 

  • Villemonteix J., Vazquez E., Sidorkiewicz M., Walter E.: Global optimization of expensive-to-evaluate functions: an empirical comparison of two sampling criteria. J. Glob. Optim. 43, 373–389 (2009)

    Article  Google Scholar 

  • Wales D.J., Scheraga H.A.: Global optimization of clusters, crystals, and biomolecules. Science 285, 1368–1372 (1999)

    Article  Google Scholar 

  • Weisstein, E.W.: Hundred-Dollar, Hundred-Digit Challenge Problems. From MathWorld—A Wolfram Web Resource, http://mathworld.wolfram.com/Hundred-DollarHundred-DigitChallengeProblems.html(2011)

  • Wolfram Research: Mathematica. (version 8.0.1) Wolfram Research, Inc., Champaign (2011)

  • Wu Y., Kumar L., Kumar A.: TRIOPT: a triangulation-based partitioning algorithm for global optimization. J. Comput. Appl. Math. 177, 35– (2005)

    Article  Google Scholar 

  • Ye K.Q., Li W., Sudjianto A.: Algorithmic construction of optimal symmetric Latin hypercube designs. J. Stat. Plan. Inference 90, 145–159 (2000)

    Article  Google Scholar 

  • Zukerman M., Mammadov M., Tan L., Ouyevsi I., Andrew L.: To be fair or efficient or a bit of both. Comput. Oper. Res. 35, 3787–3806 (2008)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Pintér Consulting Services, Inc., Nova Scotia, Canada

    János D. Pintér

  2. Özyeğin University, Istanbul, Turkey

    János D. Pintér

  3. Széchenyi István University, Győr, Hungary

    Zoltán Horváth

Authors
  1. János D. Pintér
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zoltán Horváth
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to János D. Pintér.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pintér, J.D., Horváth, Z. Integrated experimental design and nonlinear optimization to handle computationally expensive models under resource constraints. J Glob Optim 57, 191–215 (2013). https://doi.org/10.1007/s10898-012-9882-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-012-9882-7

Keywords

Search

Navigation