Skip to main content
SpringerLink
Log in

Delaunay-based derivative-free optimization via global surrogates. Part III: nonconvex constraints

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

This paper introduces a Delaunay-based derivative-free optimization algorithm, dubbed \(\varDelta \)-DOGS(\(\varOmega \)), for problems with both (a) a nonconvex, computationally expensive objective function f(x), and (b) nonlinear, computationally expensive constraint functions \(c_\ell (x)\) which, taken together, define a nonconvex, possibly even disconnected feasible domain \(\varOmega \), which is assumed to lie within a known rectangular search domain \(\varOmega _s\), everywhere within which the f(x) and \(c_\ell (x)\) may be evaluated. Approximations of both the objective function f(x) as well as the feasible domain \(\varOmega \) are developed and refined as the iterations proceed. The approach is practically limited to the problems with less than about ten adjustable parameters. The work is an extension of our original Delaunay-based optimization algorithm (see JOGO DOI: 10.1007/s10898-015-0384-2), and inherits many of the constructions and strengths of that algorithm, including: (1) a surrogate function p(x) interpolating all existing function evaluations and summarizing their trends, (2) a synthetic, piecewise-quadratic uncertainty function e(x) built on the framework of a Delaunay triangulation amongst existing datapoints, (3) a tunable balance between global exploration (large K) and local refinement (small K), (4) provable global convergence for a sufficiently large K, under the assumption that the objective and constraint functions are twice differentiable with bounded Hessians, (5) an Adaptive-K variant of the algorithm that efficiently tunes K automatically based on a target value of the objective function, and (6) remarkably fast global convergence on a variety of benchmark problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. This approach generalizes the SQP method, where a quadratic function is used to locally model the objective function, and linear function is used to locally model the constraints.

  2. If \(A,B,C > 0\) and \(A^2 \le A\,B+C\) then \(A\le B + \sqrt{C} \le 2\,\max \{B, \sqrt{C}\}.\)

References

  1. Abramson, M.A., Audet, C., Dennis, J.E., Le Digabel, S.: OrthoMADS: a deterministic MADS instance with orthogonal directions (2009)

  2. Achterberg, T.: Scip: solving constraint integer programs. Math. Program. Comput. 1(1), 1–41 (2009)

    MathSciNet  MATH  Google Scholar 

  3. Alimo, S., Beyhaghi, P., Meneghello, G., Bewley, T.: Delaunay-based optimization in cfd leveraging multivariate adaptive polyharmonic splines (maps). The American Institute of Aeronautics and Astronautics, SciTech Meeting (2017) (2017)

  4. Alimohammadi, S., He, D.: Multi-stage algorithm for uncertainty analysis of solar power forecasting. In: Power and Energy Society General Meeting (PESGM), 2016, pp. 1–5. IEEE (2016)

  5. Audet, C., Dennis, J.E.: A pattern search filter method for nonlinear programming without derivatives. SIAM J. Optim. 14(4), 980–1010 (2004)

    MathSciNet  MATH  Google Scholar 

  6. Audet, C., Dennis, J.E.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1), 188–217 (2006)

    MathSciNet  MATH  Google Scholar 

  7. Audet, C., Dennis, J.E.: A progressive barrier for derivative-free nonlinear programming. SIAM J. Optim. 20(1), 445–472 (2009)

    MathSciNet  MATH  Google Scholar 

  8. Belitz, P., Bewley, T.: New horizons in sphere-packing theory, part ii: lattice-based derivative-free optimization via global surrogates. J. Glob. Optim. pp. 1–31 (2013)

  9. Belotti, P., Lee, J., Liberti, L., Margot, F., Wächter, A.: Branching and bounds tighteningtechniques for non-convex minlp. Optim. Methods Softw. 24(4–5), 597–634 (2009)

    MathSciNet  MATH  Google Scholar 

  10. Bertsekas, D.P.: On penalty and multiplier methods for constrained minimization. SIAM J. Control Optim. 14(2), 216–235 (1976)

    MathSciNet  MATH  Google Scholar 

  11. Beyhaghi, P., Bewley, T.R.: Delaunay-based derivative-free optimization via global surrogates, part ii: convex constraints. J. Global Optim. 66(3), 383–415 (2016)

    MathSciNet  MATH  Google Scholar 

  12. Beyhaghi, P., Cavaglieri, D., Bewley, T.: Delaunay-based derivative-free optimization via global surrogates, part i: linear constraints. J. Glob. Optim. pp. 1–52 (2015)

  13. Boissonnat, J.-D., Devillers, O., Hornus, S.: Incremental construction of the delaunay triangulation and the delaunay graph in medium dimension. In: Proceedings of the Twenty-Fifth Annual Symposium on Computational Geometry, pp. 208–216. ACM (2009)

  14. Booker, A.J., Dennis Jr., J.E., Frank, P.D., Serafini, D.B., Torczon, V., Trosset, M.W.: A rigorous framework for optimization of expensive functions by surrogates. Struct. Optim. 17(1), 1–13 (1999)

    Google Scholar 

  15. Boyd, S., Vandenberghe, L.: Convex Optimization, vol. 25. Cambridge University Press, Cambridge (2010)

    MATH  Google Scholar 

  16. Jones, D.R.: A taxonomy of global optimization methods based on response surfaces. J. Global Optim. 21(4), 39 (2001)

    MathSciNet  Google Scholar 

  17. Fletcher, R., Leyffer, S., Toint, P.L.: On the global convergence of a filter-SQP algorithm. SIAM J. Optim. 13(1), 44–59 (2002)

    MathSciNet  MATH  Google Scholar 

  18. Forsgren, A., Gill, P.E.: Primal-dual interior methods for nonconvex nonlinear programming. SIAM J. Optim. 8(4), 1132–1152 (1998)

    MathSciNet  MATH  Google Scholar 

  19. Forsgren, A., Gill, P.E., Wright, M.H.: Interior Methods for Nonlinear Optimization, vol. 44. SIAM, Philadelphia (2002)

    MATH  Google Scholar 

  20. Gill, P.E., Murray, W., Saunders, M.A.: Snopt: an sqp algorithm for large-scale constrained optimization. SIAM Rev. 47(1), 99–131 (2005)

    MathSciNet  MATH  Google Scholar 

  21. Gill, P.E., Murray, W., Saunders, M.A., Wright, M.H.: A Schur-complement method for sparse quadratic programming. Technical report, DTIC Document (1987)

  22. Gill, P.E., Saunders, M.A., Wong, E.: On the performance of SQP methods for nonlinear optimization. In: Modeling and Optimization: Theory and Applications, pp. 95–123. Springer, Berlin (2015)

  23. Gramacy, R.B., Gray, G.A., Le Digabel, S., Lee, H.K.H., Ranjan, P., Wells, G., Wild, S.M.: Modeling an augmented lagrangian for blackbox constrained optimization. Technometrics 58(1), 1–11 (2016)

    MathSciNet  Google Scholar 

  24. Hoffman, K.L.: A method for globally minimizing concave functions over convex sets. Math. Program. 20(1), 22–32 (1981)

    MathSciNet  MATH  Google Scholar 

  25. Jones, D.R., Perttunen, C.D., Stuckman, B.E.: Lipschitzian optimization without the lipschitz constant. J. Optim. Theory Appl. 79(1), 157–181 (1993)

    MathSciNet  MATH  Google Scholar 

  26. Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Global Optim. 13(4), 455–492 (1998)

    MathSciNet  MATH  Google Scholar 

  27. Kort, B.W., Bertsekas, D.P.: A new penalty function method for constrained minimization. In: Proceedings of the 1972 IEEE Conference on Decision and Control, 1972 and 11th Symposium on Adaptive Processes, pp. 162–166. IEEE, New York (1972)

  28. Kort, B.W., Bertsekas, D.P.: Combined primal-dual and penalty methods for convex programming. SIAM J. Control Optim. 14(2), 268–294 (1976)

    MathSciNet  MATH  Google Scholar 

  29. Krige, D.G.: A statistical approach to some basic mine valuation problems on the Witwatersrand. J. South. Afr. Inst. Min. Metall. 52, 119–139 (1952)

    Google Scholar 

  30. Lawrence, C.T., Zhou, J.L., Tits, A.L.: User’s guide for CFSQP version 2.0: AC code for solving (large scale) constrained nonlinear (minimax) optimization problems, generating iterates satisfying all inequality constraints (1994)

  31. Le Digabel, S.: Algorithm 909: nomad: nonlinear optimization with the mads algorithm. ACM Trans. Math. Softw. (TOMS) 37(4), 44 (2011)

    MathSciNet  MATH  Google Scholar 

  32. Lewis, R.M., Torczon, V.: Pattern search algorithms for bound constrained minimization. SIAM J. Optim. 9(4), 1082–1099 (1999)

    MathSciNet  MATH  Google Scholar 

  33. Lewis, R.M., Torczon, V.: Pattern search methods for linearly constrained minimization. SIAM J. Optim. 10(3), 917–941 (2000)

    MathSciNet  MATH  Google Scholar 

  34. Lewis, R.M., Torczon, V.: A globally convergent augmented lagrangian pattern search algorithm for optimization with general constraints and simple bounds. SIAM J. Optim. 12(4), 1075–1089 (2002)

    MathSciNet  MATH  Google Scholar 

  35. Long, C.C., Marsden, A.L., Bazilevs, Y.: Shape optimization of pulsatile ventricular assist devices using FSI to minimize thrombotic risk. Comput. Mech. 54(4), 921–932 (2014)

    MathSciNet  MATH  Google Scholar 

  36. Madani, R., Ashraphijuo, M., Lavaei, J.: Promises of conic relaxation for contingency-constrained optimal power flow problem. IEEE Trans. Power Syst. 31(2), 1297–1307 (2016)

    Google Scholar 

  37. Marsden, A.L., Feinstein, J.A., Taylor, C.A.: A computational framework for derivative-free optimization of cardiovascular geometries. Comput. Methods Appl. Mech. Eng. 197(21), 1890–1905 (2008)

    MathSciNet  MATH  Google Scholar 

  38. Marsden, A.L., Wang, M., Dennis Jr., J.E., Moin, P.: Optimal aeroacoustic shape design using the surrogate management framework. Optim. Eng. 5(2), 235–262 (2004)

    MathSciNet  MATH  Google Scholar 

  39. McKay, M.D., Beckman, R.J., Conover, W.J.: A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 42(1), 55–61 (2000)

    MATH  Google Scholar 

  40. Meer, J.T.M.T., Duijne, H.V., Nieuwenhuis, R., Rijnaarts, H.H.M.: Prevention and reduction of pollution of groundwater at contaminated megasites: integrated management strategy, and its application on megasite cases. In: Quevauviller, P. (ed.) Groundwater Science and Policy: An International Overview, pp. 405–420 (2008)

  41. Misener, R., Floudas, C.A.: Antigone: algorithms for continuous/integer global optimization of nonlinear equations. J. Glob. Optim. 59(2–3), 503–526 (2014)

    MathSciNet  MATH  Google Scholar 

  42. Moghadam, M.E., Migliavacca, F., Vignon-Clementel, I.E., Hsia, T.-Y., Marsden, A.L.: Optimization of shunt placement for the norwood surgery using multi-domain modeling. J. Biomech. Eng. 134(5), 051002 (2012)

    Google Scholar 

  43. Molga, Marcin., Smutnicki, Czesław.: Test functions for optimization needs. Test functions for optimization needs, 101, (2005)

  44. Parr, J.M., Keane, A.J., Forrester, A.I.J., Holden, C.M.E.: Infill sampling criteria for surrogate-based optimization with constraint handling. Eng. Optim. 44(10), 1147–1166 (2012)

    MATH  Google Scholar 

  45. Pee, E.Y., Royset, J.O.: On solving large-scale finite minimax problems using exponential smoothing. J. Optim. Theory Appl. 148(2), 390–421 (2011)

    MathSciNet  MATH  Google Scholar 

  46. Picheny, V., Gramacy, R.B., Wild, S., Le Digabel, S.: Bayesian optimization under mixed constraints with a slack-variable augmented lagrangian. In: Advances in Neural Information Processing Systems, pp. 1435–1443 (2016)

  47. Polak, E., Royset, J.O., Womersley, R.S.: Algorithms with adaptive smoothing for finite minimax problems. J. Optim. Theory Appl. 119(3), 459–484 (2003)

    MathSciNet  MATH  Google Scholar 

  48. Pussoli, B.F., Da Silva, L.W., Barbosa, J.R., Kaviany, M.: Optimization of peripheral finned-tube evaporators using entropy generation minimization. Int. J. Heat Mass Transf. 55(25–26), 7838–7846 (2012)

    Google Scholar 

  49. Ramachandra, A.B., Sankaran, S., Humphrey, J.D., Marsden, A.L.: Computational simulation of the adaptive capacity of vein grafts in response to increased pressure. J. Biomech. Eng. 137(3), 031009 (2015)

    Google Scholar 

  50. Rasmussen, C.E.: Gaussian processes for machine learning. Int. J. Neural Syst. 14(2), 69–106 (2006)

    Google Scholar 

  51. Sacks, J., Welch, W.J., Mitchell, T.J., Wynn, H.P.: Design and analysis of computer experiments. Stat. Sci. 4(4), 409–423 (1989)

    MathSciNet  MATH  Google Scholar 

  52. Schonlau, M., Welch, W.J., Jones, D.R.: A data-analytic approach to Bayesian global optimization. In: Department of Statistics and Actuarial Science and The Institute for Improvement in Quality and Productivity, 1997 ASA Conference (1997)

  53. Simionescu, P.A.: Computer-Aided Graphing and Simulation Tools for AutoCAD Users, vol. 32. CRC Press, Boca Raton (2014)

    Google Scholar 

  54. Snoek, J., Larochelle, H., Adams, R.P.: Practical Bayesian optimization of machine learning algorithms. In: Advances in Neural Information Processing Systems, pp. 2951–2959 (2012)

  55. Tawarmalani, M., Sahinidis, N.V.: A polyhedral branch-and-cut approach to global optimization. Math. Program. 103, 225–249 (2005)

    MathSciNet  MATH  Google Scholar 

  56. Torczon, V.: On the convergence of pattern search algorithms. SIAM J. Optim. 7(1), 1–25 (1997)

    MathSciNet  MATH  Google Scholar 

  57. Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. 106(1), 25–57 (2006)

    MathSciNet  MATH  Google Scholar 

  58. Wahba, G.: Spline Models for Observational Data, vol. 59. SIAM, Philadelphia (1990)

    MATH  Google Scholar 

  59. Watson, D.F.: Computing the n-dimensional delaunay tessellation with application to voronoi polytopes. Comput. J. 24(2), 167–172 (1981)

    MathSciNet  Google Scholar 

  60. Wild, S.M., Regis, R.G., Shoemaker, C.A.: Orbit: optimization by radial basis function interpolation in trust-regions. SIAM J. Sci. Comput. 30(6), 3197–3219 (2008)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge Dr. Fred Y. Hadaegh and Dr. Firouz M. Naderi for their support, Professors Alison Marsden for her assistance in developing an efficient SMF code, Robert Gramacy and Sebastien Le Digabel for sharing their Lockwood test problem code, and Dr. Stefan Wild for his constructive feedback. The authors gratefully acknowledge funding from AFOSR FA 9550-12-1-0046, from the Cymer Center for Control Systems & Dynamics, from the Leidos corporation in support of this work. Also, the research was supported by the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration.

Author information

Authors and Affiliations

  1. Flow Control and Coordinated Robotics Labs, UC San Diego, San Diego, USA

    Ryan Alimo, Pooriya Beyhaghi & Thomas R. Bewley

  2. Jet Propulsion Laboratory, California Institute of Technology, Pasadena, USA

    Ryan Alimo

Authors
  1. Ryan Alimo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pooriya Beyhaghi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas R. Bewley
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ryan Alimo.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alimo, R., Beyhaghi, P. & Bewley, T.R. Delaunay-based derivative-free optimization via global surrogates. Part III: nonconvex constraints. J Glob Optim 77, 743–776 (2020). https://doi.org/10.1007/s10898-019-00854-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-019-00854-2

Keywords

Search

Navigation