Skip to main content
SpringerLink
Log in

Mesh adaptive direct search with second directional derivative-based Hessian update

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

The subject of this paper is inequality constrained black-box optimization with mesh adaptive direct search (MADS). The MADS search step can include additional strategies for accelerating the convergence and improving the accuracy of the solution. The strategy proposed in this paper involves building a quadratic model of the function and linear models of the constraints. The quadratic model is built by means of a second directional derivative-based Hessian update. The linear terms are obtained by linear regression. The resulting quadratic programming (QP) problem is solved with a dedicated solver and the original functions are evaluated at the QP solution. The proposed search strategy is computationally less expensive than the quadratically constrained QP strategy in the state of the art MADS implementation (NOMAD). The proposed MADS variant (QPMADS) and NOMAD are compared on four sets of test problems. QPMADS outperforms NOMAD on all four of them for all but the smallest computational budgets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Abramson, M.A., Audet, C.: Convergence of mesh adaptive direct search to second-order stationary points. SIAM J. Optim. 17(2), 606–619 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  2. Abramson, M.A., Audet, C., Dennis Jr, J.E., Le Digabel, S.: OrthoMADS: a deterministic MADS instance with orthogonal directions. SIAM J. Optim. 20(2), 948–966 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  3. Abramson, M.A., Frimannslund, L., Steihaug, T.: A subclass of generating set search with convergence to second-order stationary points. Optim. Methods Softw. 29(5), 900–918 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  4. Andersen, M. S., Dahl, J., Vandenberghe, L.: CVXOPT, Release 1.1.6. Available at http://cvxopt.org/userguide/index.html (2014)

  5. Audet, C., Dennis Jr, J.E.: Analysis of generalized pattern searches. SIAM J. Optim. 13(3), 889–903 (2002)

    Article  MathSciNet  Google Scholar 

  6. Audet, C., Dennis Jr, J.E.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1), 188–217 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  7. Audet, C., Dennis Jr, J.E.: A progressive barrier for derivative-free nonlinear programming. SIAM J. Optim. 20(1), 445–472 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  8. Audet, C., Ianni, A., Le Digabel, S., Tribes, C.: Reducing the number of function evaluations in mesh adaptive direct search algorithms. SIAM J. Optim. 24(2), 621–642 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  9. Bűrmen, Á., Puhan, J., Tuma, T.: Grid restrained Nelder-Mead algorithm. Comput. Optim. Appl. 34(3), 359–375 (2006)

    Article  MathSciNet  Google Scholar 

  10. Clarke, F.: Optimization and Nonsmooth Analysis. SIAM, Philadelphia (1983)

    MATH  Google Scholar 

  11. Conn, A.R., Scheinberg, K., Vincente, L.N.: Introduction to Derivative-Free Optimization. SIAM, Philadelphia (2009)

    Book  MATH  Google Scholar 

  12. Conn, A.R., Le Digabel, S.: Use of quadratic models with mesh-adaptive direct search for constrained black box optimization. Optim. Methods Softw. 28(1), 139–158 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  13. Coope, I.D., Price, C.J.: Frame-based methods for unconstrained optimization. J. Optim. Theory Appl. 107(2), 261–274 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  14. Custódio, A.L., Vincente, L.N.: Using sampling and simplex derivatives in patters search methods. SIAM J. Optim. 18(2), 537–555 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  15. Custódio, A.L., Rocha, H., Vincente, L.N.: Incorporating minimum Frobenius norm models in direct search. Comput. Optim. Appl. 46(2), 265–278 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  16. Davidon, W.C.: Variable metric method for minimization. SIAM J. Optim. 1(1), 1–17 (1991)

    Article  MATH  MathSciNet  Google Scholar 

  17. Frimannslund, L., Steihaug, T.: A generating set search method using curvature information. Comput. Optim. Appl. 38(1), 105–121 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  18. Le Digabel, S.: Algorithm 909: NOMAD: nonlinear optimization with the MADS algorithm. ACM Trans. Math. Softw. 37(4), 44:1–44:15 (2011)

    Article  Google Scholar 

  19. Leventhal, D., Lewis, A.S.: Randomized Hessian estimation and directional search. Optimization 60(3), 329–345 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  20. Moré, J., Wild, S.: Benchmarking derivative-free optimization algorithms. SIAM J. Optim. 20(1), 172–191 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  21. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (2006)

    MATH  Google Scholar 

  22. Oliphant, T.E.: Python for scientific computing. Comput. Sci. Eng. 9(3), 10–20 (2007)

    Article  Google Scholar 

  23. Rockafellar, R.: Generalized directional derivatives and subgradients of nonconvex functions. Can. J. Math. 32(2), 257–280 (1980)

    Article  MATH  MathSciNet  Google Scholar 

  24. Stewart, G.W.: The efficient generation of random orthogonal matrices with an application to condition estimators. SIAM. J. Numer. Anal. 17(3), 403–409 (1980)

    Article  MATH  MathSciNet  Google Scholar 

  25. Van Dyke, B., Asaki, T.J.: Using QR decomposition to obtain a new instance of mesh adaptive direct search with uniformly distributed polling directions. J. Optim. Theory Appl. 159(3), 805–821 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  26. van Rossum, G., et al.: The Python Language Reference. Python Software Foundation, available at http://docs.python.org/ (2014)

  27. Vincente, L., Custódio, A.: Analysis of direct searches for discontinuous functions. Math. Program. 133(1–2), 299–325 (2012)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

The research was co-funded by the Ministry of Education, Science, and Sport (Ministrstvo za Šolstvo, Znanost in Šport) of the Republic of Slovenia through the Programme P2-0246 Algorithms and optimization methods in telecommunications. The authors would like to thank two anonymous reviewers and the associate editor for their comments which helped to improve the paper.

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering, University of Ljubljana, Tržaška cesta 25, 1000, Ljubljana, Slovenia

    Árpád Bűrmen, Jernej Olenšek & Tadej Tuma

Authors
  1. Árpád Bűrmen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jernej Olenšek
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tadej Tuma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Árpád Bűrmen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bűrmen, Á., Olenšek, J. & Tuma, T. Mesh adaptive direct search with second directional derivative-based Hessian update. Comput Optim Appl 62, 693–715 (2015). https://doi.org/10.1007/s10589-015-9753-5

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-015-9753-5

Keywords

Mathematics Subject Classification

Search

Navigation