Skip to main content

Advertisement

SpringerLink
Log in

A sequential quadratic programming algorithm for equality-constrained optimization without derivatives

  • Original Paper
  • Published:
Optimization Letters Aims and scope Submit manuscript

Abstract

In this paper, we present a new model-based trust-region derivative-free optimization algorithm which can handle nonlinear equality constraints by applying a sequential quadratic programming (SQP) approach. The SQP methodology is one of the best known and most efficient frameworks to solve equality-constrained optimization problems in gradient-based optimization [see e.g. Lalee et al. (SIAM J Optim 8:682–706, 1998), Schittkowski (Optim Lett 5:283–296, 2011), Schittkowski and Yuan (Wiley encyclopedia of operations research and management science, Wiley, New York, 2010)]. Our derivative-free optimization (DFO) algorithm constructs local polynomial interpolation-based models of the objective and constraint functions and computes steps by solving QP sub-problems inside a region using the standard trust-region methodology. As it is crucial for such model-based methods to maintain a good geometry of the set of interpolation points, our algorithm exploits a self-correcting property of the interpolation set geometry. To deal with the trust-region constraint which is intrinsic to the approach of self-correcting geometry, the method of Byrd and Omojokun is applied. Moreover, we will show how the implementation of such a method can be enhanced to outperform well-known DFO packages on smooth equality-constrained optimization problems. Numerical experiments are carried out on a set of test problems from the CUTEst library and on a simulation-based engineering design problem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Adams, B.M., Bauman, L.E., Bohnhoff, W.J., Dalbey, K.R., Ebeida, M.S., Eddy, J.P., Eldred, M.S., Hough, P.D., Hu, K.T., Jakeman, J.D., Swiler, L.P., Vigil, D.M.: Dakota, a multilevel parallel object-oriented framework for design optimization, parameter estimation, uncertainty quantification, and sensitivity analysis: Version 5.4 user’s manual. Sandia Technical Report SAND2010-2183 (2009, updated 2013) (2013)

  2. Ashley, H., Landahl, M.: Aerodynamics of Wings and Bodies. Dover Books on Aeronautical Engineering Series. Addison-Wesley Publishing Company, New York (1965)

  3. Audet, C., Dennis, J.J.E.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17, 188–217 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  4. Audet, C., Dennis, J.J.E.: A progressive barrier for derivative-free nonlinear programming. SIAM J. Optim. 20, 445–472 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bueno, L., Friedlander, A., Martinez, J., Sobral, F.: Inexact restoration method for derivative-free optimization with smooth constraints. SIAM J. Optim. 23(2), 1189–1213 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  6. Byrd, R.H.: Robust trust region methods for constrained optimization. In: Third SIAM Conference on Optimization, Houston, Texas (1987)

  7. Colson, B.: Trust-region algorithms for derivative-free optimization and nonlinear bilevel programming. Ph.D Thesis, The University of Namur, Rempart de la Vierge 8, 5000 Namur, Belgium (2003)

  8. Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust-Region Methods. MPS-SIAM Series on Optimization. Society for Industrial and Applied Mathematics, Philadelphia (2000)

    Book  MATH  Google Scholar 

  9. Conn, A.R., Scheinberg, K., Toint, P.L.: On the convergence of derivative-free methods for unconstrained optimization. In: Iserles, A., Buhmann, M. (eds.) Approximation Theory and Optimization: Tributes to M. J. D. Powell, pp. 83–108. Cambridge, England (1997)

    Google Scholar 

  10. Conn, A.R., Scheinberg, K., Toint, P.L.: A derivative free optimization algorithm in practice. In: Proceedings of the 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, September 2–4 (1998)

  11. Conn, A.R., Scheinberg, K., Vicente, L.N.: Geometry of interpolation sets in derivative free optimization. Mathe. Program. Ser. A, B 111, 141–172 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  12. Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. MPS-SIAM Series on Optimization. SIAM, Philadelphia (2008)

    Google Scholar 

  13. Conn, A.R., Scheinberg, K., Vicente, L.N.: Global convergence of general derivative-free trust-region algorithms to first- and second-order critical points. SIAM J. Optim. 20(1), 387–415 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  14. Currie, J., Wilson, D.: OPTI: Lowering the barrier between open source optimizers and the industrial MATLAB user. In: Sahinidis, N., Pinto, J. (eds.) Foundations of Computer-Aided Process Operations. Savannah (2012)

  15. Diniz-Ehrhardt, M.A., Martinez, J.M., Pedroso, L.G.: Derivative-free methods for nonlinear programming with general lower-level constraints. Comput. Appl. Math. 30, 19–52 (2011)

    MathSciNet  MATH  Google Scholar 

  16. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–203 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  17. Driessen, L.: Simulation-based optimization for product and process design. Ph.D Thesis, Tilburg University, Tilburg, Netherlands (2006)

  18. Fasano, G., Morales, J.L., Nocedal, J.: On the geometry phase in model-based algorithms for derivative-free optimization. Optim. Methods Softw. 24(1), 145–154 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  19. Gould, N.I.M., Orban, D., Toint, P.L.: CUTEst: a constrained and unconstrained testing environment with safe threads. Comput. Optimiz. Appl. (to appear)

  20. Lalee, M., Nocedal, J., Plantenga, T.: On the implementation of an algorithm for large-scale equality constrained optimization. SIAM J. Optim. 8, 682–706 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  21. Liuzzi, G., Lucidi, S., Sciandrone, M.: Sequential penalty derivative-free methods for nonlinear constrained optimization. SIAM J. Optim. 20(5), 2614–2635 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  22. Neeb, D., Schwanekamp, T., Gülhan, A.: Preliminary aerodynamic shape optimization of the spaceliner by means of engineering methods. In: 17th AIAA International Space Planes and Hypersonic Systems and Technologies Conference, American Institute of Aeronautics and Astronautics (2011)

  23. Omojokun, E.O.: Trust region algorithms for optimization with nonlinear equality and inequality constraints. Ph.D. Thesis, University of Colorado, Boulder (1989)

  24. Perez, R., Jansen, P., Martins, J.: pyopt: A python-based object-oriented framework for nonlinear constrained optimization. Struct. Multidiscip. Optim. 45(1), 101–118 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  25. Powell, M.: A direct search optimization method that models the objective and constraint functions by linear interpolation. In: Gomez, S., Hennart, J.P. (eds.) Advances in Optimization and Numerical Analysis, Mathematics and Its Applications, vol. 275, pp. 51–67. Springer, Netherlands (1994)

    Chapter  Google Scholar 

  26. Powell, M.: Direct search algorithms for optimization calculations. Acta Numerica 7, 287–336 (1998)

    Article  Google Scholar 

  27. Powell, M.J.D.: Developments of NEWUOA for minimization without derivatives. IMA J. Numer. Anal. 28(4), 649–664 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  28. Rios, L.M., Sahinidis, N.V.: Derivative-free optimization: a review of algorithms and comparison of software implementations. J. Glob. Optim. 56(3), 1247–1293 (2013)

  29. Scheinberg, K., Toint, P.L.: Self-correcting geometry in model-based algorithms for derivative-free unconstrained optimization. SIAM J. Optim. 20(6), 3512–3532 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  30. Schittkowski, K.: A robust implementation of a sequential quadratic programming algorithm with successive error restoration. Optim. Lett. 5(2), 283–296 (2011). doi:10.1007/s11590-010-0207-9

    Article  MathSciNet  MATH  Google Scholar 

  31. Schittkowski, K., Yuan, Y.X.: Sequential quadratic programming methods. In: Cochran, J.J., Cox, L.A., Keskinocak, P., Kharoufeh, J.P., Smith, J.C. (eds.) Wiley Encyclopedia of Operations Research and Management Science. Wiley, New York (2010)

    Google Scholar 

  32. Streit, T., Martin, S., Eggers, T.: Approximate heat transfer methods for hypersonic flow in comparison with results provided by numerical Navier–Stokes solution. DLR-Forschungsbericht, 94–36 (1994)

  33. Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. 106(1), 25–57 (2006)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. German Aerospace Center (DLR), Linder Höhe, 51147, Cologne, Germany

    Anke Tröltzsch

Authors
  1. Anke Tröltzsch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anke Tröltzsch.

Appendix A: Test problem statistics and detailed results

Appendix A: Test problem statistics and detailed results

Table 1 depicts the equality-constrained test problems taken from the CUTEst testing environment. It shows the name of the problem and gives specific details on the number of variables, the number of constraints and the type of the objective and constraint functions. As used in the CUTEst library, ’N’ stands for ’no objective function’ where a feasibility problem has to be solved, ’L’ stands for ’linear function’, ’Q’ means ’quadratic function’, ’S’ stands for ’sum of squares’ and ’O’ stand for any ’other’ type of function which could for instance involve exponential terms. If problems involve constraints of different type, the most non-linear type is given in the table. Furthermore, the table shows the number of function evaluations needed by each solver to attain four significant figures of the reference objective function value \(f^*\) from IPOPT, and to satisfy \(\Vert h\Vert _\infty \le 10^{-4}\).

Table 1 Results on equality-constrained CUTEst test problems
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tröltzsch, A. A sequential quadratic programming algorithm for equality-constrained optimization without derivatives. Optim Lett 10, 383–399 (2016). https://doi.org/10.1007/s11590-014-0830-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11590-014-0830-y

Keywords

Search

Navigation