Skip to main content
SpringerLink
Log in

Quasi-Newton acceleration for equality-constrained minimization

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

Optimality (or KKT) systems arise as primal-dual stationarity conditions for constrained optimization problems. Under suitable constraint qualifications, local minimizers satisfy KKT equations but, unfortunately, many other stationary points (including, perhaps, maximizers) may solve these nonlinear systems too. For this reason, nonlinear-programming solvers make strong use of the minimization structure and the naive use of nonlinear-system solvers in optimization may lead to spurious solutions. Nevertheless, in the basin of attraction of a minimizer, nonlinear-system solvers may be quite efficient. In this paper quasi-Newton methods for solving nonlinear systems are used as accelerators of nonlinear-programming (augmented Lagrangian) algorithms, with equality constraints. A periodically-restarted memoryless symmetric rank-one (SR1) correction method is introduced for that purpose. Convergence results are given and numerical experiments that confirm that the acceleration is effective are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L.: Augmented Lagrangian methods with general lower-level constraints. SIAM J. Optim. (2007, to appear)

  2. Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L.: On Augmented Lagrangian methods with general lower-level constraints. Technical Report MCDO-050303, Department of Applied Mathematics, UNICAMP, Brazil (2005), available in Optimization On Line and in www.ime.usp.br/~egbirgin

  3. Andreani, R., Martínez, J.M., Schuverdt, M.L.: On the relation between the constant positive linear dependence condition and quasinormality constraint qualification. J. Optim. Theory Appl. 125, 473–485 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bertsekas, D.P.: Nonlinear Programming, 2nd edn. Athena Scientific, Belmont (1999)

    MATH  Google Scholar 

  5. Birgin, E.G., Castillo, R., Martínez, J.M.: Numerical comparison of augmented Lagrangian algorithms for nonconvex problems. Comput. Optim. Appl. 31, 31–56 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  6. Birgin, E.G., Krejić, N., Martínez, J.M.: Globally convergent inexact quasi-Newton methods for solving nonlinear systems. Numer. Algorithms 32, 249–260 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  7. Birgin, E.G., Martínez, J.M.: Large-scale active-set box-constrained optimization method with spectral projected gradients. Comput. Optim. Appl. 23, 101–125 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  8. Birgin, E.G., Martínez, J.M., Raydan, M.: Nonmonotone spectral projected gradient methods on convex sets. SIAM J. Optim. 10, 1196–1211 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  9. Birgin, E.G., Martínez, J.M., Raydan, M.: Algorithm 813: SPG-Software for convex-constrained optimization. ACM Trans. Math. Softw. 27, 340–349 (2001)

    Article  MATH  Google Scholar 

  10. Birgin, E.G., Martínez, J.M., Raydan, M.: Inexact spectral projected gradient methods on convex sets. IMA J. Numer. Anal. 23, 539–559 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  11. Conn, A.R., Gould, N.I.M., Toint, P.L.: A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds. SIAM J. Numer. Anal. 28, 545–572 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  12. Conn, A.R., Gould, N.I.M., Toint, P.L.: Convergence of quasi-Newton matrices generated by the symmetric rank one update. Math. Program. A 50, 177–195 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  13. Conn, A.R., Gouldm, N.I.M., Toint, P.L.: LANCELOT: A Fortran Package for Large Scale Nonlinear Optimization. Springer, Berlin (1992)

    MATH  Google Scholar 

  14. Curtis, A.R., Powell, J.D., Reid, J.K.: On the estimation of sparse Jacobian matrices. J. Inst. Math. Appl. 13, 117–119 (1974)

    MATH  Google Scholar 

  15. Dennis, J.E. Jr., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Classics in Applied Mathematics, vol. 16. SIAM, Philadelphia (1996)

    MATH  Google Scholar 

  16. Deuflhard, P.: Newton Techniques for Highly Nonlinear Problems—Theory and Algorithms. Academic Press, San Diego (1996)

    Google Scholar 

  17. Deuflhard, P., Freund, R., Walter, A.: Fast secant methods for the iterative solution of large nonsymmetric linear systems. Impact Comput. Sci. Eng. 2, 244–276 (1990)

    Article  MATH  Google Scholar 

  18. Diniz-Ehrhardt, M.A., Gomes-Ruggiero, M.A., Lopes, V.L.R., Martínez, J.M.: Discrete Newton’s method with local variations for solving large-scale nonlinear systems. Optimization 52, 417–440 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  19. Dostál, Z.: Inexact semimonotonic augmented Lagrangians with optimal feasibility convergence for convex bound and equality constrained quadratic programming. SIAM J. Numer. Anal. 43, 96–115 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  20. Dostál, Z., Friedlander, A., Santos, S.A.: Augmented Lagrangian with adaptive precision control for quadratic programming with simple bounds and equality constraints. SIAM J. Optim. 13, 1120–1140 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  21. Dostál, Z., Gomes, F.A.M., Santos, S.A.: Duality based domain decomposition with natural coarse space for variational inequalities. J. Comput. Appl. Math. 126, 397–415 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  22. Fletcher, R.: Practical Methods of Optimization. Academic Press, London (1987)

    MATH  Google Scholar 

  23. Friedlander, A., Gomes-Ruggiero, M.A., Kozakevich, D.N., Martínez, J.M., Santos, S.A.: Solving nonlinear systems of equations by means of quasi-Newton methods with a nonmonotone strategy. In: Optimization Methods and Software, vol. 8, pp. 25–51 (1997)

  24. Gertz, E.M., Gill, P.E.: A primal-dual trust region algorithm for nonlinear optimization. Math. Program. 100, 49–94 (2004)

    MathSciNet  MATH  Google Scholar 

  25. Gill, P.E., Murray, W., Saunders, M.A.: SNOPT: An SQP algorithm for large-scale constrained optimization. SIAM Rev. 47, 99–131 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  26. Goldfarb, D., Toint, P.L.: Optimal estimation of Jacobian and Hessian matrices that arise in finite difference calculations. In: Mathematics of Computation, vol. 43, pp. 69–88 (1984)

  27. Gomes-Ruggiero, M.A., Martínez, J.M., Moretti, A.C.: Comparing algorithms for solving sparse nonlinear systems of equations. SIAM J. Sci. Stat. Comput. 13, 459–483 (1992)

    Article  MATH  Google Scholar 

  28. Gould, N.I.M., Orban, D., Toint, P.L.: GALAHAD: a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization. ACM Trans. Math. Softw. 29, 353–372 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  29. Hager, W.W., Zhang, H.: A new active set algorithm for box constrained optimization. Department of Mathematics, University of Florida, 4 July 2005

  30. Hager, W.W., Zhang, H.: Algorithm 851: CG-DESCENT, a conjugate gradient method with guaranteed descent. ACM Trans. Math. Softw. 32, 113–137 (2006)

    Article  MathSciNet  Google Scholar 

  31. Hestenes, M.R.: Multiplier and gradient methods. J. Optim. Theory Appl. 4, 303–320 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  32. Hock, W., Schittkowski, K.: Test Examples for Nonlinear Programming Codes. Lecture Notes in Economic and Mathematical Systems, vol. 187. Springer, New York (1981)

    MATH  Google Scholar 

  33. Kelley, C.T.: Iterative Methods for Linear and Nonlinear Equations. SIAM, Philadelphia (1995)

    MATH  Google Scholar 

  34. Kelley, C.T., Sachs, E.W.: Local convergence of the Symmetric Rank-One iteration. Comput. Optim. Appl. 9, 43–63 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  35. Khalfan, H., Byrd, R.H., Schnabel, R.B.: A theoretical and experimental study of the symmetric rank one update. SIAM J. Optim. 3, 1–24 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  36. Kozakevich, D.N., Martínez, J.M., Santos, S.A.: Solving nonlinear systems of equations with simple constraints. Comput. Appl. Math. 16, 215–235 (1997)

    MATH  Google Scholar 

  37. Krejić, N., Martínez, J.M., Mello, M.P., Pilotta, E.A.: Validation of an augmented Lagrangian algorithm with a Gauss-Newton Hessian approximation using a set of hard-spheres problems. Comput. Optim. Appl. 16, 247–263 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  38. Li, D.-H., Fukushima, M.: A derivative-free line search and global convergence of Broyden-like method for nonlinear equations. Optim. Methods Softw. 13, 181–201 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  39. Lukšan, L., Vlček, J.: Sparse and partially separable test problems for unconstrained and equality constrained optimization. Technical Report 767, Institute of Computer Science, Academy of Sciences of the Czech Republic (1999)

  40. Martínez, J.M.: Practical quasi-Newton methods for solving nonlinear systems. J. Comput. Appl. Math. 124, 97–122 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  41. Martínez, J.M., Pilotta, E.A.: Inexact restoration methods for nonlinear programming: advances and perspectives. In: Optimization and Control with Applications, pp. 271–292. Springer, New York (2005)

    Chapter  Google Scholar 

  42. Martínez, J.M., Qi, L.: Inexact Newton methods for solving nonsmooth equations. J. Comput. Appl. Math. 60, 127–145 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  43. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (1999)

    MATH  Google Scholar 

  44. Powell, M.J.D.: A method for nonlinear constraints in minimization problems. In: Fletcher, R. (ed.) Optimization, pp. 283–298. Academic Press, New York (1969)

    Google Scholar 

  45. Qi, L., Sun, J.: A nonsmooth version of Newton method. Math. Program. 58, 353–367 (1993)

    Article  MathSciNet  Google Scholar 

  46. Qi, L., Wei, Z.: On the constant positive linear dependence condition and its application to SQP methods. SIAM J. Optim. 10, 963–981 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  47. Rockafellar, R.T.: Augmented Lagrange multiplier functions and duality in nonconvex programming. SIAM J. Control 12, 268–285 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  48. Shamanski, V.E.: A modification of Newton’s method. Ukrain Mat. Z. 19, 133–138 (1967)

    MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, IM-UFRJ, Federal University of Rio de Janeiro, CP 68530, Ilha do Fundão, 21941-590, Rio de Janeiro, RJ, Brazil

    L. Ferreira-Mendonça

  2. Department of Applied Mathematics, IMECC-UNICAMP, University of Campinas, CP 6065, 13081-970, Campinas, SP, Brazil

    V. L. R. Lopes & J. M. Martínez

Authors
  1. L. Ferreira-Mendonça
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. V. L. R. Lopes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. M. Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to V. L. R. Lopes.

Additional information

This work was supported by FAPESP, CNPq, PRONEX-Optimization (CNPq / FAPERJ), FAEPEX, UNICAMP.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ferreira-Mendonça, L., Lopes, V.L.R. & Martínez, J.M. Quasi-Newton acceleration for equality-constrained minimization. Comput Optim Appl 40, 373–388 (2008). https://doi.org/10.1007/s10589-007-9090-4

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-007-9090-4

Keywords

Search

Navigation