Skip to main content
SpringerLink
Log in

An augmented Lagrangian filter method

  • Original Article
  • Published:
Mathematical Methods of Operations Research Aims and scope Submit manuscript

Abstract

We introduce a filter mechanism to enforce convergence for augmented Lagrangian methods for nonlinear programming. In contrast to traditional augmented Lagrangian methods, our approach does not require the use of forcing sequences that drive the first-order error to zero. Instead, we employ a filter to drive the optimality measures to zero. Our algorithm is flexible in the sense that it allows for equality-constrained quadratic programming steps to accelerate local convergence. We also include a feasibility restoration phase that allows fast detection of infeasible problems. We provide a convergence proof that shows that our algorithm converges to first-order stationary points. We provide preliminary numerical results that demonstrate the effectiveness of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Abhishek K, Leyffer S, Linderoth JT (2010) FilMINT: an outer-approximation-based solver for nonlinear mixed integer programs. INFORMS J Comput 22:555–567

    MathSciNet  MATH  Google Scholar 

  • Altay-Salih A, Pinar MC, Leyffer S (2003) Constrained nonlinear programming for volatility estimation with GARCH models. SIAM Rev 45(3):485–503

    MathSciNet  MATH  Google Scholar 

  • Arrieta-Camacho JJ, Biegler LT, Subramanian D (2007) NMPC-based real-time coordination of multiple aircraft. In: Allgower RF, Biegler LT (eds) Assessment and future directions of nonlinear model predictive control. Springer, Heidelberg, pp 629–639

    MATH  Google Scholar 

  • Bautista G, Anjos MF, Vannelli A (2007) Formulation of oligopolistic competition in AC power networks: an NLP approach. IEEE Trans Power Syst 22(1):105–115

    Google Scholar 

  • Belotti P, Kirches C, Leyffer S, Linderoth J, Luedtke J, Mahajan A (2013) Mixed-integer nonlinear optimization. Acta Numer 22(5):1–131

    MathSciNet  MATH  Google Scholar 

  • Benson H, Shanno D (2007) An exact primal-dual penalty method approach to warmstarting interior-point methods for linear programming. Comput Optim Appl 38:371–399. https://doi.org/10.1007/s10589-007-9048-6

    Article  MathSciNet  MATH  Google Scholar 

  • Benson HY, Shanno DF (2008) Interior-point methods for nonconvex nonlinear programming: regularization and warmstarts. Comput Optim Appl 40(2):143–189

    MathSciNet  MATH  Google Scholar 

  • Benson HY, Shanno DF, Vanderbei RJ (2002) Interior-point methods for nonconvex nonlinear programming: filter-methods and merit functions. Comput Optim Appl 23(2):257–272

    MathSciNet  MATH  Google Scholar 

  • Bertsekas DP (1982) Constrained optimization and lagrange multiplier methods. Athena Scientific, New York

    MATH  Google Scholar 

  • Betts JT (2001) Practical methods for optimal control using nonlinear programming. Advances in design and control. SIAM, Philadelphia

    MATH  Google Scholar 

  • Biegler LT, Grossmann IE (2004a) Challenges and research issues for product and process design optimization. In: Floudas CA, Agrawal R (eds) Proceedings of foundations of computer aided process design. CACHE Corporation, Austin

    Google Scholar 

  • Biegler LT, Grossmann IE (2004b) Part I: retrospective on optimization. Comput Chem Eng 28(8):1169–1192

    Google Scholar 

  • Birgin EG, Martinez JM (2008) Improving ultimate convergence of an augmented Lagrangian method. Optim Methods Softw 23(2):177–195

    MathSciNet  MATH  Google Scholar 

  • Birgin EG, Martínez JM (2012) Augmented lagrangian method with nonmonotone penalty parameters for constrained optimization. Comput Optim Appl 51(3):941–965

    MathSciNet  MATH  Google Scholar 

  • Birgin EG, Martínez JM (2014) Practical augmented Lagrangian methods for constrained optimization, vol 10. SIAM, Philadelphia

    MATH  Google Scholar 

  • Bonami P, Biegler LT, Conn AR, Cornuejols G, Grossmann IE, Laird CD, Lee J, Lodi A, Margot F, Sawaya N, Wähter A (2005) An algorithmic framework for convex mixed integer nonlinear programs. Research report RC 23771, IBM T. J. Watson Research Center, Yorktown, NY

  • Bonami P, Lee J, Leyffer S, Waechter A (2011) More branch-and-bound experiments in convex nonlinear integer programming. Technical report ANL/MCS-P1949-0911, Argonne National Laboratory, Mathematics and Computer Science Division, Lemont, IL

  • Bonnans JF, André J (2009) Optimal structure of gas transmission trunklines. Technical report 6791, INRIA Saclay, 91893 ORSAY Cedex, France

  • Bonnard B, Faubourg L, Launay G, Trélat E (2003) Optimal control with state constraints and the space shuttle re-entry problem. J Dyn Control Syst 9(2):155–199

    MathSciNet  MATH  Google Scholar 

  • Borghetti A, Frangioni A, Lacalandra F, Nucci CA (2003) Lagrangian heuristics based on disaggregated bundle methods for hydrothermal unit commitment. IEEE Trans Power Syst 18:1–10

    Google Scholar 

  • Byrd RH, Hribar ME, Nocedal J (1999) An interior point algorithm for large scale nonlinear programming. SIAM J Optim 9(4):877–900

    MathSciNet  MATH  Google Scholar 

  • Byrd RH, Gould NIM, Nocedal J, Waltz RA (2004) An algorithm for nonlinear optimization using linear programming and equality constrained subproblems. Math Progr Ser B 100(1):27–48

    MathSciNet  MATH  Google Scholar 

  • Byrd RH, Nocedal J, Waltz RA (2006) KNITRO: an integrated package for nonlinear optimization. In: di Pillo G, Roma M (eds) Large-scale nonlinear optimization. Springer, New York, pp 35–59

    MATH  Google Scholar 

  • Calamai PH, Moré JJ (1987) Projected gradient methods for linearly constrained problems. Math Progr 39:93–116

    MathSciNet  MATH  Google Scholar 

  • Castro J, Gonzalez JA (2004) A nonlinear optimization package for long-term hydrothermal coordination. Eur J Oper Res 154(1):641–658

    MATH  Google Scholar 

  • Chin CM, Fletcher R (2003) On the global convergence of an SLP-filter algorithm that takes EQP steps. Math Progr 96(1):161–177

    MathSciNet  MATH  Google Scholar 

  • Coleman TF, Li Y, Verman A (1999) Reconstructing the unknown volatility function. J Comput Finance 2(3):77–102

    Google Scholar 

  • Conn AR, Gould NIM, Toint PL (1992) LANCELOT: a Fortran package for large-scale nonlinear optimization (release A). Springer, Heidelberg

    MATH  Google Scholar 

  • Conn AR, Gould NIM, Toint PL (2000) Trust-region methods. MPS-SIAM series on optimization. SIAM, Philadelphia

    Google Scholar 

  • Cornuejols G, Tütüncü R (2007) Optimization methods in finance. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  • Curtis FE, Jiang H, Robinson DP (2015) An adaptive augmented Lagrangian method for large-scale constrained optimization. Math Progr 152(1–2):201–245

    MathSciNet  MATH  Google Scholar 

  • Dolan ED, Moré J (2002) Benchmarking optimization software with performance profiles. Math Progr 91(2):201–213

    MathSciNet  MATH  Google Scholar 

  • Donde V, Lopez V, Lesieutre B, Pinar A, Yang C, Meza J (2005) Identification of severe multiple contingencies in electric power networks. In: Proceedings 37th North American power symposium, LBNL-57994

  • Klaus E, Steinbach Marc C (2005) Nonlinear optimization in gas networks. In: Bock HG, Kostina E, Phu HX, Ranacher R (eds) Modeling, simulation and optimization of complex processes. Springer, Berlin, pp 139–148

    MATH  Google Scholar 

  • Fletcher R, Leyffer S (1994) Solving mixed integer nonlinear programs by outer approximation. Math Progr 66:327–349

    MathSciNet  MATH  Google Scholar 

  • Fletcher R, Leyffer S (1998) User manual for filterSQP. Numerical analysis report NA/181, University of Dundee

  • Fletcher R, Leyffer S (2002) Nonlinear programming without a penalty function. Math Progr 91:239–270

    MathSciNet  MATH  Google Scholar 

  • Fletcher R, Leyffer S (2003) Filter-type algorithms for solving systems of algebraic equations and inequalities. In: di Pillo G, Murli A (eds) High performance algorithms and software for nonlinear optimization. Kluwer, Dordrecht, pp 259–278

    Google Scholar 

  • Fletcher R, Sainz de la Maza E (1989) Nonlinear programming and nonsmooth optimization by successive linear programming. Math Progr 43:235–256

    MathSciNet  MATH  Google Scholar 

  • Fletcher R, Leyffer S, Toint PL (2002) On the global convergence of a filter-SQP algorithm. SIAM J Optim 13(1):44–59

    MathSciNet  MATH  Google Scholar 

  • Forsgren A, Gill PE, Wright MH (2002) Interior methods for nonlinear optimization. SIAM Rev 4(4):525–597

    MathSciNet  MATH  Google Scholar 

  • Friedlander MP (2002) A globally convergent linearly constrained Lagrangian method for nonlinear optimization. Ph.D. thesis, Stanford University

  • Friedlander MP, Leyffer S (2008) Global and finite termination of a two-phase augmented Lagrangian filter method for general quadratic programs. SIAM J Sci Comput 30(4):1706–1729

    MathSciNet  MATH  Google Scholar 

  • Ghaoui LE, Oks M, Oustry F (2003) Worst-case value-at-risk and robust portfolio optimization: a conic programming approach. Oper Res 51(4):509–679

    MathSciNet  MATH  Google Scholar 

  • Gill PE, Murray W, Saunders MA (1997) User’s guide for SQOPT 5.3: a Fortran package for large-scale linear and quadratic programming. Technical report NA 97-4, Department of Mathematics, University of California, San Diego

  • Gill PE, Murray W, Saunders MA (2002) SNOPT: an SQP algorithm for large-scale constrained optimization. SIAM J Optim 12(4):979–1006

    MathSciNet  MATH  Google Scholar 

  • Gill PE, Murray W, Saunders MA (2005) SNOPT: an SQP algorithm for large-scale constrained optimization. SIAM Rev 47(1):99–131

    MathSciNet  MATH  Google Scholar 

  • Gondzio J, Grothey A (2002) Reoptimization with the primal-dual interior point method. SIAM J Optim 13(3):842–864

    MathSciNet  MATH  Google Scholar 

  • Gould NIM, Orban D, Toint PL (2015) Cutest: a constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput Optim Appl 60(3):545–557

    MathSciNet  MATH  Google Scholar 

  • Goux J-P, Leyffer S (2002) Solving large MINLPs on computational grids. Optim Eng 3:327–346

    MathSciNet  MATH  Google Scholar 

  • Huangfu Q, Hall JAJ (2013) Novel update techniques for the revised simplex method. Technical report ERGO-13-001, The School of Mathematics, University of Edinburgh, Edinburgh, UK

  • Kawayir Y, Laird C, Wächter A (2009) Introduction to Ipopt: a tutorial for downloading, installing, and using Ipopt. Manual. IBM T.J. Watson Research Center, Yorktown Heights

    Google Scholar 

  • Konno H, Yamazaki H (1991) Mean-absolute deviation portfolio optimization model and its application to Tokyo stock market. Manag Sci 37:519–531

    Google Scholar 

  • Leyffer S (2001) Integrating SQP and branch-and-bound for mixed integer nonlinear programming. Comput Optim Appl 18:295–309

    MathSciNet  MATH  Google Scholar 

  • Leyffer S, Munson TS (2007) A globally convergent filter method for MPECs. Preprint ANL/MCS-P1457-0907, Argonne National Laboratory, Mathematics and Computer Science Division

  • Leyffer S, Lopez-Calva G, Nocedal J (2006) Interior methods for mathematical programs with complementarity constraints. SIAM J Optim 17(1):52–77

    MathSciNet  MATH  Google Scholar 

  • Lubin M, Hall JAJ, Petra CG, Anitescu M (2013) Parallel distributed-memory simplex for large-scale stochastic LP problems. Comput Optim Appl 55(3):571–596

    MathSciNet  MATH  Google Scholar 

  • Martin A, Möller M, Moritz S (2006) Mixed integer models for the stationary case of gas network optimization. Math Progr 105:563–582

    MathSciNet  MATH  Google Scholar 

  • Momoh JA, Koessler RJ, Bond MS, Stott B, Sun D, Papalexopoulos A, Ristanovic P (1997) Challenges to optimal power flow. IEEE Trans Power Syst 12:444–455

    Google Scholar 

  • Moré JJ, Toraldo G (1991) On the solution of quadratic programming problems with bound constraints. SIAM J Optim 1(1):93–113

    MathSciNet  MATH  Google Scholar 

  • Murtagh BA, Saunders MA (1993) MINOS 5.4 user’s guide. Report SOL 83-20R, Department of Operations Research, Stanford University

  • Murtagh BA, Saunders MA (1982) A projected Lagrangian algorithm and its implementation for sparse nonlinear constraints. In: Buckley AG, Goffin J-L (eds) Algorithms for constrained minimization of smooth nonlinear functions, vol 16. Mathematical programming studies, vol 16. Springer, Berlin, pp 84–117

    Google Scholar 

  • Penfield P, Spence R, Duinker S (1970) Tellegen’s theorem and electrical networks. MIT Press, Cambridge

    Google Scholar 

  • Powell MJD (1978) Algorithms for nonlinear constraints that use Lagrangian functions. Math Progr 14(1):224–248

    MathSciNet  MATH  Google Scholar 

  • Rabinowitz G, Mehrez A, Oron G (1988) A nonlinear optimization model of water allocation for hydroelectric energy production and irrigation. Manag Sci 34(8):973–990

    MATH  Google Scholar 

  • Raghunathan A, Biegler LT (2005) An interior point method for mathematical programs with complementarity constraints (MPCCs). SIAM J Optim 15(3):720–750

    MathSciNet  MATH  Google Scholar 

  • Sheble GB, Fahd GN (1994) Unit commitment literature synopsis. IEEE Trans Power Syst 9:128–135

    Google Scholar 

  • Shen C, Leyffer S, Fletcher R (2012) A nonmonotone filter method for nonlinear optimization. Comput Optim Appl 52(3):583–607

    MathSciNet  MATH  Google Scholar 

  • Smith E, Hall JAJ (2012) A high performance primal revised simplex solver for row-linked block angular linear programming problems. Technical report ERGO-12-003, The School of Mathematics, University of Edinburgh, Edinburgh, UK

  • Vanderbei RJ, Shanno DF (1999) An interior point algorithm for nonconvex nonlinear programming. COAP 13:231–252

    MathSciNet  MATH  Google Scholar 

  • Wächter A, Biegler LT (2005a) Line search filter methods for nonlinear programming: local convergence. SIAM J Optim 16(1):32–48

    MathSciNet  MATH  Google Scholar 

  • Wächter A, Biegler LT (2005b) Line search filter methods for nonlinear programming: motivation and global convergence. SIAM J Optim 16(1):1–31

    MathSciNet  MATH  Google Scholar 

  • Wächter A, Biegler LT (2006) On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math Progr 106(1):25–57

    MathSciNet  MATH  Google Scholar 

  • Womersley RS, Lau K (1996) Portfolio optimisation problems. In: May RL, Easton AK (eds) Computational techniques and applications: CTAC95. World Scientific Publishing Co., Singapore, pp 795–802

    Google Scholar 

  • Zhu C, Byrd RH, Lu P, Nocedal J (1997) Algorithm 778: L-BFGS-B—Fortran subroutines for large-scale bound-constrained optimization. ACM Trans Math Softw (TOMS) 23(4):550–560

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We are grateful to two anonymous referees and an associate editor for their careful and detailed refereeing work that greatly helped us improve the final manuscript. This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, under Contract DE-AC02-06CH11357. This work was also supported by the U.S. Department of Energy through Grant DE-FG02-05ER25694.

Author information

Authors and Affiliations

  1. Mathematics and Computer Science Division, Argonne National Laboratory, Lemont, IL, 60439, USA

    Sven Leyffer

  2. Fraunhofer ITWM, Fraunhofer-Platz 1, 67663, Kaiserslautern, Germany

    Charlie Vanaret

Authors
  1. Sven Leyffer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Charlie Vanaret
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sven Leyffer.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

See Tables 2, 3 and 4.

Table 2 Number of evaluations of nonlinear solvers on a subset of linear CUTEst problems
Full size table
Table 3 Number of evaluations of nonlinear solvers on a subset of quadratic CUTEst problems
Full size table
Table 4 Number of evaluations of nonlinear solvers on a subset of nonlinear CUTEst problems
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Leyffer, S., Vanaret, C. An augmented Lagrangian filter method. Math Meth Oper Res 92, 343–376 (2020). https://doi.org/10.1007/s00186-020-00713-x

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00186-020-00713-x

Keywords

Mathematics Subject Classification

Search

Navigation