Skip to main content
SpringerLink
Log in

Smoothing augmented Lagrangian method for nonsmooth constrained optimization problems

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

In this paper, we propose a smoothing augmented Lagrangian method for finding a stationary point of a nonsmooth and nonconvex optimization problem. We show that any accumulation point of the iteration sequence generated by the algorithm is a stationary point provided that the penalty parameters are bounded. Furthermore, we show that a weak version of the generalized Mangasarian Fromovitz constraint qualification (GMFCQ) at the accumulation point is a sufficient condition for the boundedness of the penalty parameters. Since the weak GMFCQ may be strictly weaker than the GMFCQ, our algorithm is applicable for an optimization problem for which the GMFCQ does not hold. Numerical experiments show that the algorithm is efficient for finding stationary points of general nonsmooth and nonconvex optimization problems, including the bilevel program which will never satisfy the GMFCQ.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. ALGENCAN. http://www.ime.usp.br/~egbirgin/tango/

  2. Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L.: On augmented Lagrangian methods with general lower-level constraints. SIAM J. Optim. 18, 1286–1309 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  3. Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L.: Augmented Lagrangian methods under the constant positive linear dependence constraint qualification. Math. Program. Ser. B 111, 5–32 (2008)

    Article  MATH  Google Scholar 

  4. Andreani, R., Martínez, J.M., Schuverdt, M.L.: On the relation between the constant positive linear dependence condition and quasinormality constraint qualification. J. Optim. Theory Appl. 125, 473–483 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bard, J.F.: Practical Bilevel Optimization: Algorithms and Applications. Kluwer, Boston (1998)

    Book  MATH  Google Scholar 

  6. Bertsekas, D.P.: Constrained Optimization and Lagrangian Multiplier Methods. Academic Press, New York (1982)

    Google Scholar 

  7. Bian, W., Chen, X.: Neural network for nonsmooth, nonconvex constrained minimization via smooth approximation. IEEE Trans. Neural Netw. Learn. Syst. 25, 545–556 (2014)

    Article  Google Scholar 

  8. Birgin, E.G., Castillo, R., Martínez, J.M.: Numerical comparison of augmented Lagrangian algorithms for nonconvex problems. Comput. Optim. Appl. 23, 101–125 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  9. Birgin, E.G., Fernández, D., Martínez, J.M.: The boundedness of penalty parameters in an augmented Lagrangian method with lower level constraints. Optim. Methods Softw. 27, 1001–1024 (2012)

    Article  MathSciNet  Google Scholar 

  10. Boggs, P.T., Kearsley, A.J., Tolle, J.W.: A global convergence analysis of an algorithm for large-scale nonlinear optimization problems. SIAM J. Optim. 9, 833–862 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  11. Boggs, P.T., Tolle, J.W.: Sequential Quadratic Programming, pp. 1–51. Cambridge University Press, Cambridge (1995)

  12. Burke, J.V., Hoheisel, T., Kanzow, C.: Gradient consistency for integral-convolution smoothing functions. Set-Valued Var. Anal. 21, 359–376 (2013)

    Article  MathSciNet  Google Scholar 

  13. Byrd, R.H., Tapia, R.A., Zhang, Y.: An SQP augmented Lagrangian BFGS algorithm for constrained optimization. SIAM J. Optim. 2, 210–241 (1992)

    Article  MathSciNet  Google Scholar 

  14. Chen, B., Chen, X.: A global and local superlinear continuation-smoothing method for \(P_0\) and \(R_0\) NCP or monotone NCP. SIAM J. Optim. 9, 624–645 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  15. Chen, B., Harker, P.T.: A non-interior-point continuation method for linear complementarity problems. SIAM J. Matrix Anal. Appl. 14, 1168–1190 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  16. Chen, C., Mangasarian, O.L.: A class of smoothing functions for nonlinear and mixed complementarity problems. Math. Program. 71, 51–70 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  17. Chen, X.: Smoothing methods for nonsmooth, nonconvex minimization. Math. Program. 134, 71–99 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  18. Chen, X., Womersley, R.S., Ye, J.J.: Minimizing the condition number of a Gram matrix. SIAM J. Optim. 21, 127–148 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  19. Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley-Interscience, New York (1983)

    Google Scholar 

  20. Clarke, F.H., Ledyaev, YuS, Stern, R.J., Wolenski, P.R.: Nonsmooth Analysis and Control Theory. Springer, New York (1998)

    MATH  Google Scholar 

  21. Conn, A.R., Gould, N.I.M., Toint, PhL: A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bound. SIAM J. Numer. Anal. 28, 545–572 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  22. Conn, A.R., Gould, N.I.M., Toint, PhL: Trust Region Methods, MPS/SIAM Series on Optimization. SIAM, Philadelphia (2000)

    Book  Google Scholar 

  23. Curtis, F.E., Overton, M.L.: A sequential quadratic programming algorithm for nonconvex, nonsmooth constrained optimization. SIAM J. Optim. 22, 474–500 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  24. Dempe, S.: Foundations of Bilevel Programming. Kluwer, Dordrecht (2002)

    Google Scholar 

  25. Dempe, S.: Annotated bibliography on bilevel programming and mathematical programs with equilibrium constraints. Optimization 52, 333–359 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  26. Gill, P.E., Murray, W., Saunders, M.A.: SNOPT: an SQP algorithm for large-scale constrained optimization. SIAM J. Optim. 47, 99–131 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  27. Goldfarb, D., Polyak, R., Scheinberg, K., Yuzefovich, I.: A modified barrier-augmented Lagrangian method for constrained minimization. Comput. Optim. Appl. 14, 55–74 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  28. Griva, I., Polyak, R.: Primal-dual nonliner rescaling method with dynamic scaling parameter update. Math. Program. Ser. A 106, 237–259 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  29. Hestenes, M.R.: Multiplier and gradient methods. J. Optim. Theory Appl. 4, 303–320 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  30. Huang, X.X., Yang, X.Q.: Further study on augmented Lagrangian duality theory. J. Glob. Optim. 31, 193–210 (2005)

    Article  MATH  Google Scholar 

  31. Izmailov, A.F., Solodov, M.V., Uskov, E.I.: Global convergence of augmented Lagrangian methods applied to optimization problems with degenerate constraints, including problems with complementarity constraints. SIAM J. Optim. 22, 1579–1606 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  32. Jourani, A.: Constraint qualifications and Lagrange multipliers in nondifferentiable programming problems. J. Optim. Theory Appl. 81, 533–548 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  33. Kanzow, C.: Some noninterior continuation methods for linear complementarity problems. SIAM J. Matrix Anal. Appl. 17, 851–868 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  34. Kort, B.W., Bertsekas, D.P.: A new penalty method for constrained minimization. In: Proceddings of the 1972 IEEE Conference on Decision and Control, New Orleans, pp. 162–166 (1972)

  35. LANCELOT. http://www.cse.scitech.ac.uk/nag/lancelot/lancelot.shtml

  36. Lin, G.H., Xu, M., Ye, J.J.: On solving simple bilevel programs with a nonconvex lower level program. Math. Program. Ser. A 144, 277–305 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  37. Mirrlees, J.: The theory of moral hazard and unobservable behaviour: part I. Rev. Econ. Stud. 66, 3–22 (1999)

    Article  MATH  Google Scholar 

  38. Mitsos, A., Barton, P.: A test set for bilevel programs. Technical Report, Massachusetts Institute of Technology (2006)

  39. Mitsos, A., Lemonidis, P., Barton, P.: Global solution of bilevel programs with a nonconvex inner program. J. Glob. Optim. 42, 475–513 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  40. Nguyen, V.H., Strodiot, J.J.: On the convergence rate of a penalty function method of exponential type. J. Optim. Theory Appl. 27, 495–508 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  41. Powell, M.J.D.: A method for nonlinear constraints in minimization problems. In: Fletcher, R. (ed.) Optimization, pp. 283–298. Academic Press, London and New York (1969)

    Google Scholar 

  42. Qi, L., Wei, Z.: On the constant positive linear dependence condition and its application to SQP methods. SIAM J. Optim. 10, 963–981 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  43. Rockafellar, R.T.: A dual approach to solving nonlinear programming problems by unconstrained optimization. Math. Program. 5, 354–373 (1973)

    Article  MathSciNet  MATH  Google Scholar 

  44. Rockafellar, R.T.: Augmented Lagrange multiplier functions and duality in nonconvex programming. SIAM J. Control 12, 268–285 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  45. Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer, Berlin (1998)

    Book  MATH  Google Scholar 

  46. Shimizu, K., Ishizuka, Y., Bard, J.F.: Nondifferentiable and Two-Level Mathematical Programming. Kluwer, Boston (1997)

    Book  MATH  Google Scholar 

  47. Smale, S.: Algorithms for solving equations. In: Proceedings of the International Congress of Mathematicians, Berkeley, CA, pp. 172–195 (1986)

  48. Tseng, P., Bertsekas, D.P.: On the convergence of the exponential multiplier method for convex programming. Math. Program. 60, 1–19 (1993)

    Article  MathSciNet  Google Scholar 

  49. Vicente, L.N., Calamai, P.H.: Bilevel and multilevel programming: a bibliography review. J. Glob. Optim. 5, 291–306 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  50. Xu, M., Ye, J.J.: A smoothing augmented Lagrangian method for solving simple bilevel programs. Comput. Optim. Appl. 59, 353–377 (2013)

  51. Xu, M., Ye, J.J., Zhang, L.: Smoothing SQP method for solving degenerate nonsmooth constrained optimization problems with applications to bilevel programs. Preprint, arXiv:1403.1636v1 [Math.OC] (2014)

  52. Ye, J.J.: Necessary optimality conditions for multiobjective bilevel programs. Math. Oper. Res. 36, 165–184 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  53. Ye, J.J., Zhu, D.L.: Optimality conditions for bilevel programming problems. Optimization 33, 9–27 (1995)

    Article  MathSciNet  Google Scholar 

  54. Ye, J.J., Zhu, D.L.: A note on optimality conditions for bilevel programming problems. Optimization 39, 361–366 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  55. Ye, J.J., Zhu, D.L.: New necessary optimality conditions for bilevel programs by combining MPEC and the value function approach. SIAM J. Optim. 20, 1885–1905 (2010)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

The authors would like to thank Xijun Liang for helping with the numerical experiments, Lei Guo for giving suggestions in an earlier version and the anonymous referees for their suggestions.

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, Dalian University of Technology, Dalian, 116024, China

    Mengwei Xu

  2. Department of Mathematics and Statistics, University of Victoria, Victoria, BC, V8W 2Y2, Canada

    Jane J. Ye

  3. Dalian University of Technology, Dalian, 116024, China

    Liwei Zhang

Authors
  1. Mengwei Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jane J. Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liwei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jane J. Ye.

Additional information

The research of Jane J. Ye was partially supported by NSERC and the research of Liwei Zhang was supported by the National Natural Science Foundation of China under Projects Nos. 11071029, 91330206 and 91130007.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, M., Ye, J.J. & Zhang, L. Smoothing augmented Lagrangian method for nonsmooth constrained optimization problems. J Glob Optim 62, 675–694 (2015). https://doi.org/10.1007/s10898-014-0242-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-014-0242-7

Keywords

Mathematics Subject Classification

Search

Navigation