Abstract
This paper considers a special class of convex programming (CP) problems whose feasible regions consist of a simple compact convex set intersected with an affine manifold. We present first-order methods for this class of problems based on an inexact version of the classical augmented Lagrangian (AL) approach, where the subproblems are approximately solved by means of Nesterov’s optimal method. We then establish a bound on the total number of Nesterov’s optimal iterations, i.e., the inner iterations, performed throughout the entire inexact AL method to obtain a near primal-dual optimal solution. We also present variants with possibly better iteration-complexity bounds than the original inexact AL method, which consist of applying the original approach directly to a perturbed problem obtained by adding a strongly convex component to the objective function of the CP problem.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bertsekas, D.: Constrained Optimization and Lagrange Multiplier Methods, 1st edn. Academic Press, New York (1982)
Bertsekas, D.: Nonlinear Programming, 2nd edn. Athena Scientific, New York (1984)
Burer, S., Monteiro, R.D.C.: A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization. Math. Program. Series B 95, 329–357 (2003)
Burer, S., Monteiro, R.D.C.: Local minima and convergence in low-rank semidefinite programming. Math. Program. 103, 427–444 (2005)
Ghadimi, S., Lan, G.: Optimal stochastic approximation algorithms for strongly convex stochastic composite optimization, I: a generic algorithmic framework. SIAM J. Optim. 22, 1469–1492 (2012)
Golshtein, E.G., Tretyakov, N.V.: Modified Lagrangians and Monotone Maps in Optimization. Springer, New York (1996)
Hestenes, M.R.: Multiplier and gradient methods. J. Optim. Appl. 4, 303–320 (1969)
Jarre, F., Rendl, F.: An augmented primal-dual method for linear conic programs. Manuscript, Institut fur Mathematik, Universit at Dusseldorf, Germany, Austria, (2007)
Lan, G.: Convex optimization under inexact first-order information. Ph.D. Dissertation, School of Industrial Systems Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA, (2009)
Lan, G.: Bundle-level type methods uniformly optimal for smooth andnon-smooth convex optimization. Manuscript, Department of Industrialand Systems Engineering, University of Florida, Gainesville, FL32611, USA, January 2013. Revision submitted to MathematicalProgramming
Lan, G., Monteiro, R.D.C.: Iteration-complexity of first-order penalty methods for convex programming. Math. Program. 138, 115–139 (2013)
Monteiro, R.D.C., Svaiter, B.F.: Complexity of variants of tseng’s modified f-b splitting and korpelevich’s methods for hemi-variational inequalities with applications to saddle-point and convex optimization problems. Manuscript, School of ISyE, Georgia Tech, Atlanta, GA, 30332, USA, (2010)
Nemirovski, A.: Prox-method with rate of convergence \(o(1/t)\) for variational inequalities with lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM J. Optim. 15, 229–251 (2004)
Nemirovski, A.S., Yudin, D.: Problem complexity and method efficiency in optimization. Wiley-Interscience Series in Discrete Mathematics. John Wiley, XV, (1983)
Nesterov, Y.E.: A method for unconstrained convex minimization problem with the rate of convergence \(O(1/k^2)\). Dokl. AN SSSR 269, 543–547 (1983)
Nesterov, Y.E.: Introductory Lectures on Convex Optimization: A Basic Course. Kluwer Academic Publishers, Massachusetts (2004)
Nesterov, Y.E.: Smooth minimization of nonsmooth functions. Math. Program. 103, 127–152 (2005)
Nesterov, Y.E.: Gradient methods for minimizing composite objective functions. Technical report, Center for Operations Research and Econometrics (CORE), Catholic University of Louvain (2007)
Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (1999)
Pillo, G.D., Lucidi, S.: An augmented lagrangian function with improved exactness properties. SIAM J. Optim. 12, 376–406 (2002)
Powell, M.M.D.: An efficient method for nonlinear constraints in minimization problems. In: Fletcher, R. (ed.) Optimization, pp. 283–298. Academic Press, London (1969)
Ruszczynski, A.: Nonlinear Optimization, 1st edn. Princeton University Press, Princeton (2006)
Zhao, X., Sun, D., Toh, K.: A Newton-CG augmented Lagrangian method for semidefinite programming. Manuscript, National University of Singapore, Singapore (2008)
Author information
Authors and Affiliations
Corresponding author
Additional information
The work has been partially supported by NSF Grant CCF-0808863, CMMI-1000347, CMMI-1254446, and DMS-1319050, and ONR Grant N00014-08-1-0033 and N00014-13-1-0036.
Rights and permissions
About this article
Cite this article
Lan, G., Monteiro, R.D.C. Iteration-complexity of first-order augmented Lagrangian methods for convex programming. Math. Program. 155, 511–547 (2016). https://doi.org/10.1007/s10107-015-0861-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-015-0861-x