Abstract
In this paper, we establish the convergence properties for a majorized alternating direction method of multipliers for linearly constrained convex optimization problems, whose objectives contain coupled functions. Our convergence analysis relies on the generalized Mean-Value Theorem, which plays an important role to properly control the cross terms due to the presence of coupled objective functions. Our results, in particular, show that directly applying two-block alternating direction method of multipliers with a large step length of the golden ratio to the linearly constrained convex optimization problem with a quadratically coupled objective function is convergent under mild conditions. We also provide several iteration complexity results for the algorithm.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
We may instead directly assume the existence of a KKT point without imposing Assumption 2.1 in our convergence analysis.
References
Gabay, D.: Applications of the method of multipliers to variational inequalities. In: Fortin M., Glowinski R. (eds.) Augmented Lagrangian Methods: Applications to the Solution of Boundary Problems, pp. 299–331. North–Holland, Amsterdam (1983)
Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)
Glowinski, R.: Lectures on Numerical Methods for Non-linear Variational Problems, Tata Institute of Fundamental Research Lectures on Mathematics and Physics, Notes by Vijayasundaram, G and Adimurthi, M, vol. 65. Springer, Berlin (1980)
Glowinski, R., Marroco, A.: Approximation by finite elements of order one and solution by penalization-duality of a class of nonlinear dirichlet problems. Revue française d’automatique, informatique, recherche opérationnelle. Analyse numérique 9(2), 41–76 (1975)
Eckstein, J., Bertsekas, D.P.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55(1), 293–318 (1992)
Eckstein, J., Yao, W.: Understanding the convergence of the alternating direction method of multipliers: theoretical and computational perspectives. Pac. J. Optim. 11(4), 619–644 (2014)
Hong, M., Chang, T.H., Wang, X., Razaviyayn, M., Ma, S., Luo, Z.Q.: A block successive upper bound minimization method of multipliers for linearly constrained convex optimization. arXiv preprint arXiv:1401.7079 (2014)
Chen, C., He, B., Ye, Y., Yuan, X.: The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent. Math. Program. 155(1), 57–79 (2016)
Clarke, F.H.: Optimization and Nonsmooth Analysis, vol. 5. SIAM, Philadelphia (1990)
Hiriart-Urruty, J.B., Strodiot, J.J., Nguyen, V.H.: Generalized hessian matrix and second-order optimality conditions for problems with \(C^{1, 1}\) data. Appl. Math. Optim. 11(1), 43–56 (1984)
Li, M., Sun, D., Toh, K.C.: A majorized admm with indefinite proximal terms for linearly constrained convex composite optimization. SIAM J. Optim. (to appear)
He, B., Yuan, X.: On non-ergodic convergence rate of Douglas–Rachford alternating direction method of multipliers. Numerische Mathematik 130(3), 567–577 (2012)
Deng, W., Lai, M., Peng, Z., Yin, W.: Parallel multi-block admm with o(1/k) convergence. arXiv preprint arXiv:1312.3040 (2013)
Davis, D., Yin, W.: Convergence rate analysis of several splitting schemes. arXiv preprint arXiv:1406.4834 (2014)
Li, X., Sun, D., Toh, K.C.: A schur complement based semi-proximal admm for convex quadratic conic programming and extensions. Math. Program. 155(1), 333–373 (2016)
Acknowledgments
The authors would like to thank Caihua Chen at the Nanjing University for discussions on the iteration complexity described in the paper, and Bo Chen at the National University of Singapore for the comments on the global convergence conditions in Theorem 4.1. The research of Defeng Sun was supported in part by the Academic Research Fund (Grant No. R-146-000-207-112). The research of Kim-Chuan Toh was supported in part by the Ministry of Education, Singapore, Academic Research Fund (Grant No. R-146-000-194-112).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Cui, Y., Li, X., Sun, D. et al. On the Convergence Properties of a Majorized Alternating Direction Method of Multipliers for Linearly Constrained Convex Optimization Problems with Coupled Objective Functions. J Optim Theory Appl 169, 1013–1041 (2016). https://doi.org/10.1007/s10957-016-0877-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-016-0877-2
Keywords
- Coupled objective function
- Convex quadratic programming
- Majorization
- Iteration complexity
- Nonsmooth analysis