Abstract
This note proposes a novel approach to derive a worst-case \(O(1/k)\) convergence rate measured by the iteration complexity in a non-ergodic sense for the Douglas–Rachford alternating direction method of multipliers proposed by Glowinski and Marrocco.
Similar content being viewed by others
Notes
As [11, 12] and many others, a worst-case \(O(1/k)\) convergence rate measured by the iteration complexity means the accuracy to a solution under certain criterion is of the order \(O(1/k)\) after \(k\) iterations of an iterative scheme; or equivalently, it requires at most \(O(1/\epsilon )\) iterations to achieve an approximate solution with an accuracy of \(\epsilon \).
References
Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2010)
Deng, W., Lai, M. J., Peng, Z. M., Yin, W. T.: Parallel multi-block ADMM with \(o(1/k)\) convergence. 2014 (in press)
Eckstein, J., Yao, W.: Augmented Lagrangian and alternating direction methods for convex optimization: a tutorial and some illustrative computational results. RUTCOR Research Report RRR 32–2012 (2012)
Fortin, M., Glowinski, R.: Augmented lagrangian methods: applications to the numerical solutions of boundary value problems. Stud. Math. Appl. 15. NorthHolland, Amsterdam (1983)
Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite-element approximations. Comput. Math. Appl. 2, 17–40 (1976)
Glowinski, R.: On alternating direction methods of multipliers: a historical perspective. In: Fitzgibbon, W., Kuznetsov, Y.A., Neittaanmaki, P., Pironneau, O. (eds.) Modeling, Simulation and Optimization for Science and Technology. Computational Methods in Applied Sciences, vol. 34, pp. 59–82. Springer, Dordrecht (2014)
Glowinski, R., Marroco, A.: Approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires. ESAIM: Mathematical Modelling and Numerical Analysis - Modélisation Mathématique et Analyse Numérique, vol. 9, R2, pp. 41–76 (1975)
Glowinski, R., Le Tallec, P.: Augmented lagrangian and operator-splitting methods in nonlinear mechanics. SIAM Studies in Applied Mathematics, Philadelphia (1989)
He, B.S., Yang, H.: Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities. Oper. Res. Lett. 23, 151–161 (1998)
He, B.S., Yuan, X.M.: On the O(1/n) convergence rate of Douglas–Rachford alternating direction method. SIAM J. Numer. Anal. 50, 700–709 (2012)
Nemirovsky, A. S., Yudin, D. B.: Problem complexity and method efficiency in Optimization. Wiley-Interscience Series in Discrete Mathematics, Wiley, New York (1983)
Nesterov, Y.E.: A method for solving the convex programming problem with convergence rate \(O(1/k^2)\). Dokl. Akad. Nauk SSSR 269, 543–547 (1983)
Zhang, X.Q., Burger, M., Bresson, X., Osher, S.: Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM J. Imaging Sci. 3(3), 253–276 (2010)
Author information
Authors and Affiliations
Corresponding author
Additional information
X. Yuan was supported by the General Research Fund from Hong Kong Research Grants Council: 203613.
B. He was supported by the NSFC Grant 91130007 and 11471156.
Rights and permissions
About this article
Cite this article
He, B., Yuan, X. On non-ergodic convergence rate of Douglas–Rachford alternating direction method of multipliers. Numer. Math. 130, 567–577 (2015). https://doi.org/10.1007/s00211-014-0673-6
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00211-014-0673-6