Skip to main content
Log in

On non-ergodic convergence rate of Douglas–Rachford alternating direction method of multipliers

  • Published:
Numerische Mathematik Aims and scope Submit manuscript

Abstract

This note proposes a novel approach to derive a worst-case \(O(1/k)\) convergence rate measured by the iteration complexity in a non-ergodic sense for the Douglas–Rachford alternating direction method of multipliers proposed by Glowinski and Marrocco.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. As [11, 12] and many others, a worst-case \(O(1/k)\) convergence rate measured by the iteration complexity means the accuracy to a solution under certain criterion is of the order \(O(1/k)\) after \(k\) iterations of an iterative scheme; or equivalently, it requires at most \(O(1/\epsilon )\) iterations to achieve an approximate solution with an accuracy of \(\epsilon \).

References

  1. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2010)

    Article  MATH  Google Scholar 

  2. Deng, W., Lai, M. J., Peng, Z. M., Yin, W. T.: Parallel multi-block ADMM with \(o(1/k)\) convergence. 2014 (in press)

  3. Eckstein, J., Yao, W.: Augmented Lagrangian and alternating direction methods for convex optimization: a tutorial and some illustrative computational results. RUTCOR Research Report RRR 32–2012 (2012)

  4. Fortin, M., Glowinski, R.: Augmented lagrangian methods: applications to the numerical solutions of boundary value problems. Stud. Math. Appl. 15. NorthHolland, Amsterdam (1983)

  5. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite-element approximations. Comput. Math. Appl. 2, 17–40 (1976)

    Article  MATH  Google Scholar 

  6. Glowinski, R.: On alternating direction methods of multipliers: a historical perspective. In: Fitzgibbon, W., Kuznetsov, Y.A., Neittaanmaki, P., Pironneau, O. (eds.) Modeling, Simulation and Optimization for Science and Technology. Computational Methods in Applied Sciences, vol. 34, pp. 59–82. Springer, Dordrecht (2014)

  7. Glowinski, R., Marroco, A.: Approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires. ESAIM: Mathematical Modelling and Numerical Analysis - Modélisation Mathématique et Analyse Numérique, vol. 9, R2, pp. 41–76 (1975)

  8. Glowinski, R., Le Tallec, P.: Augmented lagrangian and operator-splitting methods in nonlinear mechanics. SIAM Studies in Applied Mathematics, Philadelphia (1989)

    Book  MATH  Google Scholar 

  9. He, B.S., Yang, H.: Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities. Oper. Res. Lett. 23, 151–161 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  10. He, B.S., Yuan, X.M.: On the O(1/n) convergence rate of Douglas–Rachford alternating direction method. SIAM J. Numer. Anal. 50, 700–709 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  11. Nemirovsky, A. S., Yudin, D. B.: Problem complexity and method efficiency in Optimization. Wiley-Interscience Series in Discrete Mathematics, Wiley, New York (1983)

  12. Nesterov, Y.E.: A method for solving the convex programming problem with convergence rate \(O(1/k^2)\). Dokl. Akad. Nauk SSSR 269, 543–547 (1983)

    MathSciNet  Google Scholar 

  13. Zhang, X.Q., Burger, M., Bresson, X., Osher, S.: Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM J. Imaging Sci. 3(3), 253–276 (2010)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xiaoming Yuan.

Additional information

X. Yuan was supported by the General Research Fund from Hong Kong Research Grants Council: 203613.

B. He was supported by the NSFC Grant 91130007 and 11471156.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, B., Yuan, X. On non-ergodic convergence rate of Douglas–Rachford alternating direction method of multipliers. Numer. Math. 130, 567–577 (2015). https://doi.org/10.1007/s00211-014-0673-6

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00211-014-0673-6

Mathematics Subject Classification

Navigation