Skip to main content
Log in

Convergence Analysis of the Generalized Alternating Direction Method of Multipliers with Logarithmic–Quadratic Proximal Regularization

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

We consider combining the generalized alternating direction method of multipliers, proposed by Eckstein and Bertsekas, with the logarithmic–quadratic proximal method proposed by Auslender, Teboulle, and Ben-Tiba for solving a variational inequality with separable structures. For the derived algorithm, we prove its global convergence and establish its worst-case convergence rate measured by the iteration complexity in both the ergodic and nonergodic senses.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Note that we follow the work [21] and many others to measure the worst-case convergence rate in term of the iteration complexity. That is, a worst-case \(O(1/t)\) convergence rate means the accuracy to a solution under certain criteria is of the order \(O(1/t)\) after \(t\) iterations of an iterative scheme; or equivalently, it requires at most \(O(1/\varepsilon )\) iterations to achieve an approximate solution with an accuracy of \(\varepsilon \).

References

  1. Eckstein, J., Bertsekas, D.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  2. Auslender, A., Teboulle, M., Ben-Tiba, S.: A logarithmic-quadratic proximal method for variational inequalities. Comput. Optim. Appl. 12, 31–40 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  3. Bertsekas, D.P., Tsitsiklis, J.N.: Parallel and Distributed Computation: Numerical Methods. Athena Scientific, Belmont (1997)

    Google Scholar 

  4. Fukushima, M.: Application of the alternating directions method of multipliers to separable convex programming problems. Comput. Optim. Appl. 1(1), 93–111 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  5. Glowinski, R.: Numerical Methods for Nonlinear Variational Problems. Springer, New York (1984)

    Book  MATH  Google Scholar 

  6. Glowinski, R., Tallec, P.L.: Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics. Society for Industrial and Applied Mathematics, Philadelphia (1989)

    Book  MATH  Google Scholar 

  7. Tseng, P.: Alternating projection-proximal methods for convex programming and variational inequalities. SIAM J. Optim. 7(4), 951–965 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  8. Glowinski, R., Marrocco, A.: Sur l’approximation par éléments finis d’ordre un et la résolution par pénalisation-dualité d’une classe de problémes de Dirichlet non linéaires. Revue Fr. Autom. Inform. Rech. Opér. Anal. Numér. 2, 41–76 (1975)

    MathSciNet  Google Scholar 

  9. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximations. Comput. Math. Appl. 2(1), 17–40 (1976)

    Article  MATH  Google Scholar 

  10. Chan, T.F., Glowinski, R.: Finite element approximation and iterative solution of a class of mildly non-linear elliptic equations. Stanford University, Technical Report (1978)

  11. He, B.S., Liao, L.-Z., Han, D., Yang, H.: A new inexact alternating directions method for monontone variational inequalities. Math. Program. 92, 103–118 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  12. Kontogiorgis, S., Meyer, R.R.: A variable-penalty alternating directions method for convex optimization. Math. Program. 83, 29–53 (1998)

    MATH  MathSciNet  Google Scholar 

  13. Tseng, P.: Applications of a splitting algorithm to decomposition in convex programming and variational inequalities. SIAM J. Con. Optim. 29, 119–138 (1991)

    Article  MATH  Google Scholar 

  14. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2010)

    Article  MATH  Google Scholar 

  15. Martinet, B.: Regularision d’inéquations variationnelles par approximations successive. Revue Fr. Autom. Inform. Rech. Opér. 126, 154–159 (1970)

    MathSciNet  Google Scholar 

  16. Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14(5), 877–898 (1976)

    Article  MATH  MathSciNet  Google Scholar 

  17. Gol’shtein, E.G., Tret’yakov, N.V.: Modified Lagrangian in convex programming and their generalizations. Math. Program. Stud. 10, 86–97 (1979)

    Article  MathSciNet  Google Scholar 

  18. Cai, X.J., Gu, G.Y., He, B.S., Yuan, X.M.: A proximal point algorithm revisit on the alternating direction method of multipliers. Sci. China Math. 56(10), 2179–2186 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  19. Auslender, A., Teboulle, M.: Entropic proximal decomposition method for convex programs and variational inequalities. Math. Program. 91(1), 33–47 (2001)

    MATH  MathSciNet  Google Scholar 

  20. Yuan, X.M., Li, M.: An LQP-based decomposition method for solving a class of variational inequalities. SIAM J. Optim. 21(4), 1309–1318 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  21. Nesterov, Y.E.: A method for solving the convex programming problem with convergence rate \(O(1/k^2)\). Dokl. Akad. Nauk SSSR 269, 543–547 (1983)

    MathSciNet  Google Scholar 

  22. He, B.S., Yuan, X.M.: On the \(O(1/n)\) convergence rate of the Douglas-Rachford alternating direction method. SIAM J. Numer. Anal. 50(2), 700–709 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  23. He, B.S., Yuan, X.M.: On non-ergodic convergence rate of Douglas-Rachford alternating direction method of multipliers. Submission, (2013)

  24. Shefi, R., Teboulle, M.: Rate of convergence analysis of decomposition methods based on the proximal method of multipliers for convex minimization. SIAM J. Optim. 24(1), 269–297 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  25. Tao, M., Yuan, X.M.: On the \(O(1/t)\) convergence rate of alternating direction method with logarithmic-quadratic proximal regularization. SIAM J. Optim. 22(4), 1431–1448 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  26. Facchinei, F., Pang, J.S.: Finite-dimensional Variational Inequalities and Complementarity Problems, Vols. I and II. Springer, New York (2003)

    Google Scholar 

  27. Nesterov, Y.: Gradient methods for minimizing composite objective function. Math. Program. Ser. B 140, 125–161 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  28. He, B.S., Liao, L.-Z., Yuan, X.M.: A LQP-based interior prediction-correction method for nonlinear complementarity problems. J. Comput. Math. 24(1), 33–44 (2006)

    MATH  MathSciNet  Google Scholar 

  29. Ng, M.K., Wang, F., Yuan, X.M.: Inexact alternating direction methods for image recovery. SIAM J. Sci. Comput. 33(4), 1643–1668 (2011)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

The first author was supported by the National Natural Science Foundation of China Grant 11001053, the Program for New Century Excellent Talents in University Grant NCET-12-0111, and the Natural Science Foundation of Jiangsu Province grant BK2012662. The third author was partially supported by the FRG Grant from Hong Kong Baptist University: FRG2/13-14/061 and the General Research Fund from Hong Kong Research Grants Council: 203613.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xiaoming Yuan.

Additional information

Communicated by Masao Fukushima.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, M., Li, X. & Yuan, X. Convergence Analysis of the Generalized Alternating Direction Method of Multipliers with Logarithmic–Quadratic Proximal Regularization. J Optim Theory Appl 164, 218–233 (2015). https://doi.org/10.1007/s10957-014-0567-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-014-0567-x

Keywords

Mathematics Subject Classification

Navigation