Skip to main content
SpringerLink
Log in

On inexact ADMMs with relative error criteria

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

In this paper, we develop two inexact alternating direction methods of multipliers (ADMMs) with relative error criteria for which only a few parameters are needed to control the error tolerance. In many practical applications, the numerical performance is often improved if a larger step-length is used. Hence in this paper we also consider to seek a larger step-length to update the Lagrangian multiplier for better numerical efficiency. Specifically, if we only allow one subproblem in the classic ADMM to be solved inexactly by a certain relative error criterion, then a larger step-length can be used to update the Lagrangian multiplier. Related convergence analysis of those proposed algorithms is also established under the assumption that the solution set to the KKT system of the problem is not empty. Numerical experiments on solving total variation (TV)-based image denosing and analysis sparse recovery problems are provided to demonstrate the effectiveness of the proposed methods and the advantage of taking a larger step-length.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  Google Scholar 

  2. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  Google Scholar 

  3. Chen, L., Li, X., Sun, D.F., Toh, K.C.: On the equivalence of inexact proximal ALM and ADMM for a class of convex composite programming. arXiv:1803.10803 (2018)

  4. Chen, L., Sun, D.F., Toh, K.C.: A note on the convergence of ADMM for linearly constrained convex optimizaiton problems. Comput. Optim. Appl. 166, 327–343 (2017)

    Article  Google Scholar 

  5. Chen, L., Sun, D.F., Toh, K.C.: An efficient inexact symmetric Gauss–Seidel based majorized ADMM for high-dimensional convex composite conic programming. Math. Program. 161(1), 237–270 (2017)

    Article  MathSciNet  Google Scholar 

  6. Davis, T.A.: Direct Methods for Sparse Linear Systems. SIAM, Philadelphia (2006)

    Book  Google Scholar 

  7. Eckstein, J.: Augmented Lagrangian and alternating direction methods for convex optimization: a tutorial and some illustrative computational results. Rutcor research report, Rutgers Center for Operations Research, Rutgers University (2012)

  8. Eckstein, J., Bertsekas, D.P.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55(1–3), 293–318 (1992)

    Article  MathSciNet  Google Scholar 

  9. Eckstein, J., Silva, P.J.S.: A practical relative error criterion for augmented Lagrangians. Math. Program. 141(1–2), 319–348 (2013)

    Article  MathSciNet  Google Scholar 

  10. Eckstein, J., Yao, W.: Approximate ADMM algorithms derived from Lagrangian splitting. Comput. Optim. Appl. 68(2), 363–405 (2017)

    Article  MathSciNet  Google Scholar 

  11. Eckstein, J., Yao, W.: Relative-error approximate versions of Douglas–Rachford splitting and special cases of the ADMM. Math. Program. 170(2), 417–444 (2018)

    Article  MathSciNet  Google Scholar 

  12. Fazel, M., Pong, T.K., Sun, D.F., Tseng, P.: Hankel matrix rank minimization with applications to system identification and realization. SIAM J. Matrix Anal. Appl. 34(3), 946–977 (2013)

    Article  MathSciNet  Google Scholar 

  13. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)

    Article  Google Scholar 

  14. Glowinski, R.: Numerical Methods for Nonlinear Variational Problems. Springer, New York (1984)

    Book  Google Scholar 

  15. Glowinski, R.: On alternating direction methods of multipliers: a historical perspective. In: Fitzgibbon, W., Kuznetsov, Y.A., Neittaanmaki, P., Pironneau, O. (eds.) Modeling, Simulation and Optimization for Science and Technology, vol. 34, pp. 59–82. Springer, Netherlands (2014)

    Google Scholar 

  16. Glowinski, R., Marroco, A.: Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de dirichlet non linéires. Revue Francaise d’Automatique, Informatique et Recherche Opérationelle. Analyse numérique 9(2), 41–76 (1975)

    Article  Google Scholar 

  17. Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. The Johns Hopkins University Press, Baltimore (1996)

    MATH  Google Scholar 

  18. Hager, W.W., Zhang, H.C.: Inexact alternating direction multiplier methods for separable convex optimization. arXiv:1604.02494 (2016)

  19. He, B.S., Liao, L.Z., Han, D., Yang, H.: A new inexact alternating directions method for monotone variational inequalities. Math. Program. 92(1), 103–118 (2002)

    Article  MathSciNet  Google Scholar 

  20. Li, M., Liao, L.Z., Yuan, X.: Inexact alternating direction methods of multipliers with logarithmic–quadratic proximal regularization. J. Optim. Theory Appl. 159(2), 412–436 (2013)

    Article  MathSciNet  Google Scholar 

  21. Nam, A.S., Davies, M.E., Elad, M., Gribonval, R.: The cosparse analysis model and algorithms. Appl. Comput. Harmon. Anal. 34(1), 30–56 (2013)

    Article  MathSciNet  Google Scholar 

  22. Ng, M.K., Wang, F., Yuan, X.: Inexact alternating direction methods for image recovery. SIAM J. Sci. Comput. 33(4), 1643–1668 (2011)

    Article  MathSciNet  Google Scholar 

  23. Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer, New York (2006)

    MATH  Google Scholar 

  24. Rockafellar, R.T.: Convex Analysis. Princeton mathematical series. Princeton University Press, Princeton (1970)

    Book  Google Scholar 

  25. Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D 60(1–4), 259–268 (1992)

    Article  MathSciNet  Google Scholar 

  26. Solodov, M.V., Svaiter, B.F.: A hybrid approximate extragradient-proximal point algorithm using the enlargement of a maximal monotone operator. Set Valued Anal. 7(4), 323–345 (1999)

    Article  MathSciNet  Google Scholar 

  27. Solodov, M.V., Svaiter, B.F.: An inexact hybrid generalized proximal point algorithm and some new results on the theory of Bregman functions. Math. Oper. Res. 25(2), 214–230 (2000)

    Article  MathSciNet  Google Scholar 

  28. Wen, Z., Goldfarb, D., Yin, W.: Alternating direction augmented Lagrangian methods for semidefinite programming. Math. Prog. Comput. 2(3), 203–230 (2010)

    Article  MathSciNet  Google Scholar 

  29. Xie, J., Liao, A., Yang, X.: An inexact alternating direction method of multipliers with relative error criteria. Optim. Lett. 11(3), 583–596 (2017)

    Article  MathSciNet  Google Scholar 

  30. Xu, M.H.: Proximal alternating directions method for structured variational inequalities. J. Optim. Theory Appl. 134, 107–117 (2007)

    Article  MathSciNet  Google Scholar 

  31. Xu, M.H., Wu, T.: A class of linearized proximal alternating direction methods. J. Optim. Theory Appl. 151(2), 321–337 (2011)

    Article  MathSciNet  Google Scholar 

  32. Zhao, T., Eldar, Y.C., Beck, A., Nehorai, A.: Smoothing and decomposition for analysis sparse recovery. IEEE Trans. Signal Process. 62(7), 1762–1774 (2014)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. LSEC, ICMSEC, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, 100190, China

    Jiaxin Xie

Authors
  1. Jiaxin Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiaxin Xie.

Additional information

This work was supported by the China Postdoctoral Science Foundation (Grant Nos. 2017LH043, 2017M620938).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, J. On inexact ADMMs with relative error criteria. Comput Optim Appl 71, 743–765 (2018). https://doi.org/10.1007/s10589-018-0022-2

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-018-0022-2

Keywords

Mathematics Subject Classification

Search

Navigation