Skip to main content
Springer Nature Link
Log in

An ADM-based splitting method for separable convex programming

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

We consider the convex minimization problem with linear constraints and a block-separable objective function which is represented as the sum of three functions without coupled variables. To solve this model, it is empirically effective to extend straightforwardly the alternating direction method of multipliers (ADM for short). But, the convergence of this straightforward extension of ADM is still not proved theoretically. Based on ADM’s straightforward extension, this paper presents a new splitting method for the model under consideration, which is empirically competitive to the straightforward extension of ADM and meanwhile the global convergence can be proved under standard assumptions. At each iteration, the new method corrects the output of the straightforward extension of ADM by some slight correction computation to generate a new iterate. Thus, the implementation of the new method is almost as easy as that of ADM’s straightforward extension. We show the numerical efficiency of the new method by some applications in the areas of image processing and statistics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2010)

    Article  MATH  Google Scholar 

  2. Cai, J.F., Chan, R.H., Nikolova, M.: Two phase methods for deblurring images corrupted by impulse plus Gaussian noise. Inverse Probl. Imaging 2, 187–204 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  3. Candés, E., Li, X., Ma, Y., Wright, J.: Robust principal component analysis. J. ACM 58(1), 1–37 (2011)

    Article  Google Scholar 

  4. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40, 120–145 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  5. Chandrasekaran, V., Sanghavi, S., Parrilo, P., Willskyc, A.: Rank-sparsity incoherence for matrix decomposition. SIAM J. Optim. 21(2), 572–596 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  6. Chen, G., Teboulle, M.: A proximal-based decomposition method for convex minimization problems. Math. Program. 64, 81–101 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  7. Chen, C.H., He, B.S., Yuan, X.M.: Matrix completion via alternating direction method. IMA J. Numer. Anal. 32, 227–245 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  8. Esser, E.: Applications of Lagrangian-based alternating direction methods and connections to split Bregman. UCLA CAM Report. 9-31 (2009)

  9. Facchinei, F., Pang, J.: Finite-Dimensional Variational Inequalities and Complementarity Problems, vols. I and II. Springer, Berlin (2003)

    Google Scholar 

  10. Fukushima, M.: Application of the alternating directions method of multipliers to separable convex programming problems. Comput. Optim. Appl. 2, 93–111 (1992)

    Article  MathSciNet  Google Scholar 

  11. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximations. Comput. Math. Appl. 2, 17–40 (1976)

    Article  MATH  Google Scholar 

  12. Glowinski, R., Le Tallec, P.: Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics. SIAM Studies in Applied Mathematics. SIAM, Philadelphia (1989)

    Book  MATH  Google Scholar 

  13. Glowinski, R., Marrocco, A.: Approximation par éléments finis d’ordre un et résolution par pénalisation-dualité d’une classe de problèmes non linéaires. RAIRO. Anal. Numér. 9(2), 41–76 (1975)

    MathSciNet  MATH  Google Scholar 

  14. Hansen, P., Nagy, J., O’Leary, D.: Deblurring Images: Matrices, Spectra, and Filtering. SIAM, Philadelphia (2006)

    Book  MATH  Google Scholar 

  15. He, B.S.: Parallel splitting augmented Lagrangian methods for monotone structured variational inequalities. Comput. Optim. Appl. 42, 195–212 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  16. He, B.S., Liao, L., Han, D., Yang, H.: A new inexact alternating direction method for monotone variational inequalities. Math. Program. 92, 103–118 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  17. He, B.S., Xu, M.H., Yuan, X.M.: Solving large-scale least squares covariance matrix problems by alternating direction methods. SIAM J. Matrix Anal. Appl. 32, 136–152 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  18. He, B.S., Tao, M., Yuan, X.M.: Alternating direction method with Gaussian back substitution for separable convex programming. SIAM J. Optim. 22, 313–340 (2012)

    Article  MATH  Google Scholar 

  19. He, B.S., Tao, M., Xu, M.H., Yuan, X.M.: Alternating directions based contraction method for generally separable linearly constrained convex programming problems. Optimization (to appear)

  20. He, B.S., Tao, M., Yuan, X.M.: A splitting method for separable convex programming. IMA J. Num. Anal. (in revision)

  21. Hestenes, M.: Multiplier and gradient methods. J. Optim. Theory Appl. 4, 303–320 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  22. Huang, Y.M., Ng, M.K., Wen, Y.W.: Fast image restoration methods for impulse and Gaussian noise removal. IEEE Signal Process. Lett. 16, 457–460 (2009)

    Article  Google Scholar 

  23. Hwang, H., Haddad, A.: Adaptive median filters: new algorithms and results. IEEE Trans. Image Process. 4, 499–502 (1995)

    Article  Google Scholar 

  24. Kontogiorgis, S., Meyer, R.: A variable-penalty alternating directions method for convex optimization. Math. Program. 83, 29–53 (1998)

    MathSciNet  MATH  Google Scholar 

  25. Martinet, B.: Regularisation, d’inéquations variationelles par approximations succesives. RAIRO. Rech. Opér. 4, 154–159 (1970)

    MathSciNet  MATH  Google Scholar 

  26. Ng, M.K., Weiss, P., Yuan, X.M.: Solving constrained total-variation problems via alternating direction methods. SIAM J. Sci. Comput. 32, 2710–2736 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  27. Ng, M.K., Wang, F., Yuan, X.M.: Inexact alternating direction methods for image recovery. SIAM J. Sci. Comput. 33(4), 1643–1668 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  28. Peng, Y., Ganesh, A., Wright, J., Xu, W., Ma, Y.: Robust alignment by sparse and low-rank decomposition for linearly correlated images. IEEE Trans. Pattern Anal. Mach. Intel. (to appear)

  29. Powell, M.: A method for nonlinear constraints in minimization problems. In: Fletcher, R. (ed.) Optimization, pp. 283–298. Academic Press, New York (1969)

    Google Scholar 

  30. Rockafellar, R.: Convex Analysis. Princeton University Press, Princeton (1970)

    MATH  Google Scholar 

  31. Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)

    Article  MATH  Google Scholar 

  32. Sun, J., Zhang, S.: A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs. Eur. J. Oper. Res. 207, 1210–1220 (2010)

    Article  MATH  Google Scholar 

  33. Tao, M., Yuan, X.M.: Recovering low-rank and sparse components of matrices from incomplete and noisy observations. SIAM J. Optim. 21, 57–81 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  34. Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., Knight, K.: Sparsity and smoothness via the fused lasso. J. R. Stat. Soc. B 67, 91–108 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  35. Wen, Z.W., Goldfarb, D.: Line search multigrid method for large-scale convex optimization. SIAM J. Optim. 20, 1478–1503 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  36. Wen, Z.W., Goldfarb, D., Yin, W.: Alternating direction augmented Lagrangian methods for semidefinite programming. Math. Program. Comput. 2, 203–230 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  37. Yang, J.F., Zhang, Y.: Alternating direction algorithms for l 1-problems in compressive sensing. SIAM J. Sci. Comput. 33, 250–278 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  38. Yuan, X.M.: Alternating direction methods for covariance selection models. J. Sci. Comput. 51, 261–273 (2012)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The research of the first author is supported by the National Natural Science Foundation of China (NSFC) grants 11071122 and 10871098, and Grant 20103207110002 from MOE of China. The research of the second author is supported by the Hong Kong General Research Fund: 203311. The third author would like to thank Dr. Caihua Chen for his helpful comments.

Author information

Authors and Affiliations

  1. School of Mathematical Sciences and Jiangsu Key Laboratory for NSLSCS, Nanjing Normal University, Nanjing, 210023, P.R. China

    Deren Han

  2. Department of Mathematics, Hong Kong Baptist University, Hong Kong, P.R. China

    Xiaoming Yuan

  3. Department of Mathematics, Nanjing University, Nanjing, 210093, P.R. China

    Wenxing Zhang & Xingju Cai

Authors
  1. Deren Han
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xiaoming Yuan
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Wenxing Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Xingju Cai
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Xiaoming Yuan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Han, D., Yuan, X., Zhang, W. et al. An ADM-based splitting method for separable convex programming. Comput Optim Appl 54, 343–369 (2013). https://doi.org/10.1007/s10589-012-9510-y

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-012-9510-y

Keywords