Skip to main content
SpringerLink
Log in

A distributed Douglas-Rachford splitting method for multi-block convex minimization problems

  • Published:
Advances in Computational Mathematics Aims and scope Submit manuscript

Abstract

A customized Douglas-Rachford splitting method (DRSM) was recently proposed to solve two-block separable convex optimization problems with linear constraints and simple abstract constraints. The algorithm has advantage over the well-known alternating direction method of multipliers (ADMM), the dual application of DRSM to the two-block convex minimization problem, in the sense that the subproblems can have larger opportunity of possessing closed-form solutions since they are unconstrained. In this paper, we further study along this way by considering the primal application of DRSM for the general case m≥3, i.e., we consider the multi-block separable convex minimization problem with linear constraints where the objective function is separable into m individual convex functions without coupled variables. The resulting method fully exploits the separable structure and enjoys decoupled subproblems which can be solved simultaneously. Both the exact and inexact versions of the new method are presented in a unified framework. Under mild conditions, we manage to prove the global convergence of the algorithm. Preliminary numerical experiments for extracting the background from corrupted surveillance video verify the encouraging efficiency of the new algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bardsley, J., Knepper, S., Nagy, J.: Structured linear algebra problems in adaptive optics imaging. Adv. Comput. Math. 35, 103–117 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bauschke, H., Combettes, P.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, New York (2011)

    Book  MATH  Google Scholar 

  3. Bertsekas, D., Tsitsiklis, J.: Parallel and Distributed Computation, Numerical Methods. Prentice-Hall, Englewood Cliffs, NJ (1989)

    MATH  Google Scholar 

  4. Bose, N., Boo, K.: High-resolution image reconstruction with multisensors. Int. J. Imag. Syst. Tech. 9, 294–304 (1998)

    Article  Google Scholar 

  5. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2010)

    Article  MATH  Google Scholar 

  6. Cai, J., Candés, E., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20, 1956–1982 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  7. Cai, J., Chan, R., Nikolova, M.: Two-phase approach for deblurring images corrupted by impulse plus gaussian noise. Inverse Prob. Imag. 2, 187–204 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  8. Cai, J., Osher, S., Shen, Z.: Split Bregman methods and frame based image restoration. Multiscale Model. Simul. 8, 337–369 (2009)

    Article  MathSciNet  Google Scholar 

  9. Candés, E., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM 58, 1–37 (2011)

    Article  Google Scholar 

  10. Chan, R., Yang, J., Yuan, X.: Alternating direction method for image inpainting in wavelet domain. SIAM J. Imaging Sci. 4, 807–826 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  11. Chandrasekaran, V., Sanghavi, S., Parrilo, P., Willskyc, A.: Rank-sparsity incoherence for matrix decomposition. SIAM J. Optim. 21, 572–596 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  12. Chen, C., He, B., Ye, Y., Yuan, X.: The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent. to appear in Mathematical Programming Series A. doi:10.1007/s10107-014-0826-5

  13. Chen, C., He, B., Yuan, X.: Matrix completion via alternating direction method. IMA J. Numer. Anal. 32, 227–245 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  14. Chen, G., Teboulle, M.: A proximal-based decomposition method for convex minimization problems. Math. Program. 64, 81–101 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  15. Combettes, P., Pesquet, J.: Proximal splitting methods in signal processing. In: Bauschke, H., Burachik, R., Combettes, P., Elser, V., Luke, D., Wolkowicz, H. (eds.) Fixed-Point Algorithms for Inverse Problems in Science and Engineering, Springer Optimization and Its Applications, Vol. 49, pp 185–212. Springer, New York (2011)

    Chapter  Google Scholar 

  16. Dinh, Q., Necoara, I., Savorgnan, C., Diehl, M.: An inexact perturbed path-following method for Lagrangian decomposition in large-scale separable convex optimization. SIAM J. Optim. 23, 95–125 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  17. Douglas, J., Rachford, H.: On the numerical solution of the heat conduction problem in 2 and 3 space variables. Trans. Amer. Math. Soc. 82, 421–439 (1956)

    Article  MathSciNet  MATH  Google Scholar 

  18. Eckstein, J.: Augmented Lagrangian and alternating direction methods for convex optimization: A tutorial and some illustrative computational results. Tech. Rep. 32-2012. Rutgers University (2012)

  19. Eckstein, J., Bertsekas, D.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  20. Facchinei, F., Pang, J.: Finite-Dimensional Variational Inequalities and Complementarity Problems. Springer, New York (2003)

    Google Scholar 

  21. Fukushima, M.: The primal Douglas-Rachford splitting algorithm for a class of monotone mappings with application to the traffic equilibrium problem. Math. Program. 72, 1–15 (1996)

    MathSciNet  MATH  Google Scholar 

  22. Gabay, D.: Applications of the method of multipliers to variational inequalities. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, pp 299–331. The Netherlands (1983)

  23. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximations. Comput. Math. Appl. 2, 16–40 (1976)

    Article  Google Scholar 

  24. Goldstein, T., Osher, S.: The split Bregman method for ℓ 1 regularized problems. SIAM J. Imaging Sci. 2, 323–343 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  25. Han, D., He, H., Xu, L.: A proximal parallel splitting method for minimizing sum of convex functions with linear constraints. J. Comput. Appl. Math. 256, 36–51 (2014)

    Article  MathSciNet  Google Scholar 

  26. Han, D., He, H., Yang, H., Yuan, X.: A customized Douglas-Rachford splitting algorithm for separable convex minimization with linear constraints. Numer. Math. 127, 167–200 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  27. Han, D., Yuan, X.: A note on the alternating direction method of multipliers. J. Optim. Theory Appl. 155(1), 227–238 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  28. Han, D., Yuan, X., Zhang, W.: An augmented-Lagrangian-based parallel splitting method for separable convex minimization with applications to image processing. Math. Comput. 83, 2263–2291 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  29. He, B.: Parallel splitting augmented Lagrangian methods for monotone structured variational inequalities. Comput. Optim. Appl. 42, 195–212 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  30. He, B., Fu, X., Jiang, Z.: Proximal-point algorithm using a linear proximal term. J. Optim. Theory Appl. 141, 299–319 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  31. He, B., Hou, L., Yuan, X.: On full Jacobian decomposition of the augmented Lagrangian method for separable convex programming. Available at. http://www.optimization-online.org/DB_FILE/2013/05/3894.pdf

  32. He, B., Liao, L., Han, D., Yang, H.: A new inexact alternating direction method for monotone variational inequalities. Math. Program. 92, 103–118 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  33. He, B., Tao, M., Xu, M., Yuan, X.: Alternating directions based contraction method for generally separable linearly constrained convex programming problems. Optimizaition 62, 573–596 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  34. He, B., Tao, M., Yuan, X.: A splitting method for separate convex programming with linking linear constraints. IMA J. Numer. Anal. doi:10.1093/imanum/drt060

  35. He, B., Tao, M., Yuan, X.: Alternating direction method with Gaussian-back substitution for separable convex programming. SIAM J. Optim. 22, 313–340 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  36. He, B., Xu, M., Yuan, X.: Solving large-scale least squares covariance matrix problems by alternating direction methods. SIAM J. Matrix Anal. Appl. 32, 136–152 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  37. Hestenes, M.: Multiplier and gradient methods. J. Optim. Theory Appl. 4, 303–320 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  38. Hong, M., Luo, Z.: On the linear convergence of the alternating direction method of multipliers. ArXiv:1208.3922v3 (2013)

  39. Larsen, R.: PROPACK-software for large and sparse SVD calculations. Http://sun.stanford.edu/srmunk/PROPACK/

  40. Lions, P., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16, 964–979 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  41. Ma, S., Xue, L., Zou, H.: Alternating direction methods for latent variable Gaussian graphical model selection. Neural Comput. 25, 2172–2198 (2013)

    Article  MathSciNet  Google Scholar 

  42. Moreau, J.: Fonctions convexe dudual et points proximaux dans un espace hilbertien. C. R. Acad. Sci. Paris Ser. A Math. 255, 2897–2899 (1962)

    MathSciNet  MATH  Google Scholar 

  43. Nediċ, A., Ozdaglar, A.: Subgradient methods for saddle point problems. J. Optim. Theory Appl. 142, 205–228 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  44. Oreifej, O., Li, X., Shah, M.: Simultaneous video stabilization and moving object detection in turbulence. IEEE Trans. Pattern Anal. Mach. Intell. 35, 450–462 (2013)

    Article  Google Scholar 

  45. Powell, M.: A method for nonlinear constraints in minimization problems. In: Fletcher, R. (ed.) Optimization, pp 283–298, London (1969)

  46. Rockafellar, R.: On the maximal monotonicity of subdifferential mappings. Pacific J. Math. 33(1), 209–216 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  47. Rockafellar, R.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  48. Setzer, S., Steidl, G., Tebuber, T.: Deblurring Poissonian images by split Bregman techniques. J. Vis. Commun. Image R 21, 193–199 (2010)

    Article  Google Scholar 

  49. Tao, M., Yuan, X.: Recovering low-rank and sparse components of matrices from incomplete and noisy observations. SIAM J. Optim. 21, 57–81 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  50. Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., Knight, K.: Sparsity and smoothness via the fused LASSO. J. R. Statist. Soc. B 67, 91–108 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  51. Varga, R.: Matrix Iterative Analysis. Prentice-Hall, Englewood Cliffs, New Jersey (1966)

    Google Scholar 

  52. Wen, Z., Goldfarb, D., Yin, W.: Alternating direction augmented Lagrangian methods for semidefinite programming. Math. Program. Comput. 2, 203–230 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  53. Yang, J., Zhang, Y.: Alternating direction algorithms for ℓ 1-problems in compressive sensing. SIAM J. Sci. Comput. 332, 250–278 (2011)

    Article  Google Scholar 

  54. Yuan, X.: Alternating direction methods for covariance selection models. J. Sci. Comput. 51, 261–273 (2012)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, School of Science, Hangzhou Dianzi University, Hangzhou, 310018, China

    Hongjin He

  2. School of Mathematical Sciences, Jiangsu Key Labratory for NSLSCS, Nanjing Normal University, Nanjing, 210023, China

    Deren Han

Authors
  1. Hongjin He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Deren Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Deren Han.

Additional information

Communicated by: Zydrunas Gimbutas

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, H., Han, D. A distributed Douglas-Rachford splitting method for multi-block convex minimization problems. Adv Comput Math 42, 27–53 (2016). https://doi.org/10.1007/s10444-015-9408-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10444-015-9408-1

Keywords

Mathematics Subject Classifications (2010)

Search

Navigation