Skip to main content
SpringerLink
Log in

A Parallel Splitting Method for Separable Convex Programs

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

In this paper, we propose a new parallel splitting augmented Lagrangian method for solving the nonlinear programs where the objective function is separable with three operators and the constraint is linear. The method is an improvement of the method of He (Comput. Optim. Appl., 2(42):195–212, 2009), where we generate a predictor using the same parallel splitting augmented Lagrangian scheme as that in He (Comput. Optim. Appl., 2(42):195–212, 2009), while adopting a new strategy to get the next iterate. Under the mild assumptions of convexity of the underlying mappings and the non-emptiness of the solution set, we prove that the proposed algorithm is globally convergent. We apply the new method in the area of image processing and to solve some quadratic programming problems. The preliminary numerical results indicate that the new method is efficient.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Bertsekas, D.P., Gafni, E.M.: Projection method for variational inequalities with applications to the traffic assignment problem. Math. Program. Stud. 17, 139–159 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bertsekas, D.P., Tsitsiklis, J.N.: Parallel and Distributed Computation, Numerical Methods. Prentice-Hall, Englewood Cliffs (1989)

    MATH  Google Scholar 

  3. Bertsekas, D.P.: Constrained Optimization and Lagrange Multiplier Methods. Academic Press, Boston (1982)

    MATH  Google Scholar 

  4. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2010)

    Article  MATH  Google Scholar 

  5. Gabay, D.: Applications of the method of multipliers to variational inequalities. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, pp. 299–331. North-Holland, Amsterdam (1983)

    Chapter  Google Scholar 

  6. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximations. Comput. Math. Appl. 2, 17–40 (1976)

    Article  MATH  Google Scholar 

  7. Chen, G., Teboulle, M.: A proximal-based decomposition method for convex minimization problems. Math. Program. 64, 81–101 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  8. Eckstein, J.: Some saddle-function splitting methods for convex programming. Optim. Methods Softw. 4, 75–83 (1994)

    Article  MathSciNet  Google Scholar 

  9. Fukushima, M.: Application of the alternating directions method of multipliers to separable convex programming problems. Comput. Optim. Appl. 2, 93–111 (1992)

    Article  MathSciNet  Google Scholar 

  10. He, B.S., Liao, L.Z.: Improvements of some projection methods for monotone nonlinear variational inequalities. J. Optim. Theory Appl. 112, 111–128 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  11. Tseng, P.: Applications of splitting algorithm to decomposition in convex programming and variational inequalities. SIAM J. Optim. 7, 951–965 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  12. Esser, E.: Applications of Lagrangian-based alternating direction methods and connections to split Bregman. UCLA CAM Report 09-31 (2009)

  13. He, B.S., Xu, M.H., Yuan, X.M.: Solving large-scale least squares covariance matrix problems by alternating direction methods. SIAM J. Matrix Anal. Appl. 32, 136–152 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  14. Ng, M.K., Weiss, P.A., Yuan, X.M.: Solving constrained total-variation problems via alternating direction methods. SIAM J. Sci. Comput. 32, 2710–2736 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  15. Sun, J., Zhang, S.: A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs. Eur. J. Oper. Res. 207, 1210–1220 (2010)

    Article  MATH  Google Scholar 

  16. Yang, J.F., Zhang, Y.: Alternating direction algorithms for ℓ 1 problems in compressive sensing. SIAM J. Sci. Comput. 33, 250–278 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  17. Han, D.R., Yuan, X.M.: A note on the alternating direction method of multipliers. J. Optim. Theory Appl. 155, 227–238 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  18. Han, D.R., Yuan, X.M., Zhang, W.X.: An augmented-Lagrangian-based parallel splitting method for separable convex programming with applications to image processing. Manuscript

  19. He, B.S.: Parallel splitting augmented Lagrangian methods for monotone structured variational inequalities. Comput. Optim. Appl. 2(42), 195–212 (2009)

    Article  Google Scholar 

  20. Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., Knight, K.: Sparsity and smoothness via the fused lasso. J. R. Stat. Soc. 67, 91–108 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  21. Wen, Z.W., Goldfarb, D.: Line search multigrid method for large-scale convex optimization. SIAM J. Optim. 20, 1478–1503 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  22. Huang, Y.M., Ng, M.K., Wen, Y.W.: Fast image restoration methods for impulse an Gaussian noise removal. IEEE Signal Process. Lett. 16, 457–460 (2009)

    Article  Google Scholar 

  23. Rockafellar, R.T.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 1, 76–112 (1976)

    Article  MathSciNet  Google Scholar 

  24. Glowinski, R., Le Tallec, P.: Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics. SIAM, Philadelphia (1989)

    Book  MATH  Google Scholar 

  25. Wang, Y., Yang, J., Yin, W., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Imaging Sci. 1, 248–272 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  26. Tao, M., Yuan, X.M.: Recovering low-rank and sparse components of matrices from incomplete and noisy observations. SIAM J. Optim. 21, 57–81 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  27. Han, D.R., Yuan, X.M., Zhang, W.X., Cai, X.J.: An ADM-based splitting method for separable convex programming. Comput. Optim. Appl. (2013). doi:10.1007/s10589-012-9510-y

    MathSciNet  Google Scholar 

  28. He, B.S., Tao, M., Xu, M.H., Yuan, X.M.: Alternating directions based contraction method for generally separable linearly constrained convex programming problems. Optimization (2013, to appear). doi:10.1080/02331934.2011.611885

  29. He, B.S., Tao, T., Yuan, X.M.: Alternating direction method with Gaussian back substitution for separable convex programming. SIAM J. Optim. 22, 313–340 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  30. Migdalas, A., Pardalos, P.M., Storoy, S.: Parallel Computing in Optimization. Kluwer Academic, Dordrecht (1997)

    Book  MATH  Google Scholar 

  31. Migdalas, A., Toraldo, G., Kumar, V.: Nonlinear optimization and parallel computing. Parallel Comput. 29(4), 375–391 (2003)

    Article  MathSciNet  Google Scholar 

  32. Pardalos, P.M., Rajasekaran, S.: Advances in Randomized Parallel Computing. Kluwer Academic, Dordrecht (1999)

    Book  MATH  Google Scholar 

  33. D’Apuzzo, M., Marino, M., Migdalas, A., Pardalos, P.M., Toraldo, G.: Parallel computing in global optimization. In: Handbook of Parallel Computing and Statistics, pp. 225–258. Chapman & Hall, London (2006)

    Google Scholar 

  34. Facchinei, F., Pang, J.S.: Finite-Dimensional Variational Inequalities and Complementarity Problems. Volumes I and II. Springer, New York (2003)

    Google Scholar 

  35. Kinderlehrer, D., Stampacchia, G.: An Introduction to Variational Inequalities and Their Application. Academic Press, New York (1980)

    Google Scholar 

  36. Partt, W.K.: Digital Image Processing: PIKS Inside, 3rd edn. Wiley, New York (2001)

    Book  Google Scholar 

  37. Cai, J.F., Chan, R., Nikolova, M.: Two phase methods for deblurring images corrupted by impulse plus Gaussian noise. Inverse Probl. Imaging 2, 187–204 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  38. Hansen, P.C., Nagy, J.G., O’Leary, D.P.: Deblurring Images: Matrices, Spectra, and Filtering. SIAM, Philadelphia (2006)

    Book  Google Scholar 

  39. Huang, H., Haddad, A.: Adaptive median filters: new algorithms and results. IEEE Trans. Image Process. 4, 499–502 (1995)

    Article  Google Scholar 

Download references

Acknowledgements

We thank the anonymous referees for the useful comments, which helped us improve the paper greatly. The research is supported by the NSFC grants 11071122, 11171159, and Grant 20103207110002 from MOE of China.

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, Jiangsu Key Labratory for NSLSCS, Nanjing Normal University, Nanjing, 210023, China

    K. Wang, D. R. Han & L. L. Xu

Authors
  1. K. Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. D. R. Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. L. L. Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to D. R. Han.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, K., Han, D.R. & Xu, L.L. A Parallel Splitting Method for Separable Convex Programs. J Optim Theory Appl 159, 138–158 (2013). https://doi.org/10.1007/s10957-013-0277-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-013-0277-9

Keywords

Search

Navigation