Skip to main content
SpringerLink
Log in

A relaxed proximal ADMM method for block separable convex programming

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

The alternating direction method of multipliers (ADMM) is an efficient solution method for block separable convex programming. To enhance the robustness of the method, the proximal technique is introduced into the minimization subproblems of the ADMM which yields a proximal ADMM method. There are two factors involved in the proximal ADMM method, one is the linearization factor and the other is the relaxation factor. Due to the importance of the two factors in numerical acceleration, this paper considers the relaxation and the optimal choice of the two factors, and presents a relaxed proximal ADMM method for block separable convex programming. The global convergence of the method is established under weaker conditions. Preliminary numerical experiments are made to illustrate the efficiency of designed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Availability of supporting data

All data generated or analyzed during this study are included in this paper.

References

  1. Boyd, S., Vandenberghe, L.: Convex programming. Cambridge University Press, Cambridge (2004)

    Google Scholar 

  2. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, Jordan, M. (ed) in Chief, 3, pp 1-122. (2011)

  3. Lin, Z.C., Li, H., Fang, C.: Alternating Direction Method of Multipliers for Machine Learning, Springer, (2022)

  4. Hestenes, M.R.: Multiplier and gradient methods. J. Optim. Theory Appl. 4, 303–320 (1969)

    Article  MathSciNet  Google Scholar 

  5. Powell, M.J.D.: A method for nonlinear constraints in minimization problems, Optimization Fletcher, R. (ed.) New York: Academic Press, pp 283-298. (1969)

  6. Glowinski, R., Marrocco, A.: Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires. Mathematical Modelling and Numerical Analysis-Modélisation Mathématique et Analyse Numérique. 9(2), 41–76 (1975)

    Google Scholar 

  7. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via fnite-element approximations. Computers and Mathematics with Applications. 2, 17–40 (1976)

    Article  Google Scholar 

  8. Fortin, M., Glowinski, R.: On decomposition-coordination methods using an augmented Lagrangian. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Solution of Boundary Problems, pp. 97–146. Elsevier, Amsterdam (1983)

    Chapter  Google Scholar 

  9. He, B.S., Hou, L.S., Yuan, X.M.: On full Jacobian decomposition of the augmented Lagrangian method for separable convex programming. SIAM J. Optim. 25(4), 2274–2312 (2015)

    Article  MathSciNet  Google Scholar 

  10. Chen, C.H., He, B.S., Ye, Y.Y., Yuan, X.M.: The direct extension of ADMM for multi-blockconvex minimization problems is not necessarily convergent. Mathematical Programming. 155(1), 57–79 (2016)

    Article  MathSciNet  Google Scholar 

  11. He, B.S., Tao, M., Yuan, X.M.: Alternating direction method with Gaussian back substitution for separable convex programming. SIAM J. Optim. 22(2), 313–340 (2012)

    Article  MathSciNet  Google Scholar 

  12. He, B.S., Tao, M., Yuan, X.M.: A splitting method for separable convex programming. IMA J. Numer Anal. 35(1), 394–426 (2015)

    Article  MathSciNet  Google Scholar 

  13. He, B.S., Ma, F., Yuan, X.M.: Optimally linearizing the alternating direction method of multipliers for convex programming. Computational Optimization and Applications 75, 361–388 (2020)

    Article  MathSciNet  Google Scholar 

  14. He, B.S., Xu, M.H., Yuan, X.M.: Block-wise ADMM with a relaxation factor for multiple-block convex programming. Journal of Operational Resarch Society of China 6, 485–505 (2018)

    Article  MathSciNet  Google Scholar 

  15. He, B.S., Yuan, X.M.: On the optimal proximal parameter of an ADMM-like splitting method for separable convex programming. Mathematical Methods in Image Processing and Inverse Problems 360, 139–163 (2018)

    Article  MathSciNet  Google Scholar 

  16. He, B.S., Ma, F., Yuan, X.M.: Convergence study on the symmetric version of ADMM with larger step sizes. SIAM Journal on Imaging Sciences 9(3), 1467–1501 (2016)

    Article  MathSciNet  Google Scholar 

  17. He, B.S., Ma, F., Yuan, X.M.: Optimal proximal augmented Lagrangian method and its application to full Jacobian splitting for multi-block separable convex minimization problems. IMA Journal of Numerical Analysis 40(2), 1188–1216 (2020)

    Article  MathSciNet  Google Scholar 

  18. Chen, J., Wang, Y., He, H., Lv, Y.: Convergence analysis of positive-indefnite proximal ADMM with a Glowinski relaxation factor. Numberical Algorithms 83, 1415–1440 (2020)

    Article  Google Scholar 

  19. Ma, F.: Convergence study on the proximal alternating direction method with larger step size. Numerical Algorithms 85, 399–425 (2020)

    Article  MathSciNet  Google Scholar 

  20. Tao, M.: Convergence study of indefinite proximal ADMM with a relaxation factor. Computational Optimization and Applications 77(1), 91–123 (2021)

    Article  MathSciNet  Google Scholar 

  21. Candés, E., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM, 58(3): 1-37. (2011)

  22. Zhang, M., Huang, Z.H., Li, Y.F.: The sparsest solution to the system of absolute value equations. Journal of the Operations Research Society of China 3, 31–51 (2015)

  23. Tao, M., Yuan, X.M.: Recovering low-rank and sparse components of matrices from incomplete and noisy observations. SIAM J. Optim 21, 57–81 (2011)

    Article  MathSciNet  Google Scholar 

  24. Sun, M., Wang, Y.J., Liu, J.: Generalized Peaceman-Rachford splitting method for multiple-block separable convex programming with applications to robust PCA. Calcolo 54, 77–94 (2017)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors wish to give their sincere thanks to the editor and two anonymous referees for their valuable suggestions and helpful comments which help improve the quality of the correspondence significantly.

Funding

The research of the second author was supported by the Natural Science Foundation of China under grant no. 12071250.

Author information

Authors and Affiliations

  1. School of Mathematics and Statistics, Zaozhuang University, Zaozhuang, Shandong, 277100, China

    Min Sun

  2. School of Management Science, Qufu Normal University, Rizhao, Shandong, 276800, China

    Yiju Wang

Authors
  1. Min Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yiju Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The first author has written the first three sections. The second has refined these and accomplished the last two sections.

Corresponding author

Correspondence to Yiju Wang.

Ethics declarations

Ethical approval

Not applicable

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, M., Wang, Y. A relaxed proximal ADMM method for block separable convex programming. Numer Algor 95, 575–603 (2024). https://doi.org/10.1007/s11075-023-01582-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-023-01582-1

Keywords

Search

Navigation