Skip to main content
SpringerLink
Log in

An efficient partial parallel method with scaling step size strategy for three-block convex optimization problems

  • Original Article
  • Published:
Mathematical Methods of Operations Research Aims and scope Submit manuscript

Abstract

A popular optimization model arising from image processing is the separable optimization problem whose objective function is the sum of three independent functions, and the variables are coupled by a linear equality. In this paper, we propose to solve such problem by a new partially parallel splitting method, whose step sizes for the primal and the dual variables in correction step are not necessarily identical. We establish the global convergence, and study the convergence rate on this varying ADMM-based prediction-correction method named as VAPCM. We derive the worst-case O(1/t) convergence rate in both the ergodic and non-ergodic senses. We also show that the convergence rate can be improved to o(1/t). Moreover, under the error bound assumptions, we establish the global linear convergence of VAPCM. We apply the new method to solve problems in robust principal component analysis and image decomposition. Numerical results indicate that the new method is efficient.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Bai JC, Li JC, Xu FM, Zhang HC (2018) Generalized symmetric ADMM for separable convex optimization. Comput Optim Appl 70(1):129–170

    Article  MathSciNet  MATH  Google Scholar 

  • Bauschke HH, Combettes PL (2017) Convex Analysis and Monotone Operator Theory in Hilbert Spaces. CMS Books in Mathematics

  • Bouwmans T, Aybat NS, Zahzah EH (2016) Handbook of Robust Low-Rank and Sparse Matrix Decomposition: Applications in Image and Video Processing. Chapman and Hall/CRC, UK

    Book  MATH  Google Scholar 

  • Cai XJ, Han DR, Yuan XM (2017) On the convergence of the direct extension of ADMM for three-block separable convex minimization models with one strongly convex function. Comput Optim Appl 66(1):39–73

    Article  MathSciNet  MATH  Google Scholar 

  • Candès EJ, Li XD, Ma Y, Wright J (2011) Robust principal component analysis? J ACM 58(3):1–37

    Article  MathSciNet  MATH  Google Scholar 

  • Cao CX, Han DR, Xu LL (2013) A new partial splitting augmented lagrangian method for minimizing the sum of three convex functions. Appl Math Comput 219(10):5449–5457

    MathSciNet  MATH  Google Scholar 

  • Chen CH, He BS, Ye YY, Yuan XM (2016) The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent. Math Program 155(1):57–79

    Article  MathSciNet  MATH  Google Scholar 

  • Chen CH, Shen Y, You YF (2013) On the convergence analysis of the alternating direction method of multipliers with three blocks. Abstr Appl Anal 2013:1–7

    MathSciNet  MATH  Google Scholar 

  • Deng W, Lai MJ, Peng ZM, Yin WT (2017) Parallel multi-block ADMM with o(1 / k) convergence. J Sci Comput 71(2):712–736

    Article  MathSciNet  MATH  Google Scholar 

  • Facchinei F, Pang JS (2003) Finite-Dimensional Variational Inequalities and Complementarity Problems, vol. I of Springer Series in Operations Research, New York

    MATH  Google Scholar 

  • Han DR, Kong WW, Zhang WX (2014) A partial splitting augmented Lagrangian method for low patch-rank image decomposition. Journal of Mathematical Imaging and Vision 51(1):145–160

    Article  MathSciNet  MATH  Google Scholar 

  • Han DR, Yuan XM (2012) A note on the alternating direction method of multipliers. J Optim Theory Appl 155(1):227–238

    Article  MathSciNet  MATH  Google Scholar 

  • Han DR, Yuan XM, Zhang WX (2014) An augmented Lagrangian based parallel splitting method for separable convex minimization with applications to image processing. Math Comput 83(289):2263–2291

    Article  MathSciNet  MATH  Google Scholar 

  • He BS (2009) Parallel splitting augmented Lagrangian methods for monotone structured variational inequalities. Comput Optim Appl 42(2):195–212

    Article  MathSciNet  MATH  Google Scholar 

  • He BS, Hou LS, Yuan XM (2015) On full jacobian decomposition of the augmented Lagrangian method for separable convex programming. SIAM J Optim 25(4):2274–2312

    Article  MathSciNet  MATH  Google Scholar 

  • He BS, Tao M, Xu MH, Yuan XM (2013) An alternating direction-based contraction method for linearly constrained separable convex programming problems. Optimization 62(4):573–596

    Article  MathSciNet  MATH  Google Scholar 

  • He BS, Tao M, Yuan XM (2012) Alternating direction method with Gaussian back substitution for separable convex programming. SIAM J Optim 22(2):313–340

    Article  MathSciNet  MATH  Google Scholar 

  • He BS, Tao M, Yuan XM (2017) Convergence rate analysis for the alternating direction method of multipliers with a substitution procedure for separable convex programming. Math Oper Res 42(3):662–691

    Article  MathSciNet  MATH  Google Scholar 

  • He BS, Yuan XM (2018) A class of ADMM-based algorithms for three-block separable convex programming. Comput Optim Appl 70(3):791–826

    Article  MathSciNet  MATH  Google Scholar 

  • Hou LS, He HJ, Yang JF (2016) A partially parallel splitting method for multiple-block separable convex programming with applications to robust PCA. Comput Optim Appl 63(1):273–303

    Article  MathSciNet  MATH  Google Scholar 

  • Jiang F, Wu ZM, Cai XJ (2018) Generalized ADMM with optimal indefinite proximal term for linearly constrained convex optimization. Journal of Industrial and Management Optimization 13(5):1–22

    Google Scholar 

  • Jiang SH, Li M (2018) A modified strictly contractive peaceman-rachford splitting method for multi-block separable convex programming. Journal of Industrial & Management Optimization 14(1):397–412

    Article  MathSciNet  MATH  Google Scholar 

  • Li M, Sun DF, Toh KC (2015) A convergent 3-block semi-proximal ADMM for convex minimization problems with one strongly convex block. Asia-Pacific J Oper Res 32(4):1550024

    Article  MathSciNet  MATH  Google Scholar 

  • Lin TY, Ma SQ, Zhang SZ (2015) On the sublinear convergence rate of multi-block ADMM. Journal of the Operations Research Society of China 3(3):251–274

    Article  MathSciNet  MATH  Google Scholar 

  • Ma SQ (2016) Alternating proximal gradient method for convex minimization. J Sci Comput 68(2):546–572

    Article  MathSciNet  MATH  Google Scholar 

  • Monteiro RDC, Svaiter BF (2013) Iteration-complexity of block-decomposition algorithms and the alternating direction method of multipliers. SIAM J Optim 23(1):475–507

    Article  MathSciNet  MATH  Google Scholar 

  • Peng YG, Ganesh A, Wright J, Xu WL, Ma Y (2012) RASL: Robust Alignment by Sparse and Low-rank decomposition for linearly correlated images. IEEE Trans Pattern Anal Mach Intell 34(11):2233–2246

    Article  Google Scholar 

  • Rockafellar RT (2015) Convex Analysis. Princeton University Press, Princeton

    Google Scholar 

  • Schaeffer H, Osher S (2013) A low patch-rank interpretation of texture. SIAM J Imag Sci 6(1):226–262

    Article  MathSciNet  MATH  Google Scholar 

  • Shen Y, Gao QM, Yin X (2022) A multi-parameter parallel ADMM for multi-block linearly constrained separable convex optimization. Appl Numer Math 171:369–388

    Article  MathSciNet  MATH  Google Scholar 

  • Shen Y, Zhang XY, Zhang XY (2021) A partial PPA block-wise ADMM for multi-block linearly constrained separable convex optimization. Optimization. A Journal of Mathematical Programming and Operations Research 70 3:631–657

    MathSciNet  MATH  Google Scholar 

  • Shen Y, Zuo YN, Yu AL (2021) A partially proximal S-ADMM for separable convex optimization with linear constraints. Appl Numer Math 160:65–83

    Article  MathSciNet  MATH  Google Scholar 

  • Sun DF, Toh KC, Yang LQ (2014) A convergent 3-block semiproximal alternating direction method of multipliers for conic programming with 4-type constraints. SIAM J Optim 25(2):882–915

    Article  MathSciNet  MATH  Google Scholar 

  • Sun HC, Liu J, Sun M (2017) A proximal fully parallel splitting method for stable principal component pursuit. Mathematical Problems in Engineering 2017, Article ID 9674528, 1-15

  • Tao M, Yuan XM (2011) Recovering low-rank and sparse components of matrices from incomplete and noisy observations. SIAM J Optim 21(1):57–81

    Article  MathSciNet  MATH  Google Scholar 

  • Wang JJ, Song W (2017) An algorithm twisted from generalized ADMM for multi-block separable convex minimization models. J Comput Appl Math 309:342–358

    Article  MathSciNet  MATH  Google Scholar 

  • Wang K, Desai J (2019) On the convergence rate of the augmented Lagrangian-based parallel splitting method. Optimization Methods and Software 34(2):278–304

    Article  MathSciNet  MATH  Google Scholar 

  • Wang K, Desai J, He HJ (2017) A proximal partially parallel splitting method for separable convex programs. Optimization Methods and Software 32(1):39–68

    Article  MathSciNet  MATH  Google Scholar 

  • Wang K, Han DR, Xu LL (2013) A parallel splitting method for separable convex programs. J Optim Theory Appl 159(1):138–158

    Article  MathSciNet  MATH  Google Scholar 

  • Wang XF (2013) Solving multiple-block separable convex minimization problems using two-block alternating direction method of multipliers. Pacific Journal of Optimization 11(4):645–667

    MathSciNet  MATH  Google Scholar 

  • Wu ZM, Liu FX, Li M (2019) A proximal peaceman-rachford splitting method for solving the multi-block separable convex minimization problems. Int J Comput Math 96(4):708–728

    Article  MathSciNet  MATH  Google Scholar 

  • Yang WH, Han DR (2016) Linear convergence of the alternating direction method of multipliers for a class of convex optimization problems. SIAM J Numer Anal 54(2):625–640

    Article  MathSciNet  MATH  Google Scholar 

  • Zheng XY, Ng KF (2014) Metric subregularity of piecewise linear multifunctions and applications to piecewise linear multiobjective optimization. SIAM J Optim 24(1):154–174

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Nanjing University, Nanjing, 210093, People’s Republic of China

    Yaning Jiang

  2. LMIB, School of Mathematical Sciences, Beihang University, Beijing, 100191, People’s Republic of China

    Deren Han

  3. Jiangsu Key Lab for NSLSCS, School of Mathematical Sciences, Nanjing Normal University, Nanjing, 210093, People’s Republic of China

    Xingju Cai

Authors
  1. Yaning Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Deren Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xingju Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xingju Cai.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

D. Han: This author was supported by the NSFC grant 11625105, 12131004.

X. Cai: This author was supported by the NSFC grant 11871279, 11571178.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, Y., Han, D. & Cai, X. An efficient partial parallel method with scaling step size strategy for three-block convex optimization problems. Math Meth Oper Res 96, 383–419 (2022). https://doi.org/10.1007/s00186-022-00796-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00186-022-00796-8

Keywords

Search

Navigation