Abstract
We consider the linearly constrained separable convex minimization problem, whose objective function consists of the sum of \(m\) individual convex functions in the absence of any coupling variables. While augmented Lagrangian-based decomposition methods have been well developed in the literature for solving such problems, a noteworthy requirement of these methods is that an additional correction step is a must to guarantee their convergence. This note shows that a straightforward Jacobian decomposition of the augmented Lagrangian method is globally convergent if the involved functions are further assumed to be strongly convex.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Auslender, A., Teboulle, M.: Lagrangian duality and related multiplier methods for variational inequality problems. SIAM J. Optim. 10, 1097–1115 (2000)
Auslender, A., Teboulle, M.: Entropic proximal decomposition methods for convex programs and variational inequalities. Math. Program. 91, 33–47 (2001)
Bardsley, J., Knepper, S., Nagy, J.: Structured linear algebra problems in adaptive optics imaging. Adv. Comput. Math. 35, 103–117 (2011)
Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, New York (2011)
Bose, N., Boo, K.: High-resolution image reconstruction with multisensors. Int. J. Imaging Syst. Technol. 9, 294–304 (1998)
Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2010)
Chen, C., He, B., Ye, Y., Yuan, X.: The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent. Math. Program. (2014). doi:10.1007/s10107-014-0826-5
Dinh, Q., Necoara, I., Savorgnan, C., Diehl, M.: An inexact perturbed path-following method for Lagrangian decomposition in large-scale separable convex optimization. SIAM J. Optim. 23, 95–125 (2013)
Eckstein, J.: Augmented Lagrangian and alternating direction methods for convex optimization: a tutorial and some illustrative computational results (2012). http://www.optimization-online.org/DB_FILE/2012/12/3704.pdf
Giannessi, F., Antonino, A.: Variational Inequalities and Network Equilibrium Problems. Plenum Press, New York (1995)
Han, D., He, H., Xu, L.: A proximal parallel splitting method for minimizing sum of convex functions with linear constraints. J. Comput. Appl. Math. 256, 36–51 (2014)
Han, D., Yuan, X.: A note on the alternating direction method of multipliers. J. Optim. Theory Appl. 155, 227–238 (2012)
Han, D., Yuan, X., Zhang, W.: An augmented-Lagrangian-based parallel splitting method for separable convex minimization with applications to image processing. Math. Comput. 83, 2263–2291 (2014)
He, B., Hou, L., Yuan, X.: On full Jacobian decomposition of the augmented Lagrangian method for separable convex programming (2013). http://www.optimization-online.org/DB_FILE/2013/05/3894.pdf
He, B., Tao, M., Xu, M., Yuan, X.: Alternating directions based contraction method for generally separable linearly constrained convex programming problems. Optimization 62, 573–596 (2013)
He, B., Tao, M., Yuan, X.: A splitting method for separate convex programming with linking linear constraints. IMA J. Numer. Anal. (2014). doi:10.1093/imanum/drt060
He, B., Tao, M., Yuan, X.: Alternating direction method with Gaussian-back substitution for separable convex programming. SIAM J. Optim. 22, 313–340 (2012)
He, B., Xu, H., Yuan, X.: On the proximal Jacobian decomposition of ALM for multiple-block separable convex minimization problems and its relationship to ADMM (2013). http://www.optimization-online.org/DB_FILE/2013/11/4142.pdf
He, H., Han, D.: A distributed Douglas-Rachford splitting method for multi-block convex minimization problems. Adv. in Computational Math. (2013). (submitted)
Hestenes, M.: Multiplier and gradient methods. J. Optim. Theory Appl. 4, 303–320 (1969)
Latorre, V., Gao, D.Y.: Canonical duality for solving general nonconvex constrained problems (2014, submitted). arXiv:1310.2014v1
Nedic, A., Ozdaglar, A.: Distributed subgradient methods for multi-agent optimization. IEEE Trans. Autom. Control 54, 48–61 (2009)
Nesterov, Y.: Introductory Lectures on Convex Optimization: Basic Course. Kluwer, Boston (2003)
Oreifej, O., Li, X., Shah, M.: Simultaneous video stabilization and moving object detection in turbulence. IEEE Trans. Pattern Anal. Mach. Intell. 35, 450–462 (2013)
Powell, M.: A method for nonlinear constraints in minimization problems. In: Fletcher, R. (ed.) Optimization, pp. 283–298. Academic Press, New York (1969)
Rockafekllar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976)
Setzer, S., Steidl, G., Tebuber, T.: Deblurring Poissonian images by split Bregman techniques. J. Vis. Commun. Image R. 21, 193–199 (2010)
Tao, M., Yuan, X.: Recovering low-rank and sparse components of matrices from incomplete and noisy observations. SIAM J. Optim. 21, 57–81 (2011)
Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., Knight, K.: Sparsity and smoothness via the fused LASSO. J. R. Stat. Soc. Ser. B 67, 91–108 (2005)
Wang, X., Hong, M., Ma, S., Luo, Z.: Solving multiple-block separable convex minimization problems using two-block alternating direction method of multipliers (2013, submitted). arXiv:1308.5294
Yang, J., Zhang, Y.: Alternating direction algorithms for \(\ell _1\)-problems in compressive sensing. SIAM J. Sci. Comput. 332, 250–278 (2011)
Acknowledgments
The authors would like to thank two anonymous referees and Professor Deren Han for their valuable comments on this paper, and in particular, one referee for bringing to our attention the relevant reference [21]. This research was supported by NSFC at Grant No. 11301123, the Zhejiang Provincial NSFC Grant No. LZ14A010003, and MOE (Singapore) Tier 1 Grant No. M4011083.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wang, K., Desai, J. & He, H. A note on augmented Lagrangian-based parallel splitting method. Optim Lett 9, 1199–1212 (2015). https://doi.org/10.1007/s11590-014-0825-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11590-014-0825-8