Abstract
In this paper, we consider a first-order block-decomposition method for minimizing the sum of a convex differentiable function with Lipschitz continuous gradient, and two other proper closed convex (possibly, nonsmooth) functions with easily computable resolvents. The method presented contains two important ingredients from a computational point of view, namely: an adaptive choice of stepsize for performing an extragradient step; and the use of a scaling factor to balance the blocks. We then specialize the method to the context of conic semidefinite programming (SDP) problems consisting of two easy blocks of constraints. Without putting them in standard form, we show that four important classes of graph-related conic SDP problems automatically possess the above two-easy-block structure, namely: SDPs for \(\theta \)-functions and \(\theta _{+}\)-functions of graph stable set problems, and SDP relaxations of binary integer quadratic and frequency assignment problems. Finally, we present computational results on the aforementioned classes of SDPs showing that our method outperforms the three most competitive codes for large-scale conic semidefinite programs, namely: the boundary point (BP) method introduced by Povh et al., a Newton-CG augmented Lagrangian method, called SDPNAL, by Zhao et al., and a variant of the BP method, called the SPDAD method, by Wen et al.
Similar content being viewed by others
Notes
Downloaded in 2010 at http://www.math.nus.edu.sg/~mattohkc/SDPNAL.html.
References
Burachik, R.S., Iusem, A.N., Svaiter, B.F.: Enlargement of monotone operators with applications to variational inequalities. Set Valued Anal. 5, 159–180 (1997). doi:10.1023/A:1008615624787
Burachik, R.S., Svaiter, B.F.: Maximal monotone operators, convex functions and a special family of enlargements. Set Valued Anal. 10, 297–316 (2002). doi:10.1023/A:1020639314056
Burer, S., Monteiro, R.D.C., Zhang, Y.: A computational study of a gradient-based log-barrier algorithm for a class of large-scale SDPs. Math. Program. 95, 359–379 (2003). doi:10.1007/s10107-002-0353-7
Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40, 120–145 (2011). doi:10.1007/s10851-010-0251-1
Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles (2002). doi:10.1007/s101070100263
Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976). doi:10.1016/0898-1221(76)90003-1
Glowinski, R., Marrocco, A.: Sur l’approximation, par éléments finis d’ordre un, et la résolution, par penalisation-dualité, d’une classe de problèmes de dirichlet non linéaires. RAIRO Anal. Numér. 2, 41–76 (1975)
Lemaréchal, C.: Extensions diverses des méthodes de gradient et applications. Tech. rep., Thèse d’Etat, Université de Paris IX (1980)
Ma, S., Yin, W., Zhang, Y., Chakraborty, A.: An efficient algorithm for compressed mr imaging using total variation and wavelets. In: IEEE Conference on Computer Vision and Pattern Recognition, 2008. CVPR 2008, pp. 1–8 (2008). doi:10.1109/CVPR.2008.4587391
Malick, J., Povh, J., Rendl, F., Wiegele, A.: Regularization methods for semidefinite programming. SIAM J. Optim. 20(1), 336–356 (2009). doi:10.1137/070704575
Monteiro, R.C.D., Svaiter, B.F.: Iteration-complexity of block-decomposition algorithms and the alternating direction method of multipliers. SIAM J. Optim. 23(1), 475–507 (2013). doi:10.1137/110849468
Monteiro, R.D.C., Ortiz, C., Svaiter, B.F.: Implementation of a block-decomposition algorithm for solving large-scale conic semidefinite programming problems. Comput. Optim. Appl., pp. 1–25 (2013). doi:10.1007/s10589-013-9590-3
Monteiro, R.D.C., Svaiter, B.F.: On the complexity of the hybrid proximal extragradient method for the iterates and the ergodic mean. SIAM J. Optim. 20(6), 2755–2787 (2010). doi:10.1137/090753127
Monteiro, R.D.C., Svaiter, B.F.: Complexity of variants of Tseng’s modified F-B splitting and Korpelevich’s methods for hemivariational inequalities with applications to saddle-point and convex optimization problems. SIAM J. Optim. 21(4), 1688–1720 (2011). doi:10.1137/100801652
Povh, J., Rendl, F., Wiegele, A.: A boundary point method to solve semidefinite programs. Computing 78, 277–286 (2006). doi:10.1007/s00607-006-0182-2
Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)
Rockafellar, R.T.: On the maximal monotonicity of subdifferential mappings. Pac. J. Math. 33, 209–216 (1970)
Solodov, M.V., Svaiter, B.F.: A hybrid approximate extragradient-proximal point algorithm using the enlargement of a maximal monotone operator. Set Valued Anal. 7(4), 323–345 (1999)
Svaiter, B.F.: A family of enlargements of maximal monotone operators. Set Valued Anal. 8, 311–328 (2000). doi:10.1023/A:1026555124541
Toh, K.C., Todd, M., Tütüncü, R.H.: Sdpt3—a matlab software package for semidefinite programming. Optim. Methods Softw. 11, 545–581 (1999)
Wen, Z., Goldfarb, D., Yin, W.: Alternating direction augmented lagrangian methods for semidefinite programming. Math. Program. Comput. 2, 203–230 (2010). doi:10.1007/s12532-010-0017-1
Zhao, X.Y., Sun, D., Toh, K.C.: A Newton-CG augmented lagrangian method for semidefinite programming. SIAM J. Optim. 20(4), 1737–1765 (2010). doi:10.1137/080718206
Author information
Authors and Affiliations
Corresponding author
Additional information
The work of R. D. C. Monteiro was partially supported by NSF Grants CCF-0808863, CMMI-0900094 and CMMI- 1300221, and ONR Grant ONR N00014-11-1-0062.
The work of B. F. Svaiter was partially supported by CNPq Grants No. 303583/2008-8, 302962/2011-5, 480101/2008-6, 474944/2010-7, FAPERJ Grants E-26/102.821/2008, E-26/102.940/2011.
Appendix: Ergodic convergence results
Appendix: Ergodic convergence results
This appendix derives an ergodic iteration-complexity bound for Algorithm 1.
We start by stating the weak transportation formula for the \(\varepsilon \)-subdifferential.
Proposition 10.1
(Proposition 1.2.10 in [8]) Suppose that \(f:{\mathcal {Z}}\rightrightarrows {[-\infty ,\infty ]}\) is a closed proper convex function. Let \(z^{i},v^{i}\in {\mathcal {Z}}\) and \(\varepsilon _{i},\alpha _{i}\in {\mathbb {R}}_{+}\), for \(i=1,\ldots ,k\), be such that
and define
Then, \(\varepsilon _{a}\ge 0\) and \(v_{a}\in \partial _{\varepsilon _{a}}f(z_{a})\).
Theorem 10.2
Consider the sequences \(\{(x^{k},y^{k})\}, \{({\tilde{x}}^{k},{\tilde{y}}^{k})\}, \{(v_{1}^{k},v_{2}^{k})\}\) and \(\{\varepsilon _{k}\}\) generated by Algorithm 1, and the sequences \(\{c^{k}\}\) and \(\{d^{k}\}\) defined in (26). For every \(k\in {{\mathbb {N}}}\), define
and
Then, for every \(k\in {{\mathbb {N}}}\),
and
where \(d_{x,0}\) and \(d_{y,0}\) are defined in (31).
Proof
Let \(k\in {\mathbb {N}}\) be given. Note that by (35) and the definition of \(\langle {\cdot },{\cdot }\rangle _{\theta }\), we have
Then, in view of Lemma 4.2 and Theorem 2.4 in [12], we have
Hence, it follows from the above relations, Lemma 4.2(d) and the fact that \(\lambda _{k}\ge {\tilde{\lambda }}\), that
Using the definition of \(\Vert (\cdot ,\cdot )\Vert _{\theta ,1}\), (30) and the definition of \({\tilde{\lambda }}\) in (22), we easily see that the above two inequalities imply (63) and (64). Now, (28), (29), (35), (61) and Proposition 10.1 imply that
and hence that
The above four inclusions are easily seen to imply (62). \(\square \)
Rights and permissions
About this article
Cite this article
Monteiro, R.D.C., Ortiz, C. & Svaiter, B.F. A first-order block-decomposition method for solving two-easy-block structured semidefinite programs. Math. Prog. Comp. 6, 103–150 (2014). https://doi.org/10.1007/s12532-013-0062-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12532-013-0062-7
Keywords
- Complexity
- Proximal
- Extragradient
- Block-decomposition
- Convex optimization
- Conic optimization
- Semidefinite programming