Abstract
In this paper, we define a class of linear conic programming (which we call matrix cone programming or MCP) involving the epigraphs of five commonly used matrix norms and the well studied symmetric cone. MCP has recently been found to have many important applications, for example, in nuclear norm relaxations of affine rank minimization problems. In order to make the defined MCP tractable and meaningful, we must first understand the structure of these epigraphs. So far, only the epigraph of the Frobenius matrix norm, which can be regarded as a second order cone, has been well studied. Here, we take an initial step to study several important properties, including its closed form solution, calm Bouligand-differentiability and strong semismoothness, of the metric projection operator over the epigraph of the \(l_1,\,l_\infty \), spectral or operator, and nuclear matrix norm, respectively. These properties make it possible to apply augmented Lagrangian methods, which have recently received a great deal of interests due to their high efficiency in solving large scale semidefinite programming, to this class of MCP problems. The work done in this paper is far from comprehensive. Rather it is intended as a starting point to call for more insightful research on MCP so that it can serve as a basic tool to solve more challenging convex matrix optimization problems in years to come.
Similar content being viewed by others
References
Bhatia, R.: Matrix Analysis. Springer, New York (1997)
Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9, 717–772 (2008)
Candès, E.J., Tao, T.: The power of convex relaxation: Near-optimal matrix completion. IEEE Trans. Inf. Theory 56, 2053–2080 (2009)
Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM. 58, article No. 11 (2011)
Chandrasekaran, V., Sanghavi, S., Parrilo, P.A., Willsky, A.: Rank-sparsity incoherence for matrix decomposition. SIAM J. Optim. 21, 572–596 (2011)
Chen, X., Tseng, P.: Non-interior continuation methods for solving semidefinite complementarity problems. Math. Program. 95, 431–474 (2003)
Chen, X., Qi, H.D., Tseng, P.: Analysis of nonsmooth symmetric-matrix-valued functions with applications to semidefinite complement problems. SIAM J. Optim. 13, 960–985 (2003)
Chen, X.D., Sun, D.F., Sun, J.: Complementarity functions and numerical experiments for second-order-cone complementarity problems. Comput. Optim. Appl. 25, 39–56 (2003)
Chu, M., Funderlic, R., Plemmons, R.: Structured low rank approximation. Linear Algebra Appl. 366, 157–172 (2003)
Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)
Donoghue, W.F.: Monotone Matrix Functions and Analytic Continuation. Springer, New York (1974)
Fan, K.: On a theorem of Weyl concerning eigenvalues of linear transformations. Proc. Natl. Acad. Sci. USA 35, 652–655 (1949)
Faraut, J., Korányi, A.: Analysis on Symmetric Cones. Clarendon Press, Oxford (1994)
Gao, Y., Sun, D.F.: A majorized penalty approach for calibrating rank constrained correlation matrix problems. Preprint available at http://www.math.nus.edu.sg/~matsundf/MajorPen.pdf
Greenbaum, A., Trefethen, L.N.: GMRES/CR and Arnoldi/Lanczos as matrix approximation problems. SIAM J. Sci. Comput. 15, 359–368 (1994)
Gross, D.: Recovering low-rank matrices from few coefficients in any basis. IEEE Trans. Inf. Theory 57, 1548–1566 (2011)
Güler, O.: Hyperbolic polynomials and interior point methods for convex programming. Math. Oper. Res. 22, 350–377 (1997)
Haraux, A.: How to differentiate the projection on a convex set in Hilbert space. Some applications to variational inequalities. J. Math. Soc. Jpn 29, 615–631 (1977)
Hardy, G.H., Littlewood, J.E., Pólya, G.: Inequalities, 2nd edn. University Press, Cambridge (1952)
Higham, N.J.: Computing a nearest symmetric positive semidefinite matrix. Linear Algebra Appl. 103, 103–118 (1988)
Horn, R.A., Johnson, C.R.: Matrix Analysis. Cambridge University Press, Cambridge (1985)
Horn, R.A., Johnson, C.R.: Topics in Matrix Analysis. Cambridge University Press, Cambridge (1991)
Keshavan, R.H., Montanari, A., Oh, S.: Matrix completion from a few entries. IEEE Trans. Inf. Theory 56, 2980–2998 (2010)
Korányi, A.: Monotone functions on formally real Jordan algebras. Mathematische Annalen 269, 73–76 (1984)
Lancaster, P.: On eigenvalues of matrices dependent on a parameter. Numerische Mathematik 6, 377–387 (1964)
Lewis, A.S., Sendov, H.S.: Twice differentiable spectral functions. SIAM J. Matrix Anal. Appl. 23, 368–386 (2001)
Lewis, A.S., Sendov, H.S.: Nonsmooth analysis of singular values. Part II: Applications. Set Valued Anal. 13, 243–264 (2005)
Liu, G., Lin, Z., Yu, Y.: Robust subspace segmentation by low-rank representation. In: Proceedings of the 26th International Conference on Machine Learning. Haifa, Israel (2010)
Löwner, K.: Über monotone matrixfunktionen. Mathematische Zeitschrift 38, 177–216 (1934)
Malick, J., Povh, J., Rendl, F., Wiegele, A.: Regularization methods for semidefinite programming. SIAM J. Optim. 20, 336–356 (2009)
Moreau, J.J.: Décomposition orthogonale d’un espace hilbertien selon deux cones mutuellement polaires. Comptes Rendus de l’Académie des Sciences 255, 238–240 (1962)
von Neumann, J.: Some matrix-inequalities and metrization of matric-space. Tomsk University Review 1, 286–300 (1937). In: Collected Works. Pergamon, Oxford, 1962, Volume IV, 205–218
Pang, J.S.: Newton’s method for B-differentiable equations. Math. Oper. Res. 15, 149–160 (1990)
Povh, J., Rendl, F., Wiegele, A.: A boundary point method to solve semidefinite programs. Computing 78, 277–286 (2006)
Qi, L.: Convergence analysis of some algorithms for solving nonsmooth equations. Math. Oper. Res. 18, 227–244 (1993)
Recht, B.: A Simpler approach to matrix completion. J. Mach. Learn. Res. 12, 3413–3430 (2011)
Recht, B., Fazel, M., Parrilo, P.A.: Guaranteed minimum rank solutions to linear matrix equations via nuclear norm minimization. SIAM Rev. 52, 471–501 (2010)
Rockafellar, R.T.: Convex Analyis. Princeton University Press, Princeton (1970)
Rockafellar, R.T.: Conjugate Duality and Optimization. SIAM, Philadelphia (1974)
Scholtes, S.: Introduction to Piecewise Differentiable Equations. Habilitation Thesis, Institut für Statistik und Mathematische Wirtschaftstheorie, Universit”at Karlsruhe (1994)
Schwertman, N.C., Allen, D.M.: Smoothing an indefinite variance-covariance matrix. J. Stat. Comput. Simul. 9, 183–194 (1979)
Shapiro, A.: On differentiability of symmetric matrix valued functions. Optimization, Online (2002)
Stewart, G.W., Sun, J.G.: Matrix Perturbation Theory. Academic Press, New York (1990)
Sun, D.F.: Algorithms and Convergence Analysis for Nonsmooth Optimization and Nonsmooth Equations. PhD Thesis, Institute of Applied Mathematics, Chinese Academy of Sciences, China (1994)
Sun, D.F., Sun, J.: Semismooth matrix-valued functions. Math. Oper. Res. 27, 150–169 (2002)
Sun, D.F., Sun, J.: Strong semismoothness of eigenvalues of symmetric matrices and its applications in inverse eigenvalue problems. SIAM J. Numer. Anal. 40, 2352–2367 (2003)
Sun, D.F., Sun, J.: Löwner’s operator and spectral functions in Euclidean Jordan algebras. Math. Oper. Res. 33, 421–445 (2008)
Toh, K.C.: GMRES vs. ideal GMRES. SIAM J. Matrix Anal. Appl. 18, 30–36 (1997)
Toh, K.C., Trefethen, L.N.: The Chebyshev polynomials of a matrix. SIAM J. Matrix Anal. Appl. 20, 400–419 (1998)
Torki, M.: Second-order directional derivatives of all eigenvalues of a symmetric matrix. Nonlinear Anal. Ser. A Theory Methods 46, 1133–1150 (2001)
Tseng, P.: Merit functions for semi-definite complementarity problems. Math. Program. 83, 159–185 (1998)
Tsing, N.K., Fan, M.K.H., Verriest, E.I.: On analyticity of functions involving eigenvalues. Linear Algebra Appl. 207, 159–180 (1994)
Vandenberghe, L., Boyd, S.: Semidefinite programming. SIAM Rev. 38, 49–95 (1996)
Wen, Z., Goldfarb, D., Yin, W.: Alternating Direction Augmented Lagrangian Methods for Semidefinite Programming. Rice University CAAM Technical, Report TR09-42, 2009 (1996)
Wright, J., Ma, Y., Ganesh, A., Rao, S.: Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization. In: Bengio, Y., Schuurmans, D., Lafferty, J., Williams, C. (eds.) Advances in Neural Information Processing Systems, vol. 22 (2009)
Zarantonello, E.H.: Projections on convex sets in Hilbert space and spectral theory I and II. In: Zarantonello, E.H. (ed.) Contributions to Nonlinear Functional Analysis, pp. 237–424. Academic Press, New York (1971)
Zhao, X.Y.: A Semismooth Newton-CG Augmented Lagrangian Method for Large Scale Linear and Convex Quadratic SDPs. PhD thesis, Department of Mathematics, National University of Singapore (2009)
Zhao, X.Y., Sun, D.F., Toh, K.C.: A Newton-CG augmented Lagrangian method for semidefinite programming. SIAM J. Optim. 20, 1737–1765 (2010)
Acknowledgments
We wish to thank the anonymous referees and the Associate Editor for helpful comments that led to an improved version of the original submission.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was done while C. Ding was with Department of Mathematics, National University of Singapore, Republic of Singapore.
Rights and permissions
About this article
Cite this article
Ding, C., Sun, D. & Toh, KC. An introduction to a class of matrix cone programming. Math. Program. 144, 141–179 (2014). https://doi.org/10.1007/s10107-012-0619-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-012-0619-7