Skip to main content

Advertisement

SpringerLink
Log in

Moment inequalities for sums of random matrices and their applications in optimization

  • Full Length Paper
  • Series A
  • Published:
Mathematical Programming Submit manuscript

Abstract

In this paper, we consider various moment inequalities for sums of random matrices—which are well-studied in the functional analysis and probability theory literature—and demonstrate how they can be used to obtain the best known performance guarantees for several problems in optimization. First, we show that the validity of a recent conjecture of Nemirovski is actually a direct consequence of the so-called non-commutative Khintchine’s inequality in functional analysis. Using this result, we show that an SDP-based algorithm of Nemirovski, which is developed for solving a class of quadratic optimization problems with orthogonality constraints, has a logarithmic approximation guarantee. This improves upon the polynomial approximation guarantee established earlier by Nemirovski. Furthermore, we obtain improved safe tractable approximations of a certain class of chance constrained linear matrix inequalities. Secondly, we consider a recent result of Delage and Ye on the so-called data-driven distributionally robust stochastic programming problem. One of the assumptions in the Delage–Ye result is that the underlying probability distribution has bounded support. However, using a suitable moment inequality, we show that the result in fact holds for a much larger class of probability distributions. Given the close connection between the behavior of sums of random matrices and the theoretical properties of various optimization problems, we expect that the moment inequalities discussed in this paper will find further applications in optimization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ai W., Zhang S.: Strong duality for the CDT subproblem: a necessary and sufficient condition. SIAM J. Optim. 19(4), 1735–1756 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  2. Alon N., Makarychev K., Makarychev Y., Naor A.: Quadratic forms on graphs. Invent. Math. 163(3), 499–522 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  3. Anstreicher K., Chen X., Wolkowicz H., Yuan Y.X.: Strong duality for a trust-region type relaxation of the quadratic assignment problem. Linear Algebra Appl. 301(1–3), 121–136 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  4. Anstreicher K., Wolkowicz H.: On Lagrangian relaxation of quadratic matrix constraints. SIAM J. Matrix Anal. Appl. 22(1), 41–55 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  5. Arora S., Lee J.R., Naor A.: Euclidean distortion and the parsest cut. J. Am. Math. Soc. 21(1), 1–21 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  6. Arora, S., Rao, S., Vazirani, U.: Expander flows, geometric embeddings and graph partitioning. J. ACM 56(2): Article 5 (2009)

  7. Barvinok A.I.: Problems of distance geometry and convex properties of quadratic maps. Discret. Comput. Geom. 13, 189–202 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  8. Beck A., Eldar Y.C.: Strong duality in nonconvex quadratic optimization with two quadratic constraints. SIAM J. Optim. 17(3), 844–860 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  9. Ben-Tal A., Nemirovski A.: On safe tractable approximations of chance-constrained linear matrix inequalities. Math. Oper. Res. 34(1), 1–25 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  10. Buchholz A.: Operator Khintchine inequality in non-commutative probability. Math. Ann. 319, 1–16 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  11. Delage, E., Ye, Y.: Distributionally robust optimization under moment uncertainty with application to data-driven problems. To appear in Oper. Res. (2009)

  12. Dupačová, J.: Stochastic programming: minimax approach. In: Floudas, C.A., Pardalos, P.M. Encyclopedia of Optimization, 2nd edn, Springer Science+Business Media, LLC, New York (2009)

  13. Goemans M.X.: Semidefinite programming in combinatorial optimization. Math. Program. 79, 143–161 (1997)

    MathSciNet  MATH  Google Scholar 

  14. Goemans M.X., Williamson D.P.: Improved approximation algorithms for Maximum Cut and satisfiability problems using semidefinite programming. J. ACM 42(6), 1115–1145 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  15. Gower J.C., Dijksterhuis G.B.: Procrustes Problems, Oxford Statistical Science Series, vol. 30. Oxford University Press, New York (2004)

    Google Scholar 

  16. Grötschel M., Lovász L., Schrijver A.: Geometric Algorithms and Combinatorial Optimization, Algorithms and Combinatorics, vol. 2, 2nd corrected edn. Springer, Berlin (1993)

    Google Scholar 

  17. Horn R.A., Johnson C.R.: Matrix Analysis. Cambridge University Press, Cambridge (1985)

    MATH  Google Scholar 

  18. Karger D., Motwani R., Sudan M.: Approximate graph coloring by semidefinite programming. J. ACM 45(2), 246–265 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  19. Khintchine A.: Über dyadische brüche. Math. Zeit. 23, 109–116 (1923)

    Article  MathSciNet  Google Scholar 

  20. Koopmans T.C., Beckmann M.: Assignment problems and the location of economic activities. Econometrica 25(1), 53–76 (1957)

    Article  MathSciNet  MATH  Google Scholar 

  21. Ledoux M., Talagrand M.: Probability in Banach Spaces: Isoperimetry and Processes, Ergebnisse der Mathematik und ihrer Grenzgebiete. 3. Folge/A Series of Modern Surveys in Mathematics, vol. 23. Springer, Berlin (1991)

    Google Scholar 

  22. Li, W.L., Zhang, Y.J., So, A.M.C., Win, M.Z.: Slow adaptive OFDMA through chance constrained programming. Preprint (2009)

  23. Luo Z.Q., Sidiropoulos N.D., Tseng P., Zhang S.: Approximation bounds for quadratic optimization with homogeneous quadratic constraints. SIAM J. Optim. 18(1), 1–28 (2007)

    MathSciNet  MATH  Google Scholar 

  24. Lust-Piquard F.: Inégalités de Khintchine dans C p (1 < p < ∞). Comptes Rendus de l’Académie des Sciences de Paris, Série I 303(7), 289–292 (1986)

    MathSciNet  MATH  Google Scholar 

  25. Nemirovski A.: Sums of random symmetric matrices and quadratic optimization under orthogonality constraints. Math. Program. Ser. B 109(2–3), 283–317 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  26. Nemirovski A., Roos C., Terlaky T.: On maximization of quadratic form over intersection of ellipsoids with common center. Math. Program. Ser. A 86, 463–473 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  27. Nemirovski A., Shapiro A.: Convex approximations of chance constrained programs. SIAM J. Optim. 17(4), 969–996 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  28. Nemirovski, A., Shapiro, A.: Scenario approximations of chance constraints. In: Calafiore, G., Dabbene, F. Probabilistic and Randomized Methods for Design under Uncertainty, pp. 3–47. Springer, London (2006)

  29. Nesterov, Yu.: Quality of semidefinite relaxation for nonconvex quadratic optimization. CORE Discussion Paper 9719, Université Catholique de Louvain, Belgium (1997)

  30. Pardalos, P.M., Wolkowicz, H. (eds.): Quadratic Assignment and Related Problems, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, vol. 16. American Mathematical Society, Providence, Rhode Island (1994)

  31. Pataki G.: On the rank of extreme matrices in semidefinite programs and the multiplicity of optimal eigenvalues. Math. Oper. Res. 23(2), 339–358 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  32. Peshkir G., Shiryaev A.N.: The Khintchine inequalities and martingale expanding sphere of their action. Russ. Math. Surv. 50(5), 849–904 (1995)

    Article  MathSciNet  Google Scholar 

  33. Pisier, G.: Non-commutative vector valued L p -spaces and completely p-summing maps. Astérisque, 247 (1998)

  34. Quine M.P.: A calculus-based proof of a Stirling formula for the gamma function. Int. J. Math. Educ. Sci. Technol. 28(6), 914–917 (1997)

    MathSciNet  Google Scholar 

  35. Scarf, H.: A min-max solution of an inventory problem. In: Arrow, K.J., Karlin, S., Scarf, H. Studies in the Mathematical Theory of Inventory and Production, pp. 201–209. Stanford University Press, Stanford (1958)

  36. Shapiro A.: Rank-reducibility of a symmetric matrix and sampling theory of minimum trace factor analysis. Psychometrika 47(2), 187–199 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  37. Shenouda, M.B., Davidson, T.N.: Outage-based designs for multi-user transceivers. In: Proceedings of the 2009 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2009), pp. 2389–2392 (2009)

  38. So, A.M.C.: On the performance of semidefinite relaxation MIMO detectors for QAM constellations. In: Proceedings of the 2009 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2009), pp. 2449–2452 (2009)

  39. So, A.M.C.: Probabilistic analysis of the semidefinite relaxation detector in digital communications. To appear in Proceedings of the 21st Annual ACM-SIAM Symposium on Discrete Algorithms (SODA 2010)

  40. So A.M.C., Ye Y., Zhang J.: A unified theorem on SDP rank reduction. Math. Oper. Res. 33(4), 910–920 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  41. So A.M.C., Zhang J., Ye Y.: On approximating complex quadratic optimization problems via semidefinite programming relaxations. Math. Program. Ser. B 110(1), 93–110 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  42. Tomczak-Jaegermann N.: The moduli of smoothness and convexity and the Rademacher averages of trace classes S p (1 ≤ p < ∞). Stud. Math. 50, 163–182 (1974)

    MathSciNet  MATH  Google Scholar 

  43. Tropp J.A.: The random paving property for uniformly bounded matrices. Stud. Math. 185(1), 67–82 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  44. Ye Y.: Approximating global quadratic optimization with convex quadratic constraints. J. Global Optim. 15(1), 1–17 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  45. Ye Y., Zhang S.: New results on quadratic minimization. SIAM J. Optim. 14(1), 245–267 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  46. Zhao Q., Karisch S.E., Rendl F., Wolkowicz H.: Semidefinite programming relaxations for the quadratic assignment problem. J. Comb. Optim. 2(1), 71–109 (1998)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong, Shatin, N. T., Hong Kong

    Anthony Man-Cho So

Authors
  1. Anthony Man-Cho So
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anthony Man-Cho So.

Additional information

A preliminary version of this paper has appeared in the Proceedings of the 20th Annual ACM-SIAM Symposium on Discrete Algorithms, 2009.

Rights and permissions

Reprints and permissions

About this article

Cite this article

So, A.MC. Moment inequalities for sums of random matrices and their applications in optimization. Math. Program. 130, 125–151 (2011). https://doi.org/10.1007/s10107-009-0330-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-009-0330-5

Keywords

Mathematics Subject Classification (2000)

Search

Navigation