Skip to main content
SpringerLink
Log in

Alternating Direction Method for Covariance Selection Models

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

The covariance selection problem captures many applications in various fields, and it has been well studied in the literature. Recently, an l 1-norm penalized log-likelihood model has been developed for the covariance selection problem, and this novel model is capable of completing the model selection and parameter estimation simultaneously. With the rapidly increasing magnitude of data, it is urged to consider efficient numerical algorithms for large-scale cases of the l 1-norm penalized log-likelihood model. For this purpose, this paper develops the alternating direction method (ADM). Some preliminary numerical results show that the ADM approach is very efficient for large-scale cases of the l 1-norm penalized log-likelihood model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Banerjee, O., El Ghaoui, L., d’Aspremont, A.: Model selection through sparse maximum likelihood estimation. J. Mach. Learn. Res. 9, 485–516 (2008)

    MathSciNet  MATH  Google Scholar 

  2. Bauschke, H.H., Combettes, P.L.: A weak-to-strong convergence principle for Fejér-monotone methods in Hilbert spaces. Math. Oper. Res. 26(2), 248–264 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bruckstein, A.M., Donoho, D.L., Elad, M.: From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev. 51(1), 34–81 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  4. Cox, D.R., Wermuth, N.: Multivariate Dependencies: Models, Analysis and Interpretation. Chapman and Hall, London (1996)

    MATH  Google Scholar 

  5. d’Aspremont, A., Banerjee, O., El Ghaoui, L.: First-order methods for sparse covariance selection. SIAM J. Matrix Anal. Appl. 30(1), 56–66 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  6. Dempster, A.: Covariance selection. Biometrics 28, 157–175 (1972)

    Article  Google Scholar 

  7. Eckstein, J., Fukushima, M.: Some reformulation and applications of the alternating directions method of multipliers. In: Hager, W.W., et al. (eds.) Large Scale Optimization: State of the Art, pp. 115–134. Kluwer Academic, Norwell (1994)

    Chapter  Google Scholar 

  8. Fazel, M., Hindi, H., Boyd, S.: A rank minimization heuristic with application to minimum order system approximation. Proc. Am. Control Conf. 6, 4734–4739 (2001)

    Google Scholar 

  9. Fukushima, M.: Application of the alternating direction method of multipliers to separable convex programming problems. Comput. Optim. Appl. 1, 93–111 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  10. Gabay, D.: Application of the method of multipliers to variational inequalities. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian methods: Application to the Numerical Solution of Boundary-Value Problem, pp. 299–331. North-Holland, Amsterdam (1983)

    Chapter  Google Scholar 

  11. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximations. Comput. Math. Appl. 2, 17–40 (1976)

    Article  MATH  Google Scholar 

  12. Glowinski, R., Le Tallec, P.: Augmented Lagrangian and Operator Splitting Methods in Nonlinear Mechanics. SIAM Studies in Applied Mathematics, (1989)

    Book  MATH  Google Scholar 

  13. Golub, G.H., van Loan, C.F.: Matrix Computation, 3rd edn. The Johns Hopkins University Press, Baltimore (1996)

    Google Scholar 

  14. He, B.S., Liao, L.Z., Han, D., Yang, H.: A new inexact alternating directions method for monotone variational inequalities. Math. Program. 92, 103–118 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  15. He, B.S., Yang, H.: Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities. Oper. Res. Lett. 23, 151–161 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  16. Hey, T., Tansley, S., Tolle, K.: The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research (2009)

  17. Hestenes, M.R.: Multiplier and gradient methods. J. Optim. Theory Appl. 4, 303–320 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  18. Hughes, T.R., Marton, M.J., Jones, A.R., Roberts, C.J., Stoughton, R., Armour, C.D., Bennett, H.A., Coffey, E., Dai, H., He, Y.D., Kidd, M.J., King, A.M., Meyer, M.R., Slade, D., Lum, P.Y., Stepaniants, S.B., Shoemaker, D.D., Gachotte, D., Chakraburtty, K., Simon, J., Bard, M., Friend, S.H.: Functional discovery via a compendium of expression profiles. Cell 102(1), 109–126 (2000)

    Article  Google Scholar 

  19. Kontogiorgis, S., Meyer, R.R.: A variable-penalty alternating directions method for convex optimization. Math. Program. 83, 29–53 (1998)

    MathSciNet  MATH  Google Scholar 

  20. Li, L., Toh, K.C.: An inexact interior point method for l 1-regularized sparse covariance selection. Math. Program. Comput. 2, 291–315 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  21. Lu, Z.: Smooth optimization approach for sparse covariance selection. SIAM J. Control Optim. 19(4), 1807–1827 (2009)

    MATH  Google Scholar 

  22. Natsoulis, G., Pearson, C.I., Gollub, J., Eynon, B.P., Ferng, J., Nair, R., Idury, R., Lee, M.D., Fielden, M.R., Brennan, R.J., Roter, A.H., Jarnagin, K.: The liver pharmacological and xenobiotic gene response repertoire. Mol. Syst. Biol. 175(4), 1–12 (2008)

    Google Scholar 

  23. Ng, M., Weiss, P.A., Yuan, X.M.: Solving constrained total-variation problems via alternating direction methods. SIAM J. Sci. Comput. 32(5), 2710–2736 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  24. Nesterov, Y.E.: A method for unconstrained convex minimization problem with the rate of convergence O(1/k 2). Dokl. Akad. Nauk SSSR 269, 543–547 (1983)

    MathSciNet  Google Scholar 

  25. Nesterov, Y.E.: Smooth minimization of nonsmooth functions. Math. Program. 103, 127–152 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  26. Nesterov, Y.E., Nemirovski, A.S.: Interior point Polynomial algorithms in Convex Programming: Theory and Applications. Society for Industrial and Applied Mathematics, Philadelphia (1994)

    Book  MATH  Google Scholar 

  27. Powell, M.J.D.: A method for nonlinear constraints in minimization problems. In: Fletcher, R. (ed.) Optimization, pp. 283–298. Academic Press, New York (1969)

    Google Scholar 

  28. Sturm, J.F.: Using SeDuMi 1.02, a Matlab toolbox for optimization over symmetric cones. Optim. Methods Softw. 11–12, 625–653 (1999)

    Article  MathSciNet  Google Scholar 

  29. Tseng, P.: Alternating projection-proximal methods for convex programming and variational inequalities. SIAM J. Control Optim. 7, 951–965 (1997)

    MathSciNet  MATH  Google Scholar 

  30. Tütüncü, R.H., Toh, K.C., Todd, M.J.: Solving semidefinite-quadratic-linear programs using SDPT3. Math. Program. 95, 189–217 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  31. Vandenberghe, L., Boyd, S., Wu, S.-P.: Determinant maximization with linear matrix inequality constraints. SIAM J. Matrix Anal. Appl. 19, 499–533 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  32. Wang, C.J., Sun, D.F., Toh, K.C.: Solving log-determinant optimization problems by a Newton-CG primal proximal point algorithm. SIAM J. Control Optim. 20, 2994–3013 (2010)

    MathSciNet  MATH  Google Scholar 

  33. Yang, J.-F., Zhang, Y.: Alternating direction method for L1 problems in compressive sensing. SIAM J. Sci. Comput., 33(1), 250–278 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  34. Ye, C.H., Yuan, X.M.: A descent method for structured monotone variational inequalities. Optim. Methods Softw. 22, 329–338 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  35. Yuan, M., Lin, Y.: Model selection and estimation in the Gaussian graphical model. Biometrika 94(1), 19–35 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  36. Yuan, Y.-X., Sun, W.Y.: Optimization Theory and Methods. Springer, Berlin (2006)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Hong Kong Baptist University, Hong Kong, China

    Xiaoming Yuan

Authors
  1. Xiaoming Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoming Yuan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yuan, X. Alternating Direction Method for Covariance Selection Models. J Sci Comput 51, 261–273 (2012). https://doi.org/10.1007/s10915-011-9507-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10915-011-9507-1

Keywords

Search

Navigation