Skip to main content
SpringerLink
Log in

Blessing of massive scale: spatial graphical model estimation with a total cardinality constraint approach

  • Full Length Paper
  • Series B
  • Published:
Mathematical Programming Submit manuscript

Abstract

We consider the problem of estimating high dimensional spatial graphical models with a total cardinality constraint (i.e., the \(\ell _0\)-constraint). Though this problem is highly nonconvex, we show that its primal-dual gap diminishes linearly with the dimensionality and provide a convex geometry justification of this “blessing of massive scale” phenomenon. Motivated by this result, we propose an efficient algorithm to solve the dual problem (which is concave) and prove that the solution achieves optimal statistical properties. Extensive numerical results are also provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Bertsekas, D.P.: Nonlinear Programming. Athena Scientific, Belmont (1999)

    MATH  Google Scholar 

  2. Bertsimas, D., King, A., Mazumder, R.: Best subset selection via a modern optimization lens. Ann. Stat. 44, 813–852 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  3. Cai, T., Liu, W., Luo, X.: A constrained \(\ell _1\) minimization approach to sparse precision matrix estimation. J. Am. Stat. Assoc. 106, 594–607 (2011)

    Article  MATH  Google Scholar 

  4. Cao, L., Fei-Fei, L.: Spatially coherent latent topic model for concurrent segmentation and classification of objects and scenes. In: IEEE 11th International Conference on Computer Vision, 2007. ICCV 2007. IEEE (2007)

  5. Fan, J., Feng, Y., Wu, Y.: Network exploration via the adaptive lasso and scad penalties. Ann Appl Stat 3, 521 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  6. Fan, Y., Lv, J.: Asymptotic equivalence of regularization methods in thresholded parameter space. J. Am. Stat. Assoc. 108, 1044–1061 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  7. Fan, Y., Lv, J.: Innovated scalable efficient estimation in ultra-large Gaussian graphical models. Ann. Stat. 44, 2098–2126 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  8. Hall, P., Jin, J.: Innovated higher criticism for detecting sparse signals in correlated noise. Ann. Stat. 38, 1686–1732 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  9. Howard, A., Matarić, M. J., Sukhatme, G. S.: Mobile sensor network deployment using potential fields: a distributed, scalable solution to the area coverage problem. In: Asama, H., Arai, T., Fukuda, T., Hasegawa, T. (eds.) Distributed Autonomous Robotic Systems, Vol. 5. Springer, pp. 299–308 (2002)

  10. Langendoen, K., Baggio, A., Visser, O.: Murphy loves potatoes: experiences from a pilot sensor network deployment in precision agriculture. In: Proceedings 20th IEEE International Parallel and Distributed Processing Symposium. IEEE (2006)

  11. Lee, S.H., Lee, S., Song, H., Lee, H.S.: Wireless sensor network design for tactical military applications: remote large-scale environments. In: Military Communications Conference, 2009. MILCOM 2009. IEEE. IEEE (2009)

  12. Liu, H., Wang, L.: Tiger: a tuning-insensitive approach for optimally estimating Gaussian graphical models. arXiv preprint arXiv:1209.2437 (2012)

  13. Liu, W.: Gaussian graphical model estimation with false discovery rate control. Ann Stat 41, 2948–2978 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  14. Magazine, M.J., Chern, M.-S.: A note on approximation schemes for multidimensional knapsack problems. Math. Oper. Res. 9, 244–247 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  15. Meinshausen, N., Bühlmann, P.: High-dimensional graphs and variable selection with the Lasso. Ann. Stat. 34, 1436–1462 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  16. Meinshausen, N., Bühlmann, P.: Stability selection. J. R. Stat. Soc. Ser. B Stat. Methodol. 72, 417–473 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  17. Meinshausen, N., Yu, B.: Lasso-type recovery of sparse representations for high-dimensional data. Ann. Stat. 37, 246–270 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  18. Optimization, G.: Inc.,“gurobi optimizer reference manual,” 2015. (2014). http://www.gurobi.com. Accessed 29 Sept 2018

  19. Pisinger, D.: A minimal algorithm for the multiple-choice knapsack problem. Eur. J. Oper. Res. 83, 394–410 (1995)

    Article  MATH  Google Scholar 

  20. Ravikumar, P., Wainwright, M.J., Raskutti, G., Yu, B.: High-dimensional covariance estimation by minimizing \(\ell _1\)-penalized log-determinant divergence. Electron. J. Stat. 5, 935–980 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  21. Ren, Z., Sun, T., Zhang, C.-H., Zhou, H.H.: Asymptotic normality and optimalities in estimation of large Gaussian graphical models. Ann. Stat. 43, 991–1026 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  22. Starr, R.M.: Quasi-equilibria in markets with non-convex preferences. Econometrica 37(1), 25–38 (1969)

  23. Sun, T., Zhang, C.-H.: Scaled sparse linear regression. Biometrika 99, 879–898 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  24. Williamson, D.P., Shmoys, D.B.: The Design of Approximation Algorithms. Cambridge University Press, Cambridge (2011)

    Book  MATH  Google Scholar 

  25. Yick, J., Mukherjee, B., Ghosal, D.: Wireless sensor network survey. Comput. Netw. 52, 2292–2330 (2008)

    Article  Google Scholar 

  26. Yuan, M., Lin, Y.: Model selection and estimation in the Gaussian graphical model. Biometrika 94, 19–35 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  27. Zhang, T.: On the consistency of feature selection using greedy least squares regression. J. Mach. Learn. Res. 10, 555–568 (2009)

    MathSciNet  MATH  Google Scholar 

  28. Zhang, Y., Wainwright, M.J., Jordan, M.I.: Lower bounds on the performance of polynomial-time algorithms for sparse linear regression. In: Proceedings of Annual Conference on Learning Theory (2014)

  29. Zhao, P., Yu, B.: On model selection consistency of Lasso. J. Mach. Learn. Res. 7, 2541–2563 (2006)

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, Department of Industrial and Manufacturing Engineering, Pennsylvania State University, University Park, PA, 16802, USA

    Ethan X. Fang

  2. Department of Operations Research and Financial Engineering, Princeton University, Princeton, NJ, 08544, USA

    Han Liu & Mengdi Wang

Authors
  1. Ethan X. Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Han Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mengdi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mengdi Wang.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 428 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fang, E.X., Liu, H. & Wang, M. Blessing of massive scale: spatial graphical model estimation with a total cardinality constraint approach. Math. Program. 176, 175–205 (2019). https://doi.org/10.1007/s10107-018-1331-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-018-1331-z

Mathematics Subject Classification

Search

Navigation