Skip to main content
SpringerLink
Log in

OGM: Online gaussian graphical models on the fly

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Gaussian Graphical Model is widely used to understand the dependencies between variables from high-dimensional data and can enable a wide range of applications such as principal component analysis, discriminant analysis, and canonical analysis. With respect to the streaming nature of big data, we study a novel Online Gaussian Graphical Model (OGM) that can estimate the inverse covariance matrix over the high-dimensional streaming data, in this paper. Specifically, given a small number of samples to initialize the learning process, OGM first estimates a low-rank estimation of inverse covariance matrix; then, when each individual new sample arrives, it updates the estimation of inverse covariance matrix using a low-complexity updating rule, without using the past data and matrix inverse. The significant edges of Gaussian graphical models can be discovered through thresholding the inverse covariance matrices. Theoretical analysis shows the convergence rate of OGM to the true parameters is guaranteed under Bernstein-style with mild conditions. We evaluate OGM using extensive experiments. The evaluation results backup our theory.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Uhler C (2019) Gaussian graphical models: An algebraic and geometric perspective. Chapter in Handbook of Graphical Models

  2. Tony Cai T, Ren Z, Zhou HH (2016) Estimating structured high-dimensional covariance and precision matrices: Optimal rates and adaptive estimation. Electron J Stat 10(1):1–59

    MathSciNet  MATH  Google Scholar 

  3. Huntenburg J, Abraham A, Loula J, Liem F, Dadi K, Varoquaux G (2017) Loading and plotting of cortical surface representations in nilearn. Res Ideas Outcomes 3:e12342

    Article  Google Scholar 

  4. Xiong H, Cheng W, Bian J, Wenqing H u, Sun Z, Guo Z (2018) Dbsda: Lowering the bound of misclassification rate for sparse linear discriminant analysis via model debiasing. IEEE Trans Neural Netw Learn Syst 30(3):707–717

    Article  MathSciNet  Google Scholar 

  5. Bian J, Yang S, Xiong H, Wang L, Fu Y, Sun Z, Guo Z, Wang J (2020) Crledd: Regularized causalities learning for early detection of diseases using electronic health record (ehr) data. IEEE Transactions on Emerging Topics in Computational Intelligence

  6. Gaber MM, Zaslavsky A, Krishnaswamy S (2005) Mining data streams: a review. ACM Sigmod Record 34(2):18–26

    Article  Google Scholar 

  7. Yang Q, Wu X (2006) 10 challenging problems in data mining research. Int J Inform Technol Dec Making 5(04):597–604

    Article  Google Scholar 

  8. Johnstone IM (2001) On the distribution of the largest eigenvalue in principal components analysis. Annals stat. 295–327

  9. Lauritzen SL (1996) Graphical models, vol 17. Clarendon Press, Oxford

    MATH  Google Scholar 

  10. Jordan MI (1998) Learning in graphical models, vol 89. Springer Science & Business Media, Berlin

    Book  Google Scholar 

  11. Xiong H, Zhang J, Huang Y u, Leach K, Barnes LE (2017) Daehr: A discriminant analysis framework for electronic health record data and an application to early detection of mental health disorders. ACM Trans Int Syst Technol (TIST) 8(3):47

    Google Scholar 

  12. Bian J, Barnes L, Chen G, Xiong H (2017) Early detection of diseases using electronic health records data and covariance-regularized linear discriminant analysis. In: IEEE International conference on biomedical health informatics. IEEE

  13. Yang S, Xiong H, Kaibo X u, Wang L, Bian J, Sun Z (2021) Improving covariance-regularized discriminant analysis for ehr-based predictive analytics of diseases. Appl Intell 51(1):377– 395

    Article  Google Scholar 

  14. Cheng W, Shi Y u, Zhang X, Wang W (2016) Sparse regression models for unraveling group and individual associations in eqtl mapping. BMC bioinformatics 17(1):136

    Article  Google Scholar 

  15. Cheng W, Guo Z, Zhang X, Wang W (2016) Cgc: A flexible and robust approach to integrating co-regularized multi-domain graph for clustering. ACM Trans Know Discov Data (TKDD) 10(4):46

    Google Scholar 

  16. Huang JZ, Liu N, Pourahmadi M, Liu L (2006) Covariance matrix selection and estimation via penalised normal likelihood. Biometrika 85–98

  17. Friedman J, Hastie T, Tibshirani R (2008) Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9(3):432–441

    Article  Google Scholar 

  18. Fan J, Feng Y, Yichao W u (2009) Network exploration via the adaptive lasso and scad penalties. Annals Appl Stat 3(2):521

    Article  MathSciNet  Google Scholar 

  19. Ravikumar P, Wainwright MJ, Raskutti G, Yu B (2011) High-dimensional covariance estimation by minimizing l1-penalized log-determinant divergence. Electron J Stat 5:935–980

    Article  MathSciNet  Google Scholar 

  20. Cai T, Liu W, Xi L (2011) A constrained ℓ1 minimization approach to sparse precision matrix estimation. J Am Stat Assoc 106(494):594–607

    Article  MathSciNet  Google Scholar 

  21. Liu Q, Ihler AT (2011) Learning scale free networks by reweighted l1 regularization. In: AISTATS, pp 40–48

  22. Liu H, Han F, Zhang C-H (2012) Transelliptical graphical models. In: NIPS, pp 809–817

  23. Tony Cai T, Zhou HH (2012) Minimax estimation of large covariance matrices under l1 norm. Stat Sin 22(4):1319–1378

    MATH  Google Scholar 

  24. Xue L, Ma S, Zou H (2012) Positive-definite ℓ1-penalized estimation of large covariance matrices. J Am Stat Assoc 107(500):1480–1491

    Article  MathSciNet  Google Scholar 

  25. Liu H, Wang L, Zhao T (2014) Sparse covariance matrix estimation with eigenvalue constraints. J Comput Graph Stat 23(2):439–459

    Article  MathSciNet  Google Scholar 

  26. Jankova J, van de Geer S (2015) Confidence intervals for high-dimensional inverse covariance estimation. Electron J Stat 9(1):1205–1229

    Article  MathSciNet  Google Scholar 

  27. Vapnik VN (2000) The nature of statistical learning theory. Springer, New York

    Book  Google Scholar 

  28. Wang H (2012) Bayesian graphical lasso models and efficient posterior computation. Bayesian Anal 7(4):867–886

    Article  MathSciNet  Google Scholar 

  29. Park T, Casella G (2008) The bayesian lasso. J Am Stat Assoc 103(482):681–686

    Article  MathSciNet  Google Scholar 

  30. Liu W (2013) Gaussian graphical model estimation with false discovery rate control. Annals Stat 41(6):2948–2978

    Article  MathSciNet  Google Scholar 

  31. Tan KM, Wang Z, Liu H, Zhang T (2018) Sparse generalized eigenvalue problem: optimal statistical rates via truncated rayleigh flow. J R Stat Soc Series B Stat Methodol 80(5):1057

    Article  MathSciNet  Google Scholar 

  32. Bian J, Xiong H, Yanjie F u, Huan J, Guo Z (2020) Mp2sda: Multi-party parallelized sparse discriminant learning. ACM Trans Know Discov Data (TKDD) 14(3):1–22

    Article  Google Scholar 

  33. Kummerfeld E, Danks D (2013) Tracking time-varying graphical structure. In: Advances in neural information processing systems (NIPS), pp 1205–1213

  34. Kummerfeld E, Danks D (2012) Online learning of time-varying causal structures. In: UAI workshop on causal structure learning

  35. Cao X, Khare K, Ghosh M (2016) Posterior graph selection and estimation consistency for high-dimensional bayesian dag models. Ann Stat 47:318–348

    MathSciNet  MATH  Google Scholar 

  36. Xiang R, Khare K, Ghosh M (2015) High dimensional posterior convergence rates for decomposable graphical models. Electron J Stat 9:2828–2854

    Article  MathSciNet  Google Scholar 

  37. Meng D, Moore KL (2020) Contraction mapping-based robust convergence of iterative learning control with uncertain, locally lipschitz nonlinearity. IEEETrans Syst Man Cybern Syst 50(2):442–454

    Article  Google Scholar 

  38. Miller KS (1981) On the inverse of the sum of matrices. Math Mag 54(2):67–72

    Article  MathSciNet  Google Scholar 

  39. Janková J, van de Geer S (2017) Honest confidence regions and optimality in high-dimensional precision matrix estimation. Test 26(1):143–162

    Article  MathSciNet  Google Scholar 

  40. Rothman AJ, Bickel PJ, Levina E, Ji Z (2008) Sparse permutation invariant covariance estimation. Electron J Stat 2:494–515

    Article  MathSciNet  Google Scholar 

  41. Joel A et al (2015) Tropp an introduction to matrix concentration inequalities. Found Trends®; in Mach Learn 8(1-2):1–230

    Article  Google Scholar 

  42. Gavish M, Donoho DL (2014) The optimal hard threshold for singular values is 4 sqrt 3. IEEE Trans Inf Theory 60(8):5040–5053

    Article  Google Scholar 

  43. Cai T, Liu W (2011) A direct estimation approach to sparse linear discriminant analysis. J Am Stat Assoc 106(496):1566–1577

    Article  MathSciNet  Google Scholar 

  44. Koller D, Friedman N (2009) Probabilistic graphical models: principles and techniques. MIT Press, Massachusetts

    MATH  Google Scholar 

  45. Mohan K, Pearl J (2014) Graphical models for recovering probabilistic and causal queries from missing data. In: Advances in neural information processing systems, pp 1520–1528

  46. Pearl J (2011) The structural theory of causation. In: M~cKay Illari P, Russo F, Williamson J (eds) Causality in the Sciences, chapter 33. Clarendon Press, Oxford, pp 697–727

  47. Witten DM, Tibshirani R (2009) Covariance-regularized regression and classification for high dimensional problems. J R Stat Soc Ser B (Stat Methodol) 71(3):615–636

    Article  MathSciNet  Google Scholar 

  48. Cai TT, Ma Z, Wu Y (2013) Sparse pca: Optimal rates and adaptive estimation. Annals Stat 41(6):3074–3110

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Key Research and Development Program of China (2018YFE0126000), the National Natural Science Foundation of China (NSFC) (No. 61972050), the Beijing Natural Science Foundation (No. L191012) and the 111 Project (No. B08004). This work was done under the joint efforts between Baidu Research and Mininglamp Academy of Sciences on the topics of federated online advertising.

Author information

Authors and Affiliations

  1. School of Cyber Space Security, State key Laboratory of Switching and Networking, Beijing University of Posts and Telecommunications, Haidian, Beijing, China

    Sijia Yang & Licheng Wang

  2. Big Data Lab, Baidu Research, Baidu Inc., Haidian, Beijing, China

    Haoyi Xiong

  3. Department of Computer Science, Missouri University of Science and Technology, Rolla, Missouri, 95001, United States

    Yunchao Zhang & Yi Ling

  4. Mininglamp Academy of Sciences, Mininglamp Technology, Shanghai, China

    Kaibo Xu & Zeyi Sun

Authors
  1. Sijia Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haoyi Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yunchao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yi Ling
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Licheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kaibo Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Zeyi Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Licheng Wang or Zeyi Sun.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, S., Xiong, H., Zhang, Y. et al. OGM: Online gaussian graphical models on the fly. Appl Intell 52, 3103–3117 (2022). https://doi.org/10.1007/s10489-021-02563-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02563-4

Keywords

Search

Navigation