Skip to main content
Log in

An O(N) algorithm for computing expectation of N-dimensional truncated multi-variate normal distribution II: computing moments and sparse grid acceleration

  • Published:
Advances in Computational Mathematics Aims and scope Submit manuscript

Abstract

In a previous paper (Huang et al., Advances in Computational Mathematics 47(5):1–34, 2021), we presented the fundamentals of a new hierarchical algorithm for computing the expectation of a N-dimensional function \(H(\mathbf {X})\) where \(\mathbf {X}\) satisfies the truncated multi-variate normal (TMVN) distribution. The algorithm assumes that \(H(\mathbf {X})\) is low-rank and the covariance matrix \(\Sigma\) and precision matrix \(A=\Sigma ^{-1}\) have low-rank blocks with low-dimensional features. Analysis and numerical results were presented when A is tridiagonal or given by the exponential model. In this paper, we first demonstrate how the hierarchical algorithm structure allows the simultaneous calculations of all the order M and less moments \(E(H(\mathbf {X})=X_1^{m_1}\cdots X_N^{m_N}|a_i<X_i<b_i, \; i=1,\ldots ,N)\), \(\sum _{i} m_i \le M\) using asymptotically optimal \(O(N^M)\) operations when \(M\ge 2\) and \(O(N\log (N))\) operations when \(M=1\). These \(O(N^M)\) moments are often required in the Expectation Maximization (EM) algorithms. We illustrate the algorithm ideas using the case when A is tridiagonal or the exponential model where the off-diagonal matrix block has rank \(K=1\) and number of effective variables \(P \le 2\) for each function associated with a hierarchical tree node. The smaller K and P values allow the use of existing FFT and Non-uniform FFT (NuFFT) solvers to accelerate the computation of the compressed features in the system. To handle cases with higher K and P values, we introduce the sparse grid technique aimed at problems with \(K+P \approx 5 \sim 20\). We present numerical results for computing both the moments and higher K and P values to demonstrate the accuracy and efficiency of the algorithms. Finally, we summarize our results and discuss the limitations and generalizations, in particular, our algorithm capability is limited by the availability of mathematical tools in higher dimensions. When \(K+P\) is greater than 20, as far as we know, there are no practical tools available for problems with 20 truly independent variables.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data availability

Open source software packages and simulation results are available upon request.

References

  1. Bäck, J., Nobile, F., Tamellini, L., Tempone, R.: Stochastic spectral galerkin and collocation methods for pdes with random coefficients: a numerical comparison. In: Spectral and high order methods for partial differential equations, pp. 43–62. Springer (2011)

  2. Barthelmann, V., Novak, E., Ritter, K.: High dimensional polynomial interpolation on sparse grids. Adv. Comput. Math. 12(4), 273–288 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  3. Genton, M.G., Keyes, D.E., Turkiyyah, G.: Hierarchical decompositions for the computation of high-dimensional multivariate normal probabilities. J. Comput. Graph. Stat. 27(2), 268–277 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  4. Gerstner, T., Griebel, M.: Numerical integration using sparse grids. Numer Algorithms 18(3), 209–232 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  5. Hackbusch, W.: A sparse matrix arithmetic based on \(\mathcal{H}\)-matrices. Part I: introduction to \(\mathcal{H}\)-matrices. Computing 62(2), 89–108 (1999)

  6. Hackbusch, W., Khoromskij, B.N.: A sparse \(\cal{H}\)-matrix arithmetic. Computing 64(1), 21–47 (2000)

  7. Ho, K.L., Greengard, L.: A fast direct solver for structured linear systems by recursive skeletonization. SIAM J. Sci. Comput. 34(5), A2507–A2532 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  8. Ho, K.L., Ying, L.: Hierarchical interpolative factorization for elliptic operators: differential equations. Commun. Pure Appl. Math. 69(8), 1415–1451 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  9. Huang, J., Cao, J., Fang, F., Genton, M.G., Keyes, D.E., Turkiyyah, G.: An o(n) algorithm for computing expectation of n-dimensional truncated multi-variate normal distribution i: fundamentals. Adv. Comput. Math. 47(5), 1–34 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  10. Nobile, F., Tempone, R., Webster, C.G.: A sparse grid stochastic collocation method for partial differential equations with random input data. SIAM J. Numer. Anal. 46(5), 2309–2345 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  11. Pflüger, D.M.: Spatially adaptive sparse grids for high-dimensional problems. Journal of Complexity 26(5), 505–522 (2010)

    Article  MathSciNet  Google Scholar 

  12. Pflüger, D.M.: Sparse Grid Toolbox \(SG^{++}\)https://sgpp.sparsegrids.org/ (2022). Accessed 24 Oct 2022

  13. Piazzola, C., Tamellini, L.: The Sparse Grids Matlab kit - a Matlab implementation of sparse grids for high-dimensional function approximation and uncertainty quantification. ArXiv. (2022). https://doi.org/10.48550/arXiv.2203.09314

  14. Shen, J., Yu, H.: Efficient spectral sparse grid methods and applications to high-dimensional elliptic problems. SIAM J. Sci. Comput. 32(6), 3228–3250 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  15. Smolyak, S.A.: Quadrature and interpolation formulas for tensor products of certain classes of functions. In: Doklady Akademii Nauk, vol. 148, pp. 1042–1045. Russian Academy of Sciences (1963)

  16. Tamellini, L., Piazzola, C., Nobile, F., Sprungk, B., Porta, G., Guignard, D., Tesei, F.: Sparse Grid Matlab Kit. https://sites.google.com/view/sparse-grids-kit (2022). Accessed 24 Oct 2022

  17. Udell, M., Townsend, A.: Why are big data matrices approximately low rank? SIAM J. Math. Data Sci 1(1), 144–160 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  18. Xi, Y., Xia, J., Chan, R.: A fast randomized eigensolver with structured ldl factorization update. SIAM J. Matrix Anal. Appl. 35(3), 974–996 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  19. Xia, J., Gu, M.: Robust approximate cholesky factorization of rank-structured symmetric positive definite matrices. SIAM J. Matrix Anal. Appl. 31(5), 2899–2920 (2010)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Funding

The work of J. Huang was supported by the NSF grant DMS-1821093 and DMS-2012451. Y. Wu was partially supported by the NSF grant DMS-1821171.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jingfang Huang.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Communicated by Zydrunas Gimbutas.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zheng, C., Tang, Z., Huang, J. et al. An O(N) algorithm for computing expectation of N-dimensional truncated multi-variate normal distribution II: computing moments and sparse grid acceleration. Adv Comput Math 48, 71 (2022). https://doi.org/10.1007/s10444-022-09988-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10444-022-09988-6

Keywords

Mathematics Subject Classification (2010)

Navigation