Abstract
In a previous paper (Huang et al., Advances in Computational Mathematics 47(5):1–34, 2021), we presented the fundamentals of a new hierarchical algorithm for computing the expectation of a N-dimensional function \(H(\mathbf {X})\) where \(\mathbf {X}\) satisfies the truncated multi-variate normal (TMVN) distribution. The algorithm assumes that \(H(\mathbf {X})\) is low-rank and the covariance matrix \(\Sigma\) and precision matrix \(A=\Sigma ^{-1}\) have low-rank blocks with low-dimensional features. Analysis and numerical results were presented when A is tridiagonal or given by the exponential model. In this paper, we first demonstrate how the hierarchical algorithm structure allows the simultaneous calculations of all the order M and less moments \(E(H(\mathbf {X})=X_1^{m_1}\cdots X_N^{m_N}|a_i<X_i<b_i, \; i=1,\ldots ,N)\), \(\sum _{i} m_i \le M\) using asymptotically optimal \(O(N^M)\) operations when \(M\ge 2\) and \(O(N\log (N))\) operations when \(M=1\). These \(O(N^M)\) moments are often required in the Expectation Maximization (EM) algorithms. We illustrate the algorithm ideas using the case when A is tridiagonal or the exponential model where the off-diagonal matrix block has rank \(K=1\) and number of effective variables \(P \le 2\) for each function associated with a hierarchical tree node. The smaller K and P values allow the use of existing FFT and Non-uniform FFT (NuFFT) solvers to accelerate the computation of the compressed features in the system. To handle cases with higher K and P values, we introduce the sparse grid technique aimed at problems with \(K+P \approx 5 \sim 20\). We present numerical results for computing both the moments and higher K and P values to demonstrate the accuracy and efficiency of the algorithms. Finally, we summarize our results and discuss the limitations and generalizations, in particular, our algorithm capability is limited by the availability of mathematical tools in higher dimensions. When \(K+P\) is greater than 20, as far as we know, there are no practical tools available for problems with 20 truly independent variables.
Similar content being viewed by others
Data availability
Open source software packages and simulation results are available upon request.
References
Bäck, J., Nobile, F., Tamellini, L., Tempone, R.: Stochastic spectral galerkin and collocation methods for pdes with random coefficients: a numerical comparison. In: Spectral and high order methods for partial differential equations, pp. 43–62. Springer (2011)
Barthelmann, V., Novak, E., Ritter, K.: High dimensional polynomial interpolation on sparse grids. Adv. Comput. Math. 12(4), 273–288 (2000)
Genton, M.G., Keyes, D.E., Turkiyyah, G.: Hierarchical decompositions for the computation of high-dimensional multivariate normal probabilities. J. Comput. Graph. Stat. 27(2), 268–277 (2018)
Gerstner, T., Griebel, M.: Numerical integration using sparse grids. Numer Algorithms 18(3), 209–232 (1998)
Hackbusch, W.: A sparse matrix arithmetic based on \(\mathcal{H}\)-matrices. Part I: introduction to \(\mathcal{H}\)-matrices. Computing 62(2), 89–108 (1999)
Hackbusch, W., Khoromskij, B.N.: A sparse \(\cal{H}\)-matrix arithmetic. Computing 64(1), 21–47 (2000)
Ho, K.L., Greengard, L.: A fast direct solver for structured linear systems by recursive skeletonization. SIAM J. Sci. Comput. 34(5), A2507–A2532 (2012)
Ho, K.L., Ying, L.: Hierarchical interpolative factorization for elliptic operators: differential equations. Commun. Pure Appl. Math. 69(8), 1415–1451 (2016)
Huang, J., Cao, J., Fang, F., Genton, M.G., Keyes, D.E., Turkiyyah, G.: An o(n) algorithm for computing expectation of n-dimensional truncated multi-variate normal distribution i: fundamentals. Adv. Comput. Math. 47(5), 1–34 (2021)
Nobile, F., Tempone, R., Webster, C.G.: A sparse grid stochastic collocation method for partial differential equations with random input data. SIAM J. Numer. Anal. 46(5), 2309–2345 (2008)
Pflüger, D.M.: Spatially adaptive sparse grids for high-dimensional problems. Journal of Complexity 26(5), 505–522 (2010)
Pflüger, D.M.: Sparse Grid Toolbox \(SG^{++}\). https://sgpp.sparsegrids.org/ (2022). Accessed 24 Oct 2022
Piazzola, C., Tamellini, L.: The Sparse Grids Matlab kit - a Matlab implementation of sparse grids for high-dimensional function approximation and uncertainty quantification. ArXiv. (2022). https://doi.org/10.48550/arXiv.2203.09314
Shen, J., Yu, H.: Efficient spectral sparse grid methods and applications to high-dimensional elliptic problems. SIAM J. Sci. Comput. 32(6), 3228–3250 (2010)
Smolyak, S.A.: Quadrature and interpolation formulas for tensor products of certain classes of functions. In: Doklady Akademii Nauk, vol. 148, pp. 1042–1045. Russian Academy of Sciences (1963)
Tamellini, L., Piazzola, C., Nobile, F., Sprungk, B., Porta, G., Guignard, D., Tesei, F.: Sparse Grid Matlab Kit. https://sites.google.com/view/sparse-grids-kit (2022). Accessed 24 Oct 2022
Udell, M., Townsend, A.: Why are big data matrices approximately low rank? SIAM J. Math. Data Sci 1(1), 144–160 (2019)
Xi, Y., Xia, J., Chan, R.: A fast randomized eigensolver with structured ldl factorization update. SIAM J. Matrix Anal. Appl. 35(3), 974–996 (2014)
Xia, J., Gu, M.: Robust approximate cholesky factorization of rank-structured symmetric positive definite matrices. SIAM J. Matrix Anal. Appl. 31(5), 2899–2920 (2010)
Funding
The work of J. Huang was supported by the NSF grant DMS-1821093 and DMS-2012451. Y. Wu was partially supported by the NSF grant DMS-1821171.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Additional information
Communicated by Zydrunas Gimbutas.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zheng, C., Tang, Z., Huang, J. et al. An O(N) algorithm for computing expectation of N-dimensional truncated multi-variate normal distribution II: computing moments and sparse grid acceleration. Adv Comput Math 48, 71 (2022). https://doi.org/10.1007/s10444-022-09988-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10444-022-09988-6
Keywords
- Exponential covariance model
- Sparse grids
- Hierarchical algorithm
- Low-dimensional structure
- Low-rank structure
- Truncated multi-variate normal distribution