Abstract
This article describes a fast direct solver (i.e., not iterative) for partial hierarchically semi-separable systems. This solver requires a storage of \(\mathcal O (N \log N)\) and has a computational complexity of \(\mathcal O (N \log N)\) arithmetic operations. The numerical benchmarks presented illustrate the method in the context of interpolation using radial basis functions. The key ingredients behind this fast solver are recursion, efficient low rank factorization using Chebyshev interpolation, and the Sherman–Morrison–Woodbury formula. The algorithm and the analysis are worked out in detail. The performance of the algorithm is illustrated for a variety of radial basis functions and target accuracies.
Similar content being viewed by others
References
Andrianakis, I., Challenor, P.: The effect of the nugget on gaussian process emulators of computer models. Comput. Stat. Data Anal. 56(12), 4215–4228 (2012)
Arnoldi, W.: The principle of minimized iterations in the solution of the matrix eigenvalue problem. Quart. Appl. Math. 9(1), 17–29 (1951)
Barnes, J., Hut, P.: A hierarchical \({\cal {O}}({N} \log {N})\) force-calculation algorithm. Nature 324(4), 446–449 (1986)
Baxter, B.: The interpolation theory of radial basis functions. ArXiv, preprint arXiv:1006.2443 (2010)
Beatson, R., Cherrie, J., Mouat, C.: Fast fitting of radial basis functions: methods based on preconditioned GMRES iteration. Adv. Comput. Math. 11(2), 253–270 (1999)
Beatson, R., Greengard, L.: A short course on fast multipole methods. Wavelets, Multilevel Methods Elliptic PDEs pp. 1–37 (1997)
Beatson, R., Newsam, G.: Fast evaluation of radial basis functions: I. Comput. Math. Appl. 24(12), 7–19 (1992)
Billings, S., Beatson, R., Newsam, G.: Interpolation of geophysical data using continuous global surfaces. Geophysics 67(6), 1810–1822 (2002)
Börm, S., Grasedyck, L., Hackbusch, W.: Hierarchical matrices. In: Lecture notes 21 (2005)
Buhmann, M.: Radial Basis Functions: Theory and Implementations, vol. 12. Cambridge University Press, Cambridge (2003)
Chandrasekaran, S., Dewilde, P., Gu, M., Pals, T., Sun, X., van der Veen, A., White, D.: Some fast algorithms for sequentially semiseparable representations. SIAM J. Matrix Anal. Appl. 27(2), 341–364 (2006)
Chandrasekaran, S., Gu, M., Pals, T.: A fast ULV decomposition solver for hierarchically semiseparable representations. SIAM J. Matrix Anal. Appl. 28(3), 603–622 (2006)
Chen, K.: An analysis of sparse approximate inverse preconditioners for boundary integral equations. SIAM J. Matrix Anal. Appl. 22, 1058 (2001)
Cheng, H., Gimbutas, Z., Martinsson, P., Rokhlin, V.: On the compression of low rank matrices. SIAM J. Sci. Comput. 26(4), 1389–1404 (2005)
Cheng, H., Greengard, L., Rokhlin, V.: A fast adaptive multipole algorithm in three dimensions. J. Comput. Phys. 155(2), 468–498 (1999)
Coifman, R., Rokhlin, V., Wandzura, S.: The fast multipole method for the wave equation: a pedestrian prescription. IEEE Antennas Propag. Mag. 35(3), 7–12 (1993)
Darve, E.: The fast multipole method: numerical implementation. J. Comput. Phys. 160(1), 195–240 (2000)
Darve, E.: The fast multipole method. I: error analysis and asymptotic complexity. SIAM J. Numer. Anal. 38(1), 98–128 (2000)
Davis, G., Morris, M.: Six factors which affect the condition number of matrices associated with kriging. Math. Geol. 29(5), 669–683 (1997)
De Boer, A., Van der Schoot, M., Bijl, H.: Mesh deformation based on radial basis function interpolation. Comput. Struct. 85(11–14), 784–795 (2007)
Dietrich, C., Newsam, G.: Efficient generation of conditional simulations by Chebyshev matrix polynomial approximations to the symmetric square root of the covariance matrix. Math. Geol. 27(2), 207–228 (1995)
Fong, W., Darve, E.: The black-box fast multipole method. J. Comput. Phys. 228(23), 8712–8725 (2009)
Freund, R.: A transpose-free quasi-minimal residual algorithm for non-Hermitian linear systems. SIAM J. Sci. Comput. 14, 470–482 (1993)
Freund, R., Nachtigal, N.: QMR: a quasi-minimal residual method for non-Hermitian linear systems. Numerische Mathematik 60(1), 315–339 (1991)
Frieze, A., Kannan, R., Vempala, S.: Fast Monte–Carlo algorithms for finding low-rank approximations. J. ACM (JACM) 51(6), 1025–1041 (2004)
Gillman, A., Young, P., Martinsson, P.: A direct solver with \({\cal {O}}(N)\) complexity for integral equations on one-dimensional domains. ArXiv, preprint arXiv:1105.5372 (2011)
Golub, G., Van Loan, C.: Matrix Computations, vol. 3. Johns Hopkins University Press, Baltimore (1996)
Goreinov, S., Tyrtyshnikov, E., Zamarashkin, N.: A theory of pseudoskeleton approximations. Linear Algebra Appl. 261(1–3), 1–21 (1997)
Grasedyck, L., Hackbusch, W.: Construction and arithmetics of \({\cal {H}}\)-matrices. Computing 70(4), 295–334 (2003)
Greengard, L., Gueyffier, D., Martinsson, P., Rokhlin, V.: Fast direct solvers for integral equations in complex three-dimensional domains. Acta Numerica 18(1), 243–275 (2009)
Greengard, L., Rokhlin, V.: A fast algorithm for particle simulations. J. Comput. Phys. 73(2), 325–348 (1987)
Greengard, L., Rokhlin, V.: A new version of the fast multipole method for the Laplace equation in three dimensions. Acta Numerica 6(1), 229–269 (1997)
Gu, M., Eisenstat, S.: Efficient algorithms for computing a strong rank-revealing QR factorization. SIAM J. Sci. Comput. 17(4), 848–869 (1996)
Guennebaud, G., Jacob, B., et al.: Eigen v3. http://eigen.tuxfamily.org (2010)
Gumerov, N., Duraiswami, R.: Fast radial basis function interpolation via preconditioned Krylov iteration. SIAM J. Sci. Comput. 29(5), 1876–1899 (2007)
Hackbusch, W.: A sparse matrix arithmetic based on \({\cal {H}}\)-matrices. Part I: introduction to \({\cal {H}}\)-matrices. Computing 62(2), 89–108 (1999)
Hackbusch, W., Börm, S.: Data-sparse approximation by adaptive \({\cal {H}}^2\)-matrices. Computing 69(1), 1–35 (2002)
Hackbusch, W., Khoromskij, B.: A sparse \(\cal {H}\)-matrix arithmetic. Computing 64(1), 21–47 (2000)
Hackbusch, W., Nowak, Z.: On the fast matrix multiplication in the boundary element method by panel clustering. Numerische Mathematik 54(4), 463–491 (1989)
Hager, W.: Updating the inverse of a matrix. SIAM Rev. 31(2), 221–239 (1989)
Hestenes, M., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49(6), 409–436 (1952)
Kong, W., Bremer, J., Rokhlin, V.: An adaptive fast direct solver for boundary integral equations in two dimensions. Appl. Comput. Harm. Anal. 31(3), 346–369 (2011)
Liberty, E., Woolfe, F., Martinsson, P., Rokhlin, V., Tygert, M.: Randomized algorithms for the low-rank approximation of matrices. Proc. Natl. Acad. Sci. 104, 20167–20172 (2007)
Martinsson, P.: A fast direct solver for a class of elliptic partial differential equations. J. Sci. Comput. 38(3), 316–330 (2009)
Martinsson, P., Rokhlin, V.: A fast direct solver for boundary integral equations in two dimensions. J. Comput. Phys. 205(1), 1–23 (2005)
Messner, M., Schanz, M., Darve, E.: Fast directional multilevel summation for oscillatory kernels based on Chebyshev interpolation. J. Comp. Phys. 231(4), 1175–1196 (2012)
Miranian, L., Gu, M.: Strong rank revealing LU factorizations. Linear Algebra Appl. 367, 1–16 (2003)
Nishimura, N.: Fast multipole accelerated boundary integral equation methods. Appl. Mech. Rev. 55(4), 299–324 (2002)
O’Dowd, R.: Conditioning of coefficient matrices of ordinary kriging. Math. Geol. 23(5), 721–739 (1991)
Paige, C., Saunders, M.: Solution of sparse indefinite systems of linear equations. SIAM J. Numer. Anal. 12(4), 617–629 (1975)
Pan, C.T.: On the existence and computation of rank-revealing LU factorizations. Linear Algebra Appl. 316(1), 199–222 (2000)
Rjasanow, S.: Adaptive cross approximation of dense matrices. In: IABEM 2002, International Association for Boundary Element Methods (2002)
Saad, Y., Schultz, M.: GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 7(3), 856–869 (1986)
Schaback, R.: Creating surfaces from scattered data using radial basis functions. Math. Methods Curves Surf. 477–496 (1995)
Schmitz, P., Ying, L.: A fast direct solver for elliptic problems on general meshes in 2D. J. Comput. Phys. 231(4), 1314–1338 (2012)
Schmitz, P., Ying, L.: A fast direct solver for elliptic problems on Cartesian meshes in 3D (in review) (2012)
Vandebril, R., Barel, M., Golub, G., Mastronardi, N.: A bibliography on semiseparable matrices. Calcolo 42(3), 249–270 (2005)
Vavasis, S.: Preconditioning for boundary integral equations. SIAM J. Matrix Anal. Appl. 13, 905–925 (1992)
Van der Vorst, H.: Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG for the solution of non-symmetric linear systems. SIAM J. Sci. Stat. Comput. 13(2), 631–644 (1992)
Wang, J., Liu, G.: A point interpolation meshless method based on radial basis functions. Int. J. Numer. Methods Eng. 54(11), 1623–1648 (2002)
Woodbury, M.A.: Inverting modified matrices. Statistical Research Group, Memo. Rep. no. 42, Princeton University, Princeton (1950)
Woolfe, F., Liberty, E., Rokhlin, V., Tygert, M.: A fast randomized algorithm for the approximation of matrices. Appl. Comput. Harmon. Anal. 25(3), 335–366 (2008)
Wu, Z., Schaback, R.: Local error estimates for radial basis function interpolation of scattered data. IMA J. Numer. Anal. 13(1), 13–27 (1993)
Xia, J., Chandrasekaran, S., Gu, M., Li, X.: Fast algorithms for hierarchically semiseparable matrices. Numer. Linear Algebra Appl. 17(6), 953–976 (2010)
Ying, L., Biros, G., Zorin, D.: A kernel-independent adaptive fast multipole algorithm in two and three dimensions. J. Comput. Phys. 196(2), 591–626 (2004)
Acknowledgments
Sivaram Ambikasaran would like to thank Krithika Narayanaswamy for proof reading the paper and helping in generating the figures.
Author information
Authors and Affiliations
Corresponding author
Additional information
The authors would like to thank the “Army High Performance Computing Research Center” (AHPCRC) and “The Global Climate and Energy Project” (GCEP) at Stanford for supporting the project.
Rights and permissions
About this article
Cite this article
Ambikasaran, S., Darve, E. An \(\mathcal O (N \log N)\) Fast Direct Solver for Partial Hierarchically Semi-Separable Matrices. J Sci Comput 57, 477–501 (2013). https://doi.org/10.1007/s10915-013-9714-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10915-013-9714-z
Keywords
- Fast direct solver
- Numerical linear algebra
- Partial hierarchically semi-separable representation
- Hierarchical matrix
- Radial basis function