Skip to main content
SpringerLink
Log in

An \(\mathcal O (N \log N)\)  Fast Direct Solver for Partial Hierarchically Semi-Separable Matrices

With Application to Radial Basis Function Interpolation

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

This article describes a fast direct solver (i.e., not iterative) for partial hierarchically semi-separable systems. This solver requires a storage of \(\mathcal O (N \log N)\) and has a computational complexity of \(\mathcal O (N \log N)\) arithmetic operations. The numerical benchmarks presented illustrate the method in the context of interpolation using radial basis functions. The key ingredients behind this fast solver are recursion, efficient low rank factorization using Chebyshev interpolation, and the Sherman–Morrison–Woodbury formula. The algorithm and the analysis are worked out in detail. The performance of the algorithm is illustrated for a variety of radial basis functions and target accuracies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Andrianakis, I., Challenor, P.: The effect of the nugget on gaussian process emulators of computer models. Comput. Stat. Data Anal. 56(12), 4215–4228 (2012)

    Google Scholar 

  2. Arnoldi, W.: The principle of minimized iterations in the solution of the matrix eigenvalue problem. Quart. Appl. Math. 9(1), 17–29 (1951)

    MathSciNet  MATH  Google Scholar 

  3. Barnes, J., Hut, P.: A hierarchical \({\cal {O}}({N} \log {N})\) force-calculation algorithm. Nature 324(4), 446–449 (1986)

    Google Scholar 

  4. Baxter, B.: The interpolation theory of radial basis functions. ArXiv, preprint arXiv:1006.2443 (2010)

  5. Beatson, R., Cherrie, J., Mouat, C.: Fast fitting of radial basis functions: methods based on preconditioned GMRES iteration. Adv. Comput. Math. 11(2), 253–270 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  6. Beatson, R., Greengard, L.: A short course on fast multipole methods. Wavelets, Multilevel Methods Elliptic PDEs pp. 1–37 (1997)

  7. Beatson, R., Newsam, G.: Fast evaluation of radial basis functions: I. Comput. Math. Appl. 24(12), 7–19 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  8. Billings, S., Beatson, R., Newsam, G.: Interpolation of geophysical data using continuous global surfaces. Geophysics 67(6), 1810–1822 (2002)

    Google Scholar 

  9. Börm, S., Grasedyck, L., Hackbusch, W.: Hierarchical matrices. In: Lecture notes 21 (2005)

  10. Buhmann, M.: Radial Basis Functions: Theory and Implementations, vol. 12. Cambridge University Press, Cambridge (2003)

    Book  Google Scholar 

  11. Chandrasekaran, S., Dewilde, P., Gu, M., Pals, T., Sun, X., van der Veen, A., White, D.: Some fast algorithms for sequentially semiseparable representations. SIAM J. Matrix Anal. Appl. 27(2), 341–364 (2006)

    Google Scholar 

  12. Chandrasekaran, S., Gu, M., Pals, T.: A fast ULV decomposition solver for hierarchically semiseparable representations. SIAM J. Matrix Anal. Appl. 28(3), 603–622 (2006)

    Google Scholar 

  13. Chen, K.: An analysis of sparse approximate inverse preconditioners for boundary integral equations. SIAM J. Matrix Anal. Appl. 22, 1058 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  14. Cheng, H., Gimbutas, Z., Martinsson, P., Rokhlin, V.: On the compression of low rank matrices. SIAM J. Sci. Comput. 26(4), 1389–1404 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  15. Cheng, H., Greengard, L., Rokhlin, V.: A fast adaptive multipole algorithm in three dimensions. J. Comput. Phys. 155(2), 468–498 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  16. Coifman, R., Rokhlin, V., Wandzura, S.: The fast multipole method for the wave equation: a pedestrian prescription. IEEE Antennas Propag. Mag. 35(3), 7–12 (1993)

    Article  Google Scholar 

  17. Darve, E.: The fast multipole method: numerical implementation. J. Comput. Phys. 160(1), 195–240 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  18. Darve, E.: The fast multipole method. I: error analysis and asymptotic complexity. SIAM J. Numer. Anal. 38(1), 98–128 (2000)

    Google Scholar 

  19. Davis, G., Morris, M.: Six factors which affect the condition number of matrices associated with kriging. Math. Geol. 29(5), 669–683 (1997)

    Article  Google Scholar 

  20. De Boer, A., Van der Schoot, M., Bijl, H.: Mesh deformation based on radial basis function interpolation. Comput. Struct. 85(11–14), 784–795 (2007)

    Article  Google Scholar 

  21. Dietrich, C., Newsam, G.: Efficient generation of conditional simulations by Chebyshev matrix polynomial approximations to the symmetric square root of the covariance matrix. Math. Geol. 27(2), 207–228 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  22. Fong, W., Darve, E.: The black-box fast multipole method. J. Comput. Phys. 228(23), 8712–8725 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  23. Freund, R.: A transpose-free quasi-minimal residual algorithm for non-Hermitian linear systems. SIAM J. Sci. Comput. 14, 470–482 (1993)

    Google Scholar 

  24. Freund, R., Nachtigal, N.: QMR: a quasi-minimal residual method for non-Hermitian linear systems. Numerische Mathematik 60(1), 315–339 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  25. Frieze, A., Kannan, R., Vempala, S.: Fast Monte–Carlo algorithms for finding low-rank approximations. J. ACM (JACM) 51(6), 1025–1041 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  26. Gillman, A., Young, P., Martinsson, P.: A direct solver with \({\cal {O}}(N)\) complexity for integral equations on one-dimensional domains. ArXiv, preprint arXiv:1105.5372 (2011)

  27. Golub, G., Van Loan, C.: Matrix Computations, vol. 3. Johns Hopkins University Press, Baltimore (1996)

    MATH  Google Scholar 

  28. Goreinov, S., Tyrtyshnikov, E., Zamarashkin, N.: A theory of pseudoskeleton approximations. Linear Algebra Appl. 261(1–3), 1–21 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  29. Grasedyck, L., Hackbusch, W.: Construction and arithmetics of \({\cal {H}}\)-matrices. Computing 70(4), 295–334 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  30. Greengard, L., Gueyffier, D., Martinsson, P., Rokhlin, V.: Fast direct solvers for integral equations in complex three-dimensional domains. Acta Numerica 18(1), 243–275 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  31. Greengard, L., Rokhlin, V.: A fast algorithm for particle simulations. J. Comput. Phys. 73(2), 325–348 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  32. Greengard, L., Rokhlin, V.: A new version of the fast multipole method for the Laplace equation in three dimensions. Acta Numerica 6(1), 229–269 (1997)

    Article  MathSciNet  Google Scholar 

  33. Gu, M., Eisenstat, S.: Efficient algorithms for computing a strong rank-revealing QR factorization. SIAM J. Sci. Comput. 17(4), 848–869 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  34. Guennebaud, G., Jacob, B., et al.: Eigen v3. http://eigen.tuxfamily.org (2010)

  35. Gumerov, N., Duraiswami, R.: Fast radial basis function interpolation via preconditioned Krylov iteration. SIAM J. Sci. Comput. 29(5), 1876–1899 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  36. Hackbusch, W.: A sparse matrix arithmetic based on \({\cal {H}}\)-matrices. Part I: introduction to \({\cal {H}}\)-matrices. Computing 62(2), 89–108 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  37. Hackbusch, W., Börm, S.: Data-sparse approximation by adaptive \({\cal {H}}^2\)-matrices. Computing 69(1), 1–35 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  38. Hackbusch, W., Khoromskij, B.: A sparse \(\cal {H}\)-matrix arithmetic. Computing 64(1), 21–47 (2000)

    MathSciNet  MATH  Google Scholar 

  39. Hackbusch, W., Nowak, Z.: On the fast matrix multiplication in the boundary element method by panel clustering. Numerische Mathematik 54(4), 463–491 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  40. Hager, W.: Updating the inverse of a matrix. SIAM Rev. 31(2), 221–239 (1989)

    Google Scholar 

  41. Hestenes, M., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49(6), 409–436 (1952)

    Google Scholar 

  42. Kong, W., Bremer, J., Rokhlin, V.: An adaptive fast direct solver for boundary integral equations in two dimensions. Appl. Comput. Harm. Anal. 31(3), 346–369 (2011)

    Google Scholar 

  43. Liberty, E., Woolfe, F., Martinsson, P., Rokhlin, V., Tygert, M.: Randomized algorithms for the low-rank approximation of matrices. Proc. Natl. Acad. Sci. 104, 20167–20172 (2007)

    Google Scholar 

  44. Martinsson, P.: A fast direct solver for a class of elliptic partial differential equations. J. Sci. Comput. 38(3), 316–330 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  45. Martinsson, P., Rokhlin, V.: A fast direct solver for boundary integral equations in two dimensions. J. Comput. Phys. 205(1), 1–23 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  46. Messner, M., Schanz, M., Darve, E.: Fast directional multilevel summation for oscillatory kernels based on Chebyshev interpolation. J. Comp. Phys. 231(4), 1175–1196 (2012)

    Google Scholar 

  47. Miranian, L., Gu, M.: Strong rank revealing LU factorizations. Linear Algebra Appl. 367, 1–16 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  48. Nishimura, N.: Fast multipole accelerated boundary integral equation methods. Appl. Mech. Rev. 55(4), 299–324 (2002)

    Google Scholar 

  49. O’Dowd, R.: Conditioning of coefficient matrices of ordinary kriging. Math. Geol. 23(5), 721–739 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  50. Paige, C., Saunders, M.: Solution of sparse indefinite systems of linear equations. SIAM J. Numer. Anal. 12(4), 617–629 (1975)

    Google Scholar 

  51. Pan, C.T.: On the existence and computation of rank-revealing LU factorizations. Linear Algebra Appl. 316(1), 199–222 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  52. Rjasanow, S.: Adaptive cross approximation of dense matrices. In: IABEM 2002, International Association for Boundary Element Methods (2002)

  53. Saad, Y., Schultz, M.: GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 7(3), 856–869 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  54. Schaback, R.: Creating surfaces from scattered data using radial basis functions. Math. Methods Curves Surf. 477–496 (1995)

  55. Schmitz, P., Ying, L.: A fast direct solver for elliptic problems on general meshes in 2D. J. Comput. Phys. 231(4), 1314–1338 (2012)

    Google Scholar 

  56. Schmitz, P., Ying, L.: A fast direct solver for elliptic problems on Cartesian meshes in 3D (in review) (2012)

  57. Vandebril, R., Barel, M., Golub, G., Mastronardi, N.: A bibliography on semiseparable matrices. Calcolo 42(3), 249–270 (2005)

    Article  MathSciNet  Google Scholar 

  58. Vavasis, S.: Preconditioning for boundary integral equations. SIAM J. Matrix Anal. Appl. 13, 905–925 (1992)

    Google Scholar 

  59. Van der Vorst, H.: Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG for the solution of non-symmetric linear systems. SIAM J. Sci. Stat. Comput. 13(2), 631–644 (1992)

  60. Wang, J., Liu, G.: A point interpolation meshless method based on radial basis functions. Int. J. Numer. Methods Eng. 54(11), 1623–1648 (2002)

    Article  MATH  Google Scholar 

  61. Woodbury, M.A.: Inverting modified matrices. Statistical Research Group, Memo. Rep. no. 42, Princeton University, Princeton (1950)

  62. Woolfe, F., Liberty, E., Rokhlin, V., Tygert, M.: A fast randomized algorithm for the approximation of matrices. Appl. Comput. Harmon. Anal. 25(3), 335–366 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  63. Wu, Z., Schaback, R.: Local error estimates for radial basis function interpolation of scattered data. IMA J. Numer. Anal. 13(1), 13–27 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  64. Xia, J., Chandrasekaran, S., Gu, M., Li, X.: Fast algorithms for hierarchically semiseparable matrices. Numer. Linear Algebra Appl. 17(6), 953–976 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  65. Ying, L., Biros, G., Zorin, D.: A kernel-independent adaptive fast multipole algorithm in two and three dimensions. J. Comput. Phys. 196(2), 591–626 (2004)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

Sivaram Ambikasaran would like to thank Krithika Narayanaswamy for proof reading the paper and helping in generating the figures.

Author information

Authors and Affiliations

  1. Institute for Computational and Mathematical Engineering, Huang Engineering Center 053B, Stanford University, 475, Via Ortega, Stanford, CA, 94305-4042, USA

    Sivaram Ambikasaran

  2. Mechanical Engineering, Stanford University, Durand 209, 496, Lomita Mall, Stanford, CA, 94305-4042, USA

    Eric Darve

Authors
  1. Sivaram Ambikasaran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eric Darve
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sivaram Ambikasaran.

Additional information

The authors would like to thank the “Army High Performance Computing Research Center” (AHPCRC) and “The Global Climate and Energy Project” (GCEP) at Stanford for supporting the project.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ambikasaran, S., Darve, E. An \(\mathcal O (N \log N)\)  Fast Direct Solver for Partial Hierarchically Semi-Separable Matrices. J Sci Comput 57, 477–501 (2013). https://doi.org/10.1007/s10915-013-9714-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10915-013-9714-z

Keywords

Mathematics Subject Classification (2000)

Search

Navigation