Skip to main content
Log in

Discussing the “big n problem”

  • Published:
Statistical Methods & Applications Aims and scope Submit manuscript

Abstract

When a large amount of spatial data is available computational and modeling challenges arise and they are often labeled as “big n problem”. In this work we present a brief review of the literature. Then we focus on two approaches, respectively based on stochastic partial differential equations and integrated nested Laplace approximation, and on the tapering of the spatial covariance matrix. The fitting and predictive abilities of using the two methods in conjunction with Kriging interpolation are compared in a simulation study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Banerjee S, Fuentes M: Bayesian modeling for large spatial datasets. WIREs Comput Stat 4, 59–66 (2012)

    Article  Google Scholar 

  • Banerjee S, Carlin BP, Gelfand AE (2004) Hierarchical modeling and analysis for spatial data. Chapman and Hall, London

  • Banerjee S, Gelfand A, Finley A, Sang H: Gaussian predictive process models for large spatial data sets. J R Stat Soc Ser B (Stat Methodol) 70, 825–848 (2008). doi:10.1111/j.1467-9868.2008.00663.x

    Article  MathSciNet  MATH  Google Scholar 

  • Bolin D, Lindgren F (2011) Spatial wavelet Markov models are more efficient than covariance tapering and process convolutions. arXiv:11061980v1. http://arxiv.org/abs/1106.1980,1106.1980

  • Brenner SC, Scott R: The mathematical theory of finite element methods. Springer, Berlin (2007)

    Google Scholar 

  • Cressie N, Johannesson G: Fixed rank kriging for very large spatial data sets. J R Stat Soc Ser B (Stat Methodol) 70, 209–226 (2008). doi:10.1111/j.1467-9868.2007.00633.x

    Article  MathSciNet  MATH  Google Scholar 

  • Finley A, Sang H, Banerjee S, Gelfand A: Improving the performance of predictive process modeling for large datasets. Computat Stat Data Anal 53, 2873–2884 (2009). doi:10.1016/j.csda.2008.09.008

    Article  MathSciNet  MATH  Google Scholar 

  • Fuentes M: Approximate likelihood for large irregularly spaced spatial data. J Am Stat Assoc 102, 321–331 (2007). doi:10.1198/016214506000000852

    Article  MathSciNet  MATH  Google Scholar 

  • Furrer R, Genton MG, Nychka D: Covariance tapering for interpolation of large spatial datasets. J Comput Graph Stat 15, 502–523 (2006). doi:10.1198/106186006X132178

    Article  MathSciNet  Google Scholar 

  • Haas T: Local prediction of a spatio-temporal process with an application to wet sulfate deposition. J Am Stat Assoc 90, 1189–1199 (1995)

    Article  MATH  Google Scholar 

  • Higdon D, Swall J, Kern J: Non-stationary spatial modeling. Bayesian Stat 6, 761–768 (1998)

    Google Scholar 

  • Ji WY, Simon W, Koray K, Ercan EK: Variant functional approximations for latent Gaussian models. Technical Report of Statistics Department. Trinity College, Dublin (2011)

    Google Scholar 

  • Kaufman C, Schervish M, Nychka D: Covariance tapering for likelihood based estimation in large spatial data set. J Am Stat Assoc 103, 1545–1555 (2008)

    Article  MathSciNet  Google Scholar 

  • Lindgren F, Rue H: Explicit construction of GMRF approximations to generalised Matérn fields on irregular grids. Scand J Stat 35, 691–700 (2007)

    Article  MathSciNet  Google Scholar 

  • Lindgren F, Rue H, Lindström J: An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach. J R Stat Soc Ser B (Stat Methodol) 73, 423–498 (2011). doi:10.1111/j.1467-9868.2011.00777.x

    Article  MathSciNet  MATH  Google Scholar 

  • Mardia K, Goodall C, Refern E, Alonso F: The kriged Kalman filter. Test 7, 217–285 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  • Matsuda Y, Yajima Y: Fourier analysis of irregularly spaced data on R d. J R Stat Soc Ser B (Stat Methodol) 71, 191–217 (2009). doi:10.1111/j.1467-9868.2008.00685.x

    Article  MathSciNet  MATH  Google Scholar 

  • Rubinstein BY: Simulation and the Monte Carlo method. Wiley, New York (1981)

    Book  MATH  Google Scholar 

  • Rue H, Held L: Gaussian Markov random fields theory and applications, 1st edn. Chapman and Hall, London (2005)

    Book  Google Scholar 

  • Rue H, Tjelmeland H (2002) Fitting Gaussian Markov random fields to Gaussian fields. Scand J Stat, pp 31–49. doi:10.1111/1467-9469.00058

  • Rue H, Martino S, Chopin N: Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. J R Stat Soc Ser B (Stat Methodol) 71, 319–392 (2009). doi:10.1111/j.1467-9868.2008.00700.x

    Article  MathSciNet  MATH  Google Scholar 

  • Sampson P, Guttorp P: Nonparametric estimation of nonstationary spatial covariance structure. J Am Stat Assoc 87, 108–119 (1992). doi:10.2307/2290458

    Article  Google Scholar 

  • Shaby B, Ruppert D (2012) Tapered covariance: Bayesian estimation and asymptotics. J Comput Graph Stat 21:433–452

    Google Scholar 

  • Stein ML, Chi Z, Welty LJ: Approximating likelihoods for large spatial data sets. J R Stat Soc Ser B (Stat Methodol) 66, 275–296 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  • Sun Y, Li B, Genton MG: Geostatistics for large datasets. In: Montero, J, Porcu, E, Schlather, M (eds) Advances and challenges in space-time modelling of natural events volume 207 of lecture notes in statistics chap 3, pp. 55–77. Springer, Berlin (2012)

    Chapter  Google Scholar 

  • Wendland H: Piecewise polynomial, positive definite and compactly supported radial functions of minimal degree. Adv Comput Math 4, 389–396 (1995). doi:10.1007/BF02123482

    Article  MathSciNet  MATH  Google Scholar 

  • Whittle P (1954) On stationary processes in the plane. Biometrika 41. doi:10.2307/2332724

  • Whittle P: Stochastic processes in several dimensions. Bull Int Stat Inst 40, 974–994 (1963)

    MathSciNet  Google Scholar 

  • Zhang H: Inconsistent estimation and asymptotically equal interpolations in model-based geostatistics. J Am Stat Assoc 99, 250–261 (2004)

    Article  MATH  Google Scholar 

  • Zhang H, Du J: Covariance tapering in spatial statistics. In: Mateu, E, Porcu, J (eds) Positive definite functions from Schoenberg to space-time challenges, Gráficas Castañ, s.l. (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Giovanna Jona Lasinio.

Electronic Supplementary Material

The Below is the Electronic Supplementary Material.

ESM 1 (R 6 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jona Lasinio, G., Mastrantonio, G. & Pollice, A. Discussing the “big n problem”. Stat Methods Appl 22, 97–112 (2013). https://doi.org/10.1007/s10260-012-0207-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10260-012-0207-2

Keywords

Navigation