Abstract
A new computational strategy produces independent samples from the joint posterior distribution for a broad class of Bayesian spatial and spatiotemporal conditional autoregressive models. The method is based on reparameterization and marginalization of the posterior distribution and massive parallelization of rejection sampling using graphical processing units (GPUs) or other accelerators. It enables very fast sampling for small to moderate-sized datasets (up to approximately 10,000 observations) and feasible sampling for much larger datasets. Even using a mid-range GPU and a high-end CPU, the GPU-based implementation is up to 30 times faster than the same algorithm run serially on a single CPU, and the numbers of effective samples per second are orders of magnitude higher than those obtained with popular Markov chain Monte Carlo software. The method has been implemented in the R package CARrampsOcl. This work provides both a practical computing strategy for fitting a popular class of Bayesian models and a proof of concept that GPU acceleration can make independent sampling from Bayesian joint posterior densities feasible.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Besag J (1974) Spatial interaction and the statistical analysis of lattice systems (with discussion). J R Stat Soc Ser B 36(2):192–225
Besag J, Higdon D (1999) Bayesian analysis of agricultural field experiments. J R Stat Soc Ser B Stat Methodol 61(1):691–746
Besag J, Kooperberg C (1995) On conditional and intrinsic autoregressions. Biometrika 82(4):733–746
Bratley P, Fox BL, Schrage LE (1987) A guide to simulation, 2nd edn. Springer, New York
Cowles MK, Yan J, Smith B (2009) Reparameterized and marginalized posterior and predictive sampling for complex Bayesian geostatistical models. J Comput Graph Stat 18(2):262–282
Cowles K, Seedorff M, Sawyer A (2013) CARrampsOcl: Reparameterized and marginalized posterior sampling for conditional autoregressive models. OpenCL implementation. R package version 0.1.3
Devroye L (1986) Nonuniform random variate generation. Springer, New York
He Y, Hodges JS, Carlin BP (2007) Re-considering the variance parameterization in multiple precision models. Bayesian Anal 2(1):1–28
Hodges JS (2013) Richly parameterized linear models. Chapman and Hall/CRC Press, Boca Raton
Kass RE, Gelman A, Carlin BP, Neal RM (1998) Markov Chain Monte Carlo in practice: a roundtable discussion. Am Stat 52:93–100
Kunsch H (1994) Robust priors for smoothing and image restoration. Ann Inst Stat Math 55(1):1–19
Laub AJ (2005) Matrix analysis for scientists and engineers. Society for Industrial and Applied Mathematics, Philadelphia
Lee D (2013) CARBayes: an R package for Bayesian spatial modeling with conditional autoregressive priors. J Stat Softw 55(13):1–24. http://www.jstatsoft.org/v55/i13/
Lee A, Yau C, Giles M, Doucet A, Holmes C (2010) On the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods. J Comput Graph Stat 19(4):769–789
Neal RM (2003) Slice sampling. Ann Stat 31(3):705–767
NVIDIA Corporation (2016) CUDA parallel computing platform. http://www.nvidia.com/object/cuda_home_new.html. Accessed 5 Aug 2016
Plummer M, Best N, Cowles K, Vines K (2008) coda: Output analysis and diagnostics for MCMC. R package version 0.13-3
R Development Core Team (2013) R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org, ISBN 3-900051-07-0
Reich BJ, Hodges JS, Carlin BP (2007) Spatial analyses of periodontal data using conditionally autoregressive priors having two classes of neighbor relations. J Am Stat Assoc 102:44–55
Rue H, Held L (2005) Gaussian Markov random fields theory and applications. Chapman and Hall CRC, Boca Raton
Rue H, Martino S, Chopin N (2009a) Approximate Bayesian inference for latent Gaussian models by using Integrated Nested Laplace Approximations. J R Stat Soc Ser B 71:319–392
Rue H, Martino S, Lindgren F, Simpson D, Riebler A (2009b) INLA: Functions which allow to perform full Bayesian analysis of latent Gaussian models using Integrated Nested Laplace Approximaxion. R package version
Sturtz S, Ligges U, Gelman A (2005) R2WinBUGS: a package for running WinBUGS from R. J Stat Softw 12(3):1–16. http://www.jstatsoft.org
The Khronos Group (2016) OpenCL: the open standard for parallel programming of heterogeneous systems. http://www.khronos.org/opencl/. Accessed 5 Aug 2016
Thomas A, O’Hara B, Ligges U, Sturtz S (2006) Making BUGS open. R News 6(1):12–17. http://cran.r-project.org/doc/Rnews/
Acknowledgements
The authors thank Dong Liang, Juan Cervantes, Danielle Dodgen, and Alex Sawyer for helpful discussions and assistance with simulation studies.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Cowles, M.K., Bonett, S. & Seedorff, M. Independent sampling for Bayesian normal conditional autoregressive models with OpenCL acceleration. Comput Stat 33, 159–177 (2018). https://doi.org/10.1007/s00180-017-0752-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00180-017-0752-0