Skip to main content
Log in

Conditional simulation from highly structured Gaussian systems, with application to blocking-MCMC for the Bayesian analysis of very large linear models

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

This paper examines strategies for simulating exactly from large Gaussian linear models conditional on some Gaussian observations. Local computation strategies based on the conditional independence structure of the model are developed in order to reduce costs associated with storage and computation. Application of these algorithms to simulation from nested hierarchical linear models is considered, and the construction of efficient MCMC schemes for Bayesian inference in high-dimensional linear models is outlined.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Barone P., Sebastiani G., and Stander J. 2000. General over-relaxation Markov chain Monte Carlo algorithms for Gaussian densities. Technical Report 9/2000, Istituto per le Applicazioni del Calcolo “M. Picone,” Consiglio Nazionale delle Ricerche, Rome, Italy.

    Google Scholar 

  • Boys R.J., Henderson D.A., and Wilkinson D.J. 2000. Detecting homogeneous segments in DNA sequences by using hidden Markov models. Journal of the Royal Statistical Society C:49(2): 269-285.

    Google Scholar 

  • Chib S. 1996. Calculating posterior distribution and modal estimates in Markov mixture models. Journal of Econometrics 75: 79-97.

    Google Scholar 

  • Chib S. and Carlin B.P. 1999. On MCMC sampling in hierarchical longdidudinal models. Statistics and Computing 9: 17-26.

    Google Scholar 

  • Cowell R.G., Dawid A.P., Lauritzen S.L., and Speigelhalter D.J. 1999. Probabilistic Networks and Expert Systems. Springer, New York.

    Google Scholar 

  • Dawid A.P. 1992. Applications of a general propagation algorithm for probabilistic expert systems. Statistics and Computing 2: 25-36.

    Google Scholar 

  • Dawid A.P., Kjærulff U., and Lauritzen S.L. 1995. Hybrid propagation in junction trees. Lecture Notes in Computer Science 945: 87-97.

  • de Jong P. and Shephard N. 1995. The simulation smoother for time-series models. Biometrika 82: 339-350.

    Google Scholar 

  • García-Cortées L.A. and Sorensen D. 1996. On a multivariate implementation of the Gibbs sampler. Genetics Selection Evolution 28: 121-126.

    Google Scholar 

  • Gelfand A.E., Sahu S.K., and Carlin B.P. 1995. Efficient parametrisations for normal linear mixed models. Biometrika 82(3): 479-488.

    Google Scholar 

  • Goldstein M. and Wilkinson D.J. 2000. Bayes linear analysis for graphical models: The geometric approach to local computation and interpretive graphics. Statistics and Computing 10(4): 311-324.

    Google Scholar 

  • Golub G.H. and Van Loan C.F. 1996. Matrix Computations, 3rd edn. John Hopkins University Press.

  • Hobert J.P. and Geyer C.J. 1998. Geometric ergodicity of Gibbs and block Gibbs samplers for a hierarchical random effects model. Journal of Multivariate Analysis 67: 414-430.

    Google Scholar 

  • Jensen C.S. and Kong A. 1996. Blocking-Gibbs sampling for linkage analysis in large pedigrees with many loops. Technical Report R-96-2048, Aalborg University.

  • Jensen C.S., Kong A., and Kjærulff U. 1995. Blocking-Gibbs sampling in very large probabilistic expert systems. International Journal of Human-Computer Studies 42: 647-666.

    Google Scholar 

  • Lauritzen S.L. 1992. Propagation of probabilities, means, and variances in mixed graphical association models. Journal of the American Statistical Association 87(420): 1098-1108.

    Google Scholar 

  • Lauritzen, S.L. 1996. Graphical Models. Oxford Science Publications, Oxford.

    Google Scholar 

  • Lauritzen S.L. and Speigelhalter D.J. 1988. Local computations with probabilities on graphical structures and their application to expert systems. Journal of the Royal Statistical Society B:50(2): 157-224.

    Google Scholar 

  • Liu J.S., Wong W.H., and Kong A. 1994. Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes. Biometrika 81: 27-40.

    Google Scholar 

  • MacDonald I.L. and Zucchini W. 1997. Hidden Markov and Other Models for Discrete-Valued Time Series. Chapman and Hall.

  • Normand S.-L. and Tritchler D. 1992. Parameter updating in a Bayes network. Journal of the American Statistical Association 87(420): 1109-1115.

    Google Scholar 

  • Pearl J. 1988. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Mateo, CA.

    Google Scholar 

  • Remington K.A. and Pozo R. 1996. NIST Sparse BLAS: User's guide. Technical Report, National Institute of Standards and Technology.

  • Roberts G.O. and Sahu S.K. 1997. Updating schemes, correlation structure, blocking and parameterisation for the Gibbs sampler. Journal of the Royal Statistical Society B:59(2): 291-317.

    Google Scholar 

  • Rue H. 1999. A fast and exact simulation algorithm for general Gaussian Markov random fields. Statistics no. 8, Department of Mathematical Sciences, Norwegian University of Science and Technology, Trondheim, Norway.

    Google Scholar 

  • Rue H. 2000. Fast sampling of Gaussian Markov random fields with applications. Statistics no. 1, Department of Mathematical Sciences, Norwegian University of Science and Technology, Trondheim, Norway.

    Google Scholar 

  • Simpson A. and Wilkinson D.J. 2000. Multivariate DLMs for financial time series, with application to the management of portfolios. In Bethlehem J.G. and van der Heijden P.G.M. (Eds.), Proceedings in Computational Statistics 2000, Physica-Verlag, Heidelberg, pp. 457-462.

    Google Scholar 

  • Tanner M.A. and Wong W.H. 1987. The calculation of posterior distributions by data augmentation. Journal of the American Statistical Association 82(398): 528-540.

    Google Scholar 

  • West M. and Harrison J. 1997. Bayesian Forecasting and Dynamic Models, 2nd edn. Springer, New York.

    Google Scholar 

  • Whittaker J. 1990. Graphical Models in Applied Multivariate Statistics. Wiley, Chichester.

    Google Scholar 

  • Wilkinson D.J. 1998. An object-oriented approach to local computation in Bayes linear belief networks. In Payne R. and Green P.J. (Eds.), Proceedings in Computational Statistics 1998, Physica-Verlag, Heidelberg, pp. 491-496.

    Google Scholar 

Download references

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wilkinson, D.J., Yeung, S.K.H. Conditional simulation from highly structured Gaussian systems, with application to blocking-MCMC for the Bayesian analysis of very large linear models. Statistics and Computing 12, 287–300 (2002). https://doi.org/10.1023/A:1020711129064

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1020711129064

Navigation