Abstract
This paper examines strategies for simulating exactly from large Gaussian linear models conditional on some Gaussian observations. Local computation strategies based on the conditional independence structure of the model are developed in order to reduce costs associated with storage and computation. Application of these algorithms to simulation from nested hierarchical linear models is considered, and the construction of efficient MCMC schemes for Bayesian inference in high-dimensional linear models is outlined.
Similar content being viewed by others
References
Barone P., Sebastiani G., and Stander J. 2000. General over-relaxation Markov chain Monte Carlo algorithms for Gaussian densities. Technical Report 9/2000, Istituto per le Applicazioni del Calcolo “M. Picone,” Consiglio Nazionale delle Ricerche, Rome, Italy.
Boys R.J., Henderson D.A., and Wilkinson D.J. 2000. Detecting homogeneous segments in DNA sequences by using hidden Markov models. Journal of the Royal Statistical Society C:49(2): 269-285.
Chib S. 1996. Calculating posterior distribution and modal estimates in Markov mixture models. Journal of Econometrics 75: 79-97.
Chib S. and Carlin B.P. 1999. On MCMC sampling in hierarchical longdidudinal models. Statistics and Computing 9: 17-26.
Cowell R.G., Dawid A.P., Lauritzen S.L., and Speigelhalter D.J. 1999. Probabilistic Networks and Expert Systems. Springer, New York.
Dawid A.P. 1992. Applications of a general propagation algorithm for probabilistic expert systems. Statistics and Computing 2: 25-36.
Dawid A.P., Kjærulff U., and Lauritzen S.L. 1995. Hybrid propagation in junction trees. Lecture Notes in Computer Science 945: 87-97.
de Jong P. and Shephard N. 1995. The simulation smoother for time-series models. Biometrika 82: 339-350.
García-Cortées L.A. and Sorensen D. 1996. On a multivariate implementation of the Gibbs sampler. Genetics Selection Evolution 28: 121-126.
Gelfand A.E., Sahu S.K., and Carlin B.P. 1995. Efficient parametrisations for normal linear mixed models. Biometrika 82(3): 479-488.
Goldstein M. and Wilkinson D.J. 2000. Bayes linear analysis for graphical models: The geometric approach to local computation and interpretive graphics. Statistics and Computing 10(4): 311-324.
Golub G.H. and Van Loan C.F. 1996. Matrix Computations, 3rd edn. John Hopkins University Press.
Hobert J.P. and Geyer C.J. 1998. Geometric ergodicity of Gibbs and block Gibbs samplers for a hierarchical random effects model. Journal of Multivariate Analysis 67: 414-430.
Jensen C.S. and Kong A. 1996. Blocking-Gibbs sampling for linkage analysis in large pedigrees with many loops. Technical Report R-96-2048, Aalborg University.
Jensen C.S., Kong A., and Kjærulff U. 1995. Blocking-Gibbs sampling in very large probabilistic expert systems. International Journal of Human-Computer Studies 42: 647-666.
Lauritzen S.L. 1992. Propagation of probabilities, means, and variances in mixed graphical association models. Journal of the American Statistical Association 87(420): 1098-1108.
Lauritzen, S.L. 1996. Graphical Models. Oxford Science Publications, Oxford.
Lauritzen S.L. and Speigelhalter D.J. 1988. Local computations with probabilities on graphical structures and their application to expert systems. Journal of the Royal Statistical Society B:50(2): 157-224.
Liu J.S., Wong W.H., and Kong A. 1994. Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes. Biometrika 81: 27-40.
MacDonald I.L. and Zucchini W. 1997. Hidden Markov and Other Models for Discrete-Valued Time Series. Chapman and Hall.
Normand S.-L. and Tritchler D. 1992. Parameter updating in a Bayes network. Journal of the American Statistical Association 87(420): 1109-1115.
Pearl J. 1988. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Mateo, CA.
Remington K.A. and Pozo R. 1996. NIST Sparse BLAS: User's guide. Technical Report, National Institute of Standards and Technology.
Roberts G.O. and Sahu S.K. 1997. Updating schemes, correlation structure, blocking and parameterisation for the Gibbs sampler. Journal of the Royal Statistical Society B:59(2): 291-317.
Rue H. 1999. A fast and exact simulation algorithm for general Gaussian Markov random fields. Statistics no. 8, Department of Mathematical Sciences, Norwegian University of Science and Technology, Trondheim, Norway.
Rue H. 2000. Fast sampling of Gaussian Markov random fields with applications. Statistics no. 1, Department of Mathematical Sciences, Norwegian University of Science and Technology, Trondheim, Norway.
Simpson A. and Wilkinson D.J. 2000. Multivariate DLMs for financial time series, with application to the management of portfolios. In Bethlehem J.G. and van der Heijden P.G.M. (Eds.), Proceedings in Computational Statistics 2000, Physica-Verlag, Heidelberg, pp. 457-462.
Tanner M.A. and Wong W.H. 1987. The calculation of posterior distributions by data augmentation. Journal of the American Statistical Association 82(398): 528-540.
West M. and Harrison J. 1997. Bayesian Forecasting and Dynamic Models, 2nd edn. Springer, New York.
Whittaker J. 1990. Graphical Models in Applied Multivariate Statistics. Wiley, Chichester.
Wilkinson D.J. 1998. An object-oriented approach to local computation in Bayes linear belief networks. In Payne R. and Green P.J. (Eds.), Proceedings in Computational Statistics 1998, Physica-Verlag, Heidelberg, pp. 491-496.
Rights and permissions
About this article
Cite this article
Wilkinson, D.J., Yeung, S.K.H. Conditional simulation from highly structured Gaussian systems, with application to blocking-MCMC for the Bayesian analysis of very large linear models. Statistics and Computing 12, 287–300 (2002). https://doi.org/10.1023/A:1020711129064
Issue Date:
DOI: https://doi.org/10.1023/A:1020711129064