Abstract
MCMC methods have effectively revolutionised the field of Bayesian statistics over the past few years. Such methods provide invaluable tools to overcome problems with analytic intractability inherent in adopting the Bayesian approach to statistical modelling.
However, any inference based upon MCMC output relies critically upon the assumption that the Markov chain being simulated has achieved a steady state or “converged”. Many techniques have been developed for trying to determine whether or not a particular Markov chain has converged, and this paper aims to review these methods with an emphasis on the mathematics underpinning these techniques, in an attempt to summarise the current “state-of-play” for convergence assessment techniques and to motivate directions for future research in this area.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Ahmad, R. (1976) On the Multivariate K-Sample Problem and the Generalization of the Kolmogorov-Smirnov Test. Annals of the Institute of Statistical Mathematics 28, 259–265.
Amit, Y. (1991) On Rates of Convergence of Stochastic Relaxation for Gaussian and Non-Gaussian Distributions. Journal of Multivariate Analysis 38, 82–99.
Amit, Y. and Grenander, U. (1991) Comparing Sweep Strategies for Stochastic Relaxation. Journal of Multivariate Analysis 37, 197–222.
Applegate, D., Kannan, R. and Polson, N. G. (1990) Random Polynomial Time Algorithms for Sampling from Joint Distributions. Technical report, Carnegie-Mellon University, Tech. Rep. No 500.
Asmussen, S., Glynn, P. W. and Thorisson, H. (1992) Stationarity Detection in the Initial Transient Problem. ACM Transactions on Modelling and Computer Simulation 2, 130–157.
Brittain, E. H. (1987) P-Values for the Multi-Sample Kolmogorov-Smirnov Test using the Expanded Bonferroni Approximation. Communications in Statistics — Theory and Methods 16(3), 821–835.
Brooks, S. P. (1997) Discussion to Richardson and Green (1997) Journal of the Royal Statistical Society, Series B 59, 774–775
Brooks, S. P. (1998a), Markov Chain Monte Carlo Method and its Application. The Statistician 47, 69–100.
Brooks, S. P. (1998b) MCMC Convergence Diagnosis via Multivariate Bounds on Log-Concave Densities. The Annals of Statistics 26, 398–433.
Brooks, S. P. (1998c) Quantitative Convergence Diagnosis for MCMC via CUSUMS. Statistics and Computing 8, 267–274.
Brooks, S. P., Dellaportas, P. and Roberts, G. O. (1997) A Total Variation Method for Diagnosing Convergence of MCMC Algorithms. Journal of Computational and Graphical Statistics 6, 251–265.
Brooks, S. P. and Gelman, A. (1998) Alternative Methods for Monitoring Convergence of Iterative Simulations. Journal of Computational and Graphical Statistics 7, 434–455.
Brooks, S. P. and Roberts, G. O. (1997) On Quantile Estimation and MCMC Convergence. Biometrika In press.
Chan, K. S. (1993) Asymptotic Behaviour of the Gibbs Sampler. Journal of the American Statistical Association 88, 320–328.
Chorneyko, I. Z. and Zing, L. L. K. (1982) K-Sample Analogues of Two-Sample Kolmogorov-Smirnov Statistics. Selecta Statistica Canadiana 6, 37–62.
Conover, W. J. (1965) Several K-Sample Kolmogorov-Smirnov Tests. Annals of Mathematical Statistics 36, 1019–1026.
Conover, W. J. (1967) A k-Sample Extension of the One-Sided Two-Sample Smirnov Test Statistic. Annals of Mathematical Statistics 38, 1726–1730.
Cowles, M. K. and Carlin, B. P. (1996) Markov Chain Monte Carlo Convergence Diagnostics: A Comparative Review. Journal of the American Statistical Association 91, 883–904.
Cowles, M. K. and Rosenthal, J. S. (1998) A Simulation Approach to Convergence Rates for Markov Chain Monte Carlo. Technical report, Harvard School of Public Health.
Diaconis, P. (1988) Group Representations in Probability and Statistics, vol. 11 of Lecture Notes — Monograph Series. Institute of Mathematical Statistics.
Diaconis, P. and Stroock, D. (1991) Geometric Bounds for Eigenvalues of Markov Chains. Annals of Applied Probability 1, 36–61.
Fasano, G. and Franceschini, A. (1987) A Multidimensional Version of the Kolmogorov-Smirnov Test. Monthly Notices of the Royal Astronomical Society 225, 155–170.
Garren, S. and Smith, R. L. (1995) Estimating the Second Largest Eigenvalue of a Markov Transition Matrix. Technical report, University of Cambridge.
Gelfand, A. E. (1992) Discussion of Gelman and Rubin (1992) Statistical Science 7, 486–487.
Gelfand, A. E., Hills, S. E., Racine-Poon, A. and Smith, A. F. M. (1990) Illustration of Bayesian Inference in Normal data Models using Gibbs Sampling. Journal of the American Statistical Association 85, 972–985.
Gelfand, A. E. and Smith, A. F. M. (1990) Sampling Based Approaches to Calculating Marginal Densities. Journal of the American Statistical Association 85, 398–409.
Gelman, A. and Rubin, D. B. (1992) Inference from Iterative Simulation using Multiple Sequences. Statistical Science 7, 457–511.
Geman, S. and Geman, D. (1984) Stochastic Relaxation, Gibbs Distributions and the Bayesian Restoration of Images. IEEE Transactions on pattern analysis and machine intelligence 6, 721–741.
Geweke, J. (1989) Bayesian Inference in Econometric Models using Monte-Carlo Integration. Econometrica 57, 1317–1339.
Geweke, J. (1992) Evaluating the Accuracy of Sampling-Based Approaches to the Calculation of Posterior Moments. In J. M. Bernardo, A. F. M. Smith, A. P. Dawid and J. O. Berger (eds.), Bayesian Statistics 4, pp. 169–193, New York: Oxford University Press.
Geyer, C. J. and Thompson, E. A. (1995) Annealing Markov Chain Monte Carlo with Applications to Ancestral Inference. Journal of the American Statistical Association 90, 909–920.
Gilks, W. R., Roberts, G. O. and Sahu, S. K. (1996) Adaptive Markov Chain Monte Carlo. Technical report, MRC Biostatistics Unit, Cambridge.
Green, P. J. (1995) Reversible Jump MCMC Computation and Bayesian Model determination. Biometrika 82, 711–732.
Hastings, W. K. (1970) Monte Carlo Sampling Methods using Markov Chains and their Applications. Biometrika 57, 97–109.
Heidelberger, P. and Welch, P. D. (1981a) Adaptive Spectral Methods for Simulation Output Analysis. IBM Journal of Research and Development 25, 860–876.
Heidelberger, P. and Welch, P. D. (1981b) A Spectral Method for Confidence Interval Generation and Run Length Control in Simulations. Communications of the Association Computing Machinery 24, 233–245.
Heidelberger, P. and Welch, P. D. (1983) Simulation Run Length Control in the Presence of an Initial Transient. Operations Research 31, 1109–1144.
Jennison, C. (1993) Discussion to Smith and Roberts (1993) Journal of the Royal Statistical Society, Series B 55, 54–56.
Johnson, V. E. (1996) Studying Convergence of Markov chain Monte Carlo Algorithms using Coupled Sample Paths. Journal of the American Statistical Association 91, 154–166.
Kendall, W. S. (1997) Perfect simulation for the area interaction process. In C. C. Heyde and L. Acardi (eds.), Probability Perspective, World Scientific Press.
Kolmogorov, A. N. (1933) Sulla Determiazione Empirica delle Leggi di Probabilita. Giornal Istitutto Italia Attuari 4, 1–11.
Lawler, G. F. and Sokal, A. D. (1988) Bounds on the L 2 spectrum for Markov Chains and their Applications. Transactions of the American Mathematical Society 309, 557–580.
Lin, Z. Y. (1992) On the Increments of Partial Sums of a ø-mixing Sequence. Theory of Probability and its Applications 36, 316–326.
Lindvall, T. (1992) Lectures on the Coupling Method. Wiley.
Liu, C., J. Liu and Rubin, D. B. (1993) A Control Variable for Assessment the Convergence of the Gibbs Sampler. In Proceedings of the Statistical Computing Section of the American Statistical Association, pp. 74–78.
Liu, J. S. (1994) Metropolized Independent Sampling with Comparisons to Rejection Sampling and Importance Sampling. Technical report, Harvard University.
Müller, P. (1991) A Generic Approach to Posterior Integration and Gibbs Sampling. Technical report, Dept. of Statistics, Purdue University.
Mengersen, K. L. and Tweedie, R. L. (1995) Rates of Convergence of the Hastings and Metropolis Algorithms. Annals of Statistics 24, 101–121.
Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. and Teller, E. (1953) Equations of State Calculations by Fast Computing Machines. Journal of Chemical Physics 21, 1087–1091.
Meyn, S. P. and Tweedie, R. L. (1993) Markov Chains and Stochastic Stability. Springer-Verlag.
Meyn, S. P. and Tweedie, R. L. (1994) Computable Bounds for geometric Convergence Rates of Markov Chains. Annals of Applied Probability 4, 981–1011.
Mises von, R. (1931) Wahrscheinlichkeitschrechnung. Deutiche, Vienna.
Møller, J. (1997) Markov chain Monte Carlo and spatial point processes. In W. S. Kendall and M. N. M. vanLieshout (eds.), Stochastic geometry, likelihood and computation, Chapman and Hall.
Murdoch, D. J. and Green, P. J. (1997) Exact Sampling from a Continuous State Space. Technical report, Queen's University, Ontario.
Mykland, P., Tierney, L., and Yu, B. (1995) Regeneration in Markov Chain Samplers. Journal of the American Statistical Association 90, 233–241.
Nummelin, E. (1984) General Irreducible Markov Chains and Non-Negative Operators. Cambridge University Press.
Peskun, P. H. (1973) Optimum Monte Carlo Sampling using Markov Chains. Biometrika 60, 607–612.
Polson, N. G. (1996) Convergence of Markov Chain Monte Carlo Algorithms. In J. M. Bernardo, A. F. M. Smith, A. P. Dawid and J. O. Berger (eds.), Bayesian Statistics 5, Oxford University Press.
Propp, J. G. and Wilson, D. B. (1996) Exact Sampling with Coupled Markov Chains and Applications to Statistical Mechanics. Random Structures and Algorithms 9, 223–252.
Raftery, A. E. and Lewis, S. M. (1992) How Many Iterations in the Gibbs Sampler? In J. M. Bernardo, A. F. M. Smith, A. P. Dawid and J. O. Berger (eds.), Bayesian Statistics 4, Oxford University Press.
Reutter, A. and Johnson, V. (1995) General Strategies for Assessing Convergence of MCMC Algorithms Using Coupled Sample Paths. Technical report, Duke University.
Ripley, B. D. (1987) Stochastic Simulation. John Wiley and Sons.
Ritter, C. and Tanner, M. A. (1992) Facilitating the Gibbs Sampler: The Gibbs Stopper and the Griddy-Gibbs Sampler. Journal of the American Statistical Association 87, 861–868.
Robert, C. P. (1996) Convergence Assessments for Markov Chain Monte Carlo Methods. Statistical Science 10, 231–253.
Roberts, G. O. (1992) Convergence Diagnostics of the Gibbs sampler. In J. M. Bernardo, A. F. M. Smith, A. P. Dawid and J. O. Berger (eds.), Bayesian Statistics 4, Oxford University Press.
Roberts, G. O. (1994) Methods for Estimating L 2 Convergence of Markov Chain Monte Carlo. In D. Berry, K. Chaloner and J. Geweke (eds.), Bayesian Statistics and Econometrics: Essays in Honor of Arnold Zellner, North Holland: Amsterdam.
Roberts, G. O. and Polson, N. G. (1994) On the Geometric Convergence of the Gibbs Sampler. Journal of the Royal Statistical Society, Series B 56, 377–384.
Roberts, G. O. and Rosenthal, J. S. (1998) Convergence of Slice Sampler Markov Chains. Journal of the Royal Statistical Society, Series B In press.
Roberts, G. O. and Sahu, S. K. (1996) Rate of Convergence of the Gibbs Sampler by Gaussian Approximation. Technical report, University of Cambridge.
Roberts, G. O. and Sahu, S. K. (1997) Updating Schemes, Covariance Structure, Blocking and Parameterisation for the Gibbs Sampler. Journal of the Royal Statistical Society, Series B 59, 291–318.
Roberts, G. O. and Smith, A. F. M. (1994) Simple Conditions for the Convergence of the Gibbs Sampler and Metropolis Hastings Algorithms. Stochastic Processes and Applications 49, 207–216.
Roberts, G. O. and Tweedie, R. L. (1996a) Exponential Convergence of Langevin Diffusions and their Discrete Approximations. Bernoulli 2, 341–363.
Roberts, G. O. and Tweedie, R. L. (1996b) Geometric Convergence and Central Limit Theorems, for Multidimensional Hastings and Metropolis Algorithms. Biometrika 83, 95–110.
Roberts, G. O. and Tweedie, R. L. (1998) Bounds on Regeneration Times and Convergence Rates for Markov Chains. Technical report, University of Cambridge.
Rosenthal, J. (1995a) Rates of Convergence for Gibbs Sampling for Variance Component Models. Annals of Statistics 23, 740–761.
Rosenthal, J. S. (1995b) Minorization Conditions and Convergence Rates for Markov Chain Monte Carlo. Journal of the American Statistical Association 90, 558–566.
Rosenthal, J. S. (1995c) Rates of Convergence for Data Augmentation on Finite Sample Spaces. Annals of Applied Probability 3, 319–339.
Schervish, M. J. and Carlin, B. P. (1992) On the Convergence of Successive Substitution Sampling. Journal of Computational and Graphical statistics 1, 111–127.
Schruben, L., Singh, H. and Tierney, L. (1983) Optimal Tests for Initialization Bias in Simulation Output. Operations Research 31, 1167–1178.
Schruben, L. W. (1982) Detecting Initialization Bias in Simulation Output. Operations Research 30, 569–590.
Silverman, B. W. (1986) Density Estimation for Statistics and Data Analysis. London: Chapman and Hall.
Sinclair, A. J. and Jerrum, M. R. (1988) Conductance and the Rapid Mixing Property for Markov Chains: The Approximation of the Permanent Resolved. In Proceedings of the 20th annual ACM symposium on the Theory of Computing.
Smith, A. F. M. and Gelfand, A. E. (1992) Bayesian Statistics Without Tears: A Sampling-Resampling Perspective. The American Statistician 46, 84–88.
Smith, R. L. and Tierney, L. (1996) Exact Transition Probabilities for the Independence Metropolis Sampler. Technical report, Statistical Laboratory, Cambridge.
Tierney, L. (1994) Markov Chains for Exploring Posterior Distributions. Annals of Statistics 22, 1701–1762.
Yu, B. (1995a) Discussion to Besag et al. (1995) Statistical Science 10, 3–66.
Yu, B. (1995b) Estimating L 1 Error of Kernel Estimator: Monitoring Convergence of Markov Samplers. Technical report, Dept. of Statistics, University of California, Berkeley.
Yu, B. and Mykland, P. (1998) Looking at Markov Sampler through Cusum Path Plots: A Simple Diagnostic Idea. Statistics and Computing 8, 275–286.
Zellner, A. and Min, C. (1995) Gibbs Sampler Convergence Criteria. Journal of the American Statistical Association 90, 921–927.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
BROOKS, S.P., ROBERTS, G.O. Convergence assessment techniques for Markov chain Monte Carlo. Statistics and Computing 8, 319–335 (1998). https://doi.org/10.1023/A:1008820505350
Issue Date:
DOI: https://doi.org/10.1023/A:1008820505350