Skip to main content
SpringerLink
Log in

Metropolized independent sampling with comparisons to rejection sampling and importance sampling

  • Papers
  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

Although Markov chain Monte Carlo methods have been widely used in many disciplines, exact eigen analysis for such generated chains has been rare. In this paper, a special Metropolis-Hastings algorithm, Metropolized independent sampling, proposed first in Hastings (1970), is studied in full detail. The eigenvalues and eigenvectors of the corresponding Markov chain, as well as a sharp bound for the total variation distance between the nth updated distribution and the target distribution, are provided. Furthermore, the relationship between this scheme, rejection sampling, and importance sampling are studied with emphasis on their relative efficiencies. It is shown that Metropolized independent sampling is superior to rejection sampling in two respects: asymptotic efficiency and ease of computation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Besag, J. and Green, P.J. (1993). Spatial statistics and Baysian computation. Journal of the Royal Statistical Society B, 55, 24–35.

    Google Scholar 

  • Diaconis, P. (1988) Group Representations in Probability and Statistics, Lecture Notes-Monograph Series 11, IMS, Hayward California.

    Google Scholar 

  • Diaconis, P. and Hanlon, P. (1992) Eigen analysis for some examples of the Metropolis algorithm. Contemporary Mathematics, 138, 99–117.

    Google Scholar 

  • Gelfand, A. E. and Smith, A. F. M. (1990) Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association, 85, 398–409.

    Google Scholar 

  • Gelman, A. and Rubin, D. B. (1993) Discussion on Gibbs sampler and other MCMC methods. Journal of the Royal Statistical Society B, 55, 73–73.

    Google Scholar 

  • Geman, S. and Geman, D. (1984) Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–41.

    Google Scholar 

  • Hastings, W. K. (1970) Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57, 97–109.

    Google Scholar 

  • Kong, A. (1992) A note on importance sampling using renormalized weights. Technical report, Department of Statistics, University of Chicago.

  • Lindvall, T. (1992) Lectures on the Coupling Method. Wiley, New York.

    Google Scholar 

  • Liu, J. S. (1994) The collapsed Gibbs sampler in Bayesian computations with applications to a gene regulation problem. Joural of the American Statistical Association, 89, 958–66.

    Google Scholar 

  • Liu, J. S., Kong, A. and Wong, W. H. (1994) Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes. Biometrika, 81, 27–40.

    Google Scholar 

  • Lovasz, L. and Simonovits, M. (1990) The mixing rate of Markov chains, an isoperimetric inequality, and computing the volume. Preprint 27, Hungarian Academy of Sciences.

  • Marshall, A. W. (1956) The use of multi-stage sampling schemes in Monte Carlo computations. In Symposium on Monte Carlo Methods, ed. M. A. Meyer, pp. 123–40, Wiley, New York.

    Google Scholar 

  • Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. and Teller, E. (1953) Equations of state calculations by fast computing machines. Journal of Chemical Physics, 21, 1087–91.

    Google Scholar 

  • von Neumann, J. (1951) Various techniques used in connection with random digits. National Bureau of Standards Applied Mathematics Series, 12, 36–8.

    Google Scholar 

  • Peskun, P. H. (1973) Optimum Monte-Carlo sampling using Markov chains. Biometrika, 60, 607–12.

    Google Scholar 

  • Rosenthal, J. S. (1995) Rates of convergence for Gibbs sampler for variance components models. Ann. Statist., 23, 740–61.

    Google Scholar 

  • Smith, A. F. M. and Roberts, G. O. (1993) Bayesian Computation via the Gibbs sampler and related Markov chain Monte Carlo methods. Journal of the Royal Statistical Society, B, 55, 3–23.

    Google Scholar 

  • Smith, R. L. (1994) Exact transition probabilities for Metropolized independent sampling. Technical Report, Dept. Statistics, Univ. of North Carolina.

  • Tanner, M. A. and Wong, W. H. (1987) The calculation of posterior distributions by data augmentation (with discussion). Journal of American Statistical Association, 82, 528–50.

    Google Scholar 

  • Tierney, L. (1991) Markov chains for exploring posterior distributions. In Computer Science and Statistics: Proc. 23rd Symp. Interface.

  • Yoida, K. (1978) Functional Analysis. Springer-Verlag, New York.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, Stanford University, 94305, Stanford, CA, USA

    Jun S. Liu

Authors
  1. Jun S. Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, J.S. Metropolized independent sampling with comparisons to rejection sampling and importance sampling. Stat Comput 6, 113–119 (1996). https://doi.org/10.1007/BF00162521

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00162521

Keywords

Search

Navigation