Skip to main content
SpringerLink
Log in

Adaptive proposal distribution for random walk Metropolis algorithm

  • Published:
Computational Statistics Aims and scope Submit manuscript

Summary

The choice of a suitable MCMC method and further the choice of a proposal distribution is known to be crucial for the convergence of the Markov chain. However, in many cases the choice of an effective proposal distribution is difficult. As a remedy we suggest a method called Adaptive Proposal (AP). Although the stationary distribution of the AP algorithm is slightly biased, it appears to provide an efficient tool for, e.g., reasonably low dimensional problems, as typically encountered in non-linear regression problems in natural sciences. As a realistic example we include a successful application of the AP algorithm in parameter estimation for the satellite instrument ‘GOMOS’. In this paper we also present systematic performance criteria for comparing Adaptive Proposal algorithm with more traditional Metropolis algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6

Similar content being viewed by others

References

  • Gelfand, A. E. & Smith, A. F. M. (1990), ‘Sampling-based approaches to calculate marginal densities’, J. Am. Stat. Ass. 85, 853–409.

    Google Scholar 

  • Gelman, A. G., Roberts, G. O. & Gilks, W. R. (1996), Efficient Metropolis jumping rules, in J. M. Bernardo, J. O. Berger, A. F. David & A. F. M. Smith, eds, ‘Bayesian Statistics V’, Oxford University press, Oxford, pp. 599–608.

    Google Scholar 

  • Gilks, W., Best, N. & Tan, K. (1995), ‘Adaptive rejection Metropolis sampling’, Applied Statistics 44, 455–472.

    Article  Google Scholar 

  • Gilks, W. R., Richardson, S. & Spiegelhalter, D. J. (1995), Introducing Markov chain Monte Carlo, in W. R. Gilks, S. Richardson & D. J. Spiegelhalter, eds, ‘Markov Chain Monte Carlo in Practice’, Chapman & Hall, pp. 1–19.

  • Gilks, W. R., Roberts, G. O. & George, E. I. (1994), ‘Adaptive direction sampling’, Statistical Science 43, 179–189.

    Google Scholar 

  • Gilks, W. & Roberts, G. (1995), Stategies for improving MCMC, in W. R. Gilks, S. Richardson & D. J. Spiegelhalter, eds, ‘Markov Chain Monte Carlo in Practice’, Chapman & Hall, pp. 75–88.

  • Gilks, W., Roberts, G. & Sahu, S. (1998), ‘Adaptive Markov chain Monte Carlo through regeneration’, J. Amer. Statist. Assoc. To appear.

  • Gilks, W. & Wild, P. (1992), ‘Adaptive rejection sampling for Gibbs sampling’, Applied Statistics 41, 337–348.

    Article  Google Scholar 

  • Haario, H., Saksman, E. & Tamminen, J. (1998a), Adaptive proposal distribution for random walk Metropolis algorithm. Reports of the Department of Mathematics, University of Helsinki, Preprint 176.

  • Haario, H., Saksman, E. & Tamminen, J. (1998b), An Adaptive Metropolis algorithm. Reports of the Department of Mathematics, University of Helsinki, Preprint 187.

  • Hastings, W. (1970), ‘Monte Carlo sampling methods using Markov chains and their applications’, Biometrica 57, 97–109.

    Article  MathSciNet  Google Scholar 

  • Kyrölä, E., Sihvola, E., Kotivuori, Y., Tikka, M., Tuomi, T. & Haario, H. (1993), ‘Inverse Theory for Occultation Measurements, 1. Spectral Inversion’, J. Geophys. Res. 98, 7367–7381.

    Article  Google Scholar 

  • Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. & Teller, E. (1953), ‘Equations of state calculations by fast computing machine’, J. Chem. Phys. 21, 1087–1091.

    Article  Google Scholar 

  • Press, W. H., Teukolsky, S. A., Vetterling, W. T. & Flannery, B. P. (1992), Numerical Recipes in FORTRAN, The art of scientific computing, Clasendon Press, Oxford.

    MATH  Google Scholar 

  • Roberts, G. O., Gelman, A. & Gilks, W. R. (1994), Weak convergence and optimal scaling of random walk metropolis algorithms. Preprint. URL: https://doi.org/www.stats.bris.ac.uk/MCMC/

  • Sahu, S. K. & Zhigljavsky, A. A. (1998a), Self regenerative Markov chain Monte Carlo. Preprint. URL: https://doi.org/www.stats.bris.ac.uk/MCMC/

  • Sahu, S. K. & Zhigljavsky, A. A. (1998b), Adaptation for self regenerative MCMC. Preprint. URL: https://doi.org/www.stats.bris.ac.uk/MCMC/

Download references

Acknowledgments

The second author (E.S) has been supported by the Academy of Finland, Project 32837. We thank Elja Arjas, Esa Nummelin and Antti Penttinen for useful discussions on the topics of the paper.

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Helsinki, P.O.Box 4, Yliopistonkatu 5, FIN-00014, Helsinki, Finland

    Heikki Haario & Eero Saksman

  2. Geophysical Research Division, Finnish Meteorological Insitute, P.O.Box 503, FIN-00101, Helsinki, Finland

    Johanna Tamminen

Authors
  1. Heikki Haario
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eero Saksman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Johanna Tamminen
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Haario, H., Saksman, E. & Tamminen, J. Adaptive proposal distribution for random walk Metropolis algorithm. Computational Statistics 14, 375–395 (1999). https://doi.org/10.1007/s001800050022

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s001800050022

Keywords

Search

Navigation