Abstract
New Metropolis–Hastings algorithms using directional updates are introduced in this paper. Each iteration of a directional Metropolis–Hastings algorithm consists of three steps (i) generate a line by sampling an auxiliary variable, (ii) propose a new state along the line, and (iii) accept/reject according to the Metropolis–Hastings acceptance probability. We consider two classes of directional updates. The first uses a point in \({\cal R}\) n as auxiliary variable, the second an auxiliary direction vector. The proposed algorithms generalize previous directional updating schemes since we allow the distribution of the auxiliary variable to depend on properties of the target at the current state. By letting the proposal distribution along the line depend on the density of the auxiliary variable, we identify proposal mechanisms that give unit acceptance rate. When we use direction vector as auxiliary variable, we get the advantageous effect of large moves in the Markov chain and hence the autocorrelation length of the samples is small. We apply the directional Metropolis–Hastings algorithms to a Gaussian example, a mixture of Gaussian densities, and a Bayesian model for seismic data.
Similar content being viewed by others
References
Buland A., Kolbjørnsen O. and Omre H. 2003. Rapid spatially coupled AVO inversion in the Fourier domain. Geophysics 68: 824–836.
Buland A. and Omre H. 2003. Bayesian linearized AVO inversion. Geophysics 68: 185–198.
Chan G. and Wood A.T.A. 1997. An algorithm for simulating stationary Gaussian random fields. Applied Statistics 46: 171–181.
Chen M. and Schmeiser B. 1993. Performance of the Gibbs, Hit-and-Run and Metropolis Samplers. Journal of Computational and Graphical Statistics 2: 251–272.
Chib S. and Greenberg E. 1995. Understanding the Metropolis–Hastings algorithm. American Statistician 90: 1313–1321.
Chib S. and Greenberg E. 1998. Analysis of multivariate probit models. Biometrika 85: 347–361.
Erland S. 2003. On Eigen-Decompositions and Adaptivity of Markov Chains. PhD Thesis 2003:102. Norwegian University of Science and Technology.
Geyer C.J. 1992. Practical Markov chain Monte Carlo. Statistical Science 7: 473–483.
Gilks W.R. and Wild P. 1992. Adaptive rejection sampling for Gibbs sampling. Applied Statistics 41: 337–348.
Gilks W.R., Roberts G.O., and George E.I. 1994. Adaptive direction sampling. The Statistician 43: 179–189.
Goodman J. and Sokal A.D. 1989. Multigrid Monte Carlo method. Conceptual foundations. Physical Review D 40: 2035–2072.
Green P. 1995. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82: 711–732.
Gustafson P., Macnab Y.C., and Wen S. 2004. On the value of derivative evaluations and random walk suppression in Markov Chain Monte Carlo algorithms. Statistics and Computing 14: 23–38.
Hall P., Marron J.S., and Neeman A. 2005. Geometric representation of high dimension, low sample size data. Journal of Royal Statistical Society, Series B, 67: 427–444.
Hastings W.K. 1970. Monte Carlo simulation methods using Markov chains and their applications. Biometrika 57: 97–109.
Kaufman D.E. and Smith R.L. 1998. Direction choice for accelerated convergence in hit-and-run sampling. Operations Research 46: 84–95.
Landrø M. and Strønen L.K. 2003. 4D study of fluid effects on seismic data in the Gullfaks Field, North Sea. Geofluids 3: 233–244.
Liu J.S., Liang F., and Wong W.H. 2000. The multiple-try method and local optimization in Metropolis sampling. Journal of American Statistical Association 95: 121–134.
Liu J.S. and Sabatti C. 2000. Generalized Gibbs sampler and multigrid Monte Carlo for Bayesian computation. Biometrika 87: 353–369.
Liu J.S. 2001. Monte Carlo Strategies in Scientific Computing, Springer.
Neal R.M. 1996. Sampling from multimodal distributions using tempered transitions. Statistics and Computing 6: 353–366.
Peskun P.H. 1973. Optimum Monte Carlo sampling using Markov chains. Biometrika 60: 607–612.
Press W.H., Teukolsky S.A., Vetterling W.T. and Flannery B.P. 1996. Numerical Recipes in C: The art of Scientific Computing, Cambridge University Press.
Pukkila T.M. and Rao C.R. 1988. Pattern recognition based on scale invariant discriminant functions. Information Sciences 45: 379–389.
Ripley B. 1987. Stochastic Simulation, Wiley.
Roberts G.O. and Gilks W.R. 1994. Convergence of adaptive direction sampling. Journal of Multivariate Analysis 49: 287–298.
Roberts G.O., Gelman A. and Gilks W.R. 1997. Weak convergence and optimal scaling of random walk Metropolis algorithms. Annals of Applied Probability 7: 110–120.
Roberts G.O. and Rosenthal J.S. 1998. Optimal scaling of discrete approximations to Langevin diffusions. Journal of Royal Statistical Society, Series B, 60: 255–268.
Sheriff R.E. and Geldart L.P. 1995. Exploration Seismology, Cambridge.
Stramer O. and Tweedie R.L. 2003. Langevin-Type Models II: Self-targeting candidates for MCMC algorithms. Methodology & Computing in Applied Probability 1: 307–328.
Tierney L. and Mira A. 1999. Some adaptive Monte Carlo methods for Bayesian inference. Statistics in Medicine 18: 2507–2515.
Tjelmeland H. and Eidsvik J. 2004a. On the use of local optimisations within Metropolis–Hastings updates. Journal of Royal Statistical Society, Series B 66: 411–427.
Tjelmeland H. and Eidsvik J. 2004b. Directional Metropolis–Hastings updates for posteriors with nonlinear likelihoods. Technical report 2004:5, Norwegian University of Science and Technology, www.math.ntnu.no/preprint/statistics/2004/
Waagepetersen R. and Sørensen D. 2001. A tutorial on reversible jump MCMC with a view towards applications in QTL-mapping. International Statistical Review 69: 49–61.
Watson G.S. 1983. Statistics on Spheres, Wiley-Interscience.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Eidsvik, J., Tjelmeland, H. On directional Metropolis–Hastings algorithms. Stat Comput 16, 93–106 (2006). https://doi.org/10.1007/s11222-006-5536-2
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s11222-006-5536-2