Approximating Message Lengths of Hierarchical Bayesian Models Using Posterior Sampling

Abstract

Inference of complex hierarchical models is an increasingly common problem in modern Bayesian data analysis. Unfortunately, there are few computationally efficient and widely applicable methods for selecting between competing hierarchical models. In this paper we adapt ideas from the information theoretic minimum message length principle and propose a powerful yet simple model selection criteria for general hierarchical Bayesian models called MML-h. Computation of this criterion requires only that a set of samples from the posterior distribution be available. The flexibility of this new algorithm is demonstrated by a novel application to state-of-the-art Bayesian hierarchical regression estimation. Simulations show that the MML-h criterion is able to adaptively select between classic ridge regression and sparse horseshoe regression estimators, and the resulting procedure exhibits excellent robustness to the underlying structure of the regression coefficients.