MCMC-based local parametric sensitivity estimations

doi:10.1016/j.csda.2005.09.005

Computational Statistics & Data Analysis

Volume 51, Issue 2, 15 November 2006, Pages 823-835

https://doi.org/10.1016/j.csda.2005.09.005 Get rights and content

Abstract

Bayesian inferences for complex models need to be made by approximation techniques, mainly by Markov chain Monte Carlo (MCMC) methods. For these models, sensitivity analysis is a difficult task. A novel computationally low-cost approach to estimate local parametric sensitivities in Bayesian models is proposed. This method allows to estimate the sensitivity measures and their errors with the same random sample that has been generated to estimate the quantity of interest. Conditions to allow a derivative-integral interchange in the operator of interest are required. Two illustrative examples have been considered to show how sensitivity computations with respect to the prior distribution and the loss function are easily obtained in practice.

Introduction

Bayesian statistics has become more popular, thanks to the appearance of Markov chain Monte Carlo (MCMC) methods (see Brooks, 1998 for a review and Gilks et al., 1998 for a monograph). The application of these simulation techniques allows to obtain a numerical solution of problems based on really complex models. Sometimes, MCMC methods are the only computationally efficient alternative.

When performing a Bayesian analysis, inferences depend on some input models as the prior distribution, the likelihood or the loss function. Besides the model solution, some description of its sensitivity to the specification of these inputs is necessary. Sensitivity of inferences to the choice of the prior distribution has been widely investigated (see, for example, Berger, 1994). Sivaganesan (1993) and Dey et al. (1996) studied sensitivity with respect to the prior and the likelihood. Martín et al. (1998) considered the loss function and Martín et al. (2003) investigated joint sensitivity with respect to utility and prior distribution. Two relevant monographs on robust Bayesian analysis are provided by Berger et al. (1996) and Ríos Insua and Ruggeri (2000).

Sensitivity analysis can be studied from two viewpoints: local and global. Local sensitivity considers the behavior of posterior quantities of interest under infinitesimal perturbations from a specified input (prior or loss in this paper). On the other hand, global sensitivity quantifies the range of a posterior quantity when the prior (loss) varies in a class. See Sivaganesan (2000) for a comparative review on the local and global approaches to Bayesian robustness.

Sensitivity analyses are demanded by several authors to be applied in complex models that need to be solved by MCMC methods (see, for example, Ríos Insua and Ruggeri, 2000). Some authors, like Richarson and Green (1997), Hall et al. (2003), Halekoh and Vach (2003) study parametric sensitivity by solving the model for some values of the prior parameters. They, basically, re-run the Markov chain for different parameters of the prior distributions and estimate the quantities of interest under those parameter specifications. This is computationally costly and, generally, not enough. Therefore, it would be convenient to develop a general method that can be applied to estimate local sensitivities. This is the issue addressed in the paper.

The outline of the paper is as follows. In Section 2, the problem is described and some relevant results on local sensitivity analysis are summarized. Section 3 focuses on the parametric sensitivity case. A computationally low-cost method to estimate local parametric sensitivities in models solved by MCMC methods is proposed. Section 4 presents two illustrative examples which show how the proposed method is easily applied in practice. A discussion is presented in Section 5. Finally, an appendix containing the proofs is included.

Section snippets

Local sensitivity estimations

Suppose the interest is focused on the estimation of a quantity that can be expressed as the integral of a real valued function f over a multiple dimension domain with respect to a density p, i.e. $\int_{Θ} f (θ) p (θ) d θ .$ When p is the posterior distribution for $θ$ , i.e., $p (θ | x)$ , the integral (1) becomes the posterior expectation of $f (θ)$ . In this case, the posterior mean is recovered when f is the identity function.

In Bayesian decision theory and inference, the elicitation of the prior distribution $π (θ)$ (or $f$

Parametric classes

In this section, a parametric class of prior distributions $Γ = \{π_{λ} : λ \in Λ\}$ and a parametric class of functions $F = \{f_{ξ} : ξ \in Ξ\}$ are considered. Let $I_{λ} (π, f)$ and $I_{ξ} (π, f)$ ( $I_{λ}$ and $I_{ξ}$ for short) denote the dependence of the posterior quantity $I (π, f)$ on $λ = (λ_{1}, \dots, λ_{m})$ and $ξ = (ξ_{1}, \dots, ξ_{m})$ , respectively.

Firstly, sensitivity with respect to the prior distribution is considered. Assume that a local sensitivity analysis around $λ = λ^{0}$ is required in the class $Γ$ . Then, $I_{λ}$ is, as a function of parameters, an operator from $R^{m}$ to the

Applications

Two illustrative examples are considered in this section: an application to a normal mixture model and a decision problem based on a first-order autoregressive model.

Example 1 Normal mixture model

Bowmaker et al. (1985) analyzed data on the peak sensitivity wavelengths for individual microspectrophotometric records on a small set of monkey's eyes. Part of the analysis involves fitting a mixture of two normal distributions with common variance, so that each observation $y_{i}$ is assumed to be drawn from one of two groups. Let $T_{i} =$

Discussion

It is widely recognized that formal sensitivity analysis is a difficult task in Bayesian modeling. Several local sensitivity measures based on functional derivatives have been proposed in the recent literature, for example, the Fréchet derivative and its norm is one of them. In the particular case of parametric classes, the Fréchet derivative is the gradient vector. The method proposed in this paper uses the components of the gradient vector, i.e., the partial derivatives as sensitivity

Acknowledgements

The authors thank a referee and an associate editor for comments and suggestions which have substantially improved the readability and the content of this paper. Discussions with David Ríos are also gratefully acknowledged. This work has been partially supported by Junta de Extremadura, Spain (Project IPR00A075).

References (32)

J.K. Bowmaker et al.
Two types of trichromatic squirrel monkey share pigment in the red-green spectral region
Vision Res.
(1985)
C.B. Hall et al.
Bayesian and profile likelihood change point methods for modeling cognitive function over time
Comput. Statist. Data Anal.
(2003)
F. Ruggeri et al.
Density based classes of priors: infinitesimal properties and approximations
J. Statist. Plann. Inference
(1995)
S. Sivaganesan
Robust Bayesian diagnostics
J. Statist. Plann. Inference
(1993)
J.O. Berger
Statistical Decision Theory and Bayesian Analysis
(1985)
J.O. Berger
An overview of robust Bayesian analysis
Test
(1994)
Berger, J.O., Betro, B., Moreno, E., Pericchi, L.R., Ruggeri, F., Salinetti, G., Wasserman, L.e., 1996. Bayesian...
J.O. Berger et al.
Robust Bayesian analysis
Best, N.G., Cowles, M.K., Vines, S.K., 1997. Coda: convergence diagnosis and output analysis software for Gibbs...
S.P. Brooks
Markov chain Monte Carlo method and its application
The Statistician
(1998)

A. Cuevas et al.

On differentiability properties of Bayes operators

Dey, D., Ghosh, S., Lou, K., 1996. On local sensitivity measures in bayesian analysis (with discussion). In: Berger,...

P. Diaconis et al.

On the consistency of Bayes estimates

Annals of Statistics

(1986)

T.J. DiCiccio et al.

Computing Bayes factors by combining simulation and asymptotic approximations

J. Amer. Statist. Assoc.

(1997)

R.M. Dorazio et al.

Bayesian inference and decision theory—a framework for decision making in natural resource management

Ecol. Appl.

(2003)

S. French et al.

Statistical Decision Theory

(2000)

Cited by (26)

An in-depth analysis of Markov-Chain Monte Carlo ensemble samplers for inverse vadose zone modeling
2023, Journal of Hydrology
This study elucidates the behavior of Markov-Chains Monte Carlo ensemble samplers for vadose zone inverse modeling by performing an in-depth comparison of four algorithms that use Affine-Invariant (AI) moves or Differential Evolution (DE) strategies to approximate the target density. Two Rosenbrock toy distributions, and one synthetic and one actual case study focusing on the inverse estimation of soil hydraulic parameters using HYDRUS-1D, are used to compare samplers in different dimensions d. The analysis reveals that an ensemble with $N = d + 1$ chains evolved using DE-based strategies converges to the wrong stationary posterior, while AI does not suffer from this issue but exhibits delayed convergence. DE-based samplers regain their ergodic properties when using $N \geq 2 d$ chains. Increasing the number of chains above this threshold has only minor effects on the samplers’ performance, while initializing the ensemble in a high-likelihood region facilitates its convergence. AI strategies exhibit shorter autocorrelation times in the 7d synthetic vadose zone scenario, while DE-based samplers outperform them when the number of soil parameters increases to 16 in the actual scenario. All evaluation metrics degrade as d increases, thus suggesting that sampling strategies based only on interpolation between chains tend to become inefficient when the bulk of the posterior lays in increasingly small portions of the parameters’ space.
Adversarial life testing: A Bayesian negotiation model
2014, Reliability Engineering and System Safety
Citation Excerpt :
It would be very interesting to carry out sensitivity analyses for the parameters of both, the prior distribution and the utility functions corresponding to the manufacturer and/or the consumer. Local parametric sensitivity analysis can be performed by using the approach in Pérez et al. [30] and Rufo et al. [34]. Finally, extensions of this study could be performed, by considering other quality measures for the product different from the lifetime or by considering a quality control-based approach.
Life testing is a procedure intended for facilitating the process of making decisions in the context of industrial reliability. On the other hand, negotiation is a process of making joint decisions that has one of its main foundations in decision theory. A Bayesian sequential model of negotiation in the context of adversarial life testing is proposed. This model considers a general setting for which a manufacturer offers a product batch to a consumer. It is assumed that the reliability of the product is measured in terms of its lifetime. Furthermore, both the manufacturer and the consumer have to use their own information with respect to the quality of the product. Under these assumptions, two situations can be analyzed. For both of them, the main aim is to accept or reject the product batch based on the product reliability. This topic is related to a reliability demonstration problem. The procedure is applied to a class of distributions that belong to the exponential family. Thus, a unified framework addressing the main topics in the considered Bayesian model is presented. An illustrative example shows that the proposed technique can be easily applied in practice.
Bayesian influence analysis of generalized partial linear mixed models for longitudinal data
2014, Journal of Multivariate Analysis
This paper develops a Bayesian local influence approach to assess the effects of minor perturbations to the prior, sampling distribution and individual observations on the statistical inference in generalized partial linear mixed models (GPLMMs) with the distribution of random effects specified by a truncated and centered Dirichlet process (TCDP) prior. A perturbation manifold is defined. The metric tensor is employed to select an appropriate perturbation vector. Several Bayesian local influence measures are proposed to quantify the degree of various perturbations to statistical models based on the first and second-order approximations to the objective functions including the $ϕ$ -divergence, the posterior mean distance and Bayes factor. We develop two Bayesian case influence measures to detect the influential observations in GPLMMs based on the $ϕ$ -divergence and Cook’s posterior mean distance. The computationally feasible formulae for Bayesian influence analysis are given. Several simulation studies and a real example are presented to illustrate the proposed methodologies.
Measuring prior sensitivity and prior informativeness in large Bayesian models
2012, Journal of Monetary Economics
Citation Excerpt :
Berger (1994), Gustafson (2000) and Sivaganesan (2000) provide overviews and references. More specifically, Basu et al. (1996), Geweke (1999) and Perez et al. (2006) study the local sensitivity of the posterior mean in a parametric class of priors, which amounts to the computation of the posterior mean derivative with respect to the prior hyperparameter. Millar (2004) observes that if the scalar marginal prior distribution is in the exponential family, then the derivative with respect to the prior mean is simply given by the ratio of the posterior to prior variance.
In large Bayesian models, such as modern DSGE models, it is difficult to assess how much the prior affects the results. This paper derives measures of prior sensitivity and prior informativeness that account for the high dimensional interaction between prior and likelihood information. The basis for both measures is the derivative matrix of the posterior mean with respect to the prior mean, which is easily obtained from Markov Chain Monte Carlo output. We illustrate the approach by examining posterior results in the small model of Lubik and Schorfheide (2004) and the large model of Smets and Wouters (2007).
Bayesian analysis of a generalized lognormal distribution
2009, Computational Statistics and Data Analysis
Citation Excerpt :
Sensitivity analysis is an important task in this context. Pérez et al. (2006) presented a computationally low-cost approach to estimate local parametric sensitivities for models solved by MCMC-methods. The following step is to choose the hyperparameter values for the prior distributions.
Many data arising in reliability engineering can be modeled by a lognormal distribution. Empirical evidences from many sources support this argument. However, sometimes the lognormal distribution does not completely satisfy the fitting expectations in real situations. This fact motivates the use of a more flexible family of distributions with both heavier and lighter tails compared to the lognormal one, which is always an advantage for robustness. A generalized form of the lognormal distribution is presented and analyzed from a Bayesian viewpoint. By using a mixture representation, inferences are performed via Gibbs sampling. Although the interest is focused on the analysis of lifetime data coming from engineering studies, the developed methodology is potentially applicable to many other contexts. A simulated and a real data set are presented to illustrate the applicability of the proposed approach.
Bayesian robustness for decision making problems: Applications in medical contexts
2009, International Journal of Approximate Reasoning
Practical implementation of Bayesian decision making is hindered by the fact that optimal decisions may be sensitive to the model inputs: the prior, the likelihood and/or the underlying utility function. Given the structure of a problem, the analyst has to decide which sensitivity measures are relevant and compute them efficiently. We address the issue of robustness of the optimal action in a decision making problem with respect to the prior model and the utility function. We discuss some general principles and apply novel computational strategies in the context of two relatively complex medical decision making problems.

View all citing articles on Scopus

View full text

MCMC-based local parametric sensitivity estimations

Abstract

Introduction

Section snippets

Local sensitivity estimations

Parametric classes

Applications

Discussion

Acknowledgements

Vision Res.

Comput. Statist. Data Anal.

J. Statist. Plann. Inference

J. Statist. Plann. Inference

Statistical Decision Theory and Bayesian Analysis

An overview of robust Bayesian analysis

Test

Robust Bayesian analysis

Markov chain Monte Carlo method and its application

The Statistician