Objective Bayesian testing on the common mean of several normal distributions under divergence-based priors

Abstract

This paper considers the problem of testing on the common mean of several normal distributions. We propose a solution based on a Bayesian model selection procedure in which no subjective input is considered. We construct the proper priors for testing hypotheses about the common mean based on measures of divergence between competing models. This method is called the divergence-based priors (Bayarri and García-Donato in J R Stat Soc B 70:981–1003, 2008). The behavior of the Bayes factors based DB priors is compared with the fractional Bayes factor in a simulation study and compared with the existing tests in two real examples.

Appendices

Appendix 1: Proof of Theorem 1

Consider model $M_1$,

$$\begin{aligned} M_1: f_1(\mathbf{x}\vert \mu _{0}, \sigma _1,\ldots ,\sigma _{k})= \prod _{i=1}^k N(\mathbf{x}_i \vert \mu _0,\sigma _i^2), \pi ^N(\sigma _1,\ldots ,\sigma _{k})=\prod _{i=1}^k \sigma _i^{-1}\quad \end{aligned}$$

(21)

and model $M_2$

$$\begin{aligned} M_2: f_2(\mathbf{x}\vert \mu , \sigma _1,\ldots ,\sigma _{k})= \prod _{i=1}^k N(\mathbf{x}_i \vert \mu ,\sigma _i^2), \pi ^N(\mu ,\sigma _1,\ldots ,\sigma _{k})=\prod _{i=1}^k \sigma _i^{-1}.\qquad \end{aligned}$$

(22)

Let ${\varvec{\theta }}=\mu$ and ${\varvec{\nu }}=(\sigma _1,\ldots ,\sigma _k)$. Then the Kullback–Leibler directed divergence $KL[({\varvec{\theta }}_0,{\varvec{\nu }}):({\varvec{\theta }},{\varvec{\nu }})]$ is given by

$$\begin{aligned} KL[({\varvec{\theta }}_0,{\varvec{\nu }}):({\varvec{\theta }},{\varvec{\nu }})]= & {} \int \log \left( { \prod _{i=1}^k N(\mathbf{x}_i \vert \mu ,\sigma _i^2) \over \prod _{i=1}^k N(\mathbf{x}_i \vert \mu _0,\sigma _i^2)}\right) \left( \prod _{i=1}^k N(\mathbf{x}_i \vert \mu ,\sigma _i^2)\right) d\mathbf{x}\nonumber \\= & {} {1\over 2}\sum _{i=1}^{k} {n_i(\mu _0-\mu )^2\over \sigma _i^2}. \end{aligned}$$

Moreover, the Kullback–Leibler directed divergence $KL[({\varvec{\theta }},{\varvec{\nu }}):({\varvec{\theta }}_0,{\varvec{\nu }})]$ is given by

$$\begin{aligned} KL[({\varvec{\theta }},{\varvec{\nu }}):({\varvec{\theta }}_0,{\varvec{\nu }})]= & {} \int \log \left( { \prod _{i=1}^k N(\mathbf{x}_i \vert \mu _0,\sigma _i^2) \over \prod _{i=1}^k N(\mathbf{x}_i \vert \mu ,\sigma _i^2)}\right) \left( \prod _{i=1}^k N(\mathbf{x}_i \vert \mu _0,\sigma _i^2)\right) d\mathbf{x}\nonumber \\= & {} {1\over 2}\sum _{i=1}^{k} {n_i(\mu _0-\mu )^2\over \sigma _i^2}. \end{aligned}$$

Therefore the sum divergence measure is

$$\begin{aligned} D^S[({\varvec{\theta }},{\varvec{\theta }}_0)\vert {\varvec{\nu }})]= & {} \sum _{i=1}^{k} {n_i(\mu _0-\mu )^2\over \sigma _i^2}. \end{aligned}$$

Additionally, since $KL[({\varvec{\theta }}_0,{\varvec{\nu }}):({\varvec{\theta }},{\varvec{\nu }})]$ and $KL[({\varvec{\theta }},{\varvec{\nu }}):({\varvec{\theta }}_0,{\varvec{\nu }})]$ is the same, the minimum divergence measure is the same as the sum divergence measure.

We take the effective sample size $n^*=n$. This effective sample size is a key part in the definition of DB prior, ensuring that the information incorporated into the prior is equivalent to the information contained in an imaginary sample size (Bayarri and García-Donato 2008; García-Donato and Sun 2007). The idea of using unitary sample information for priors in model selection has been proved to be successful for many authors (see Smith and Spiegelhalter 1980; Kass and Wasserman 1995).

Since

$$\begin{aligned} {\bar{D}}^S[({\varvec{\theta }},{\varvec{\theta }}_0)\vert {\varvec{\nu }})]= \sum _{i=1}^{k}{n_i\over n} \left( {\mu _0-\mu \over \sigma _i}\right) ^2, \end{aligned}$$

and thus,

$$\begin{aligned} c_S(q,{\varvec{\nu }})= & {} \int (1+{\bar{D}}^S[({\varvec{\theta }},{\varvec{\theta }}_0) \vert {\varvec{\nu }}])^{-q}\pi ^N({\varvec{\theta }}\vert {\varvec{\nu }})\hbox {d}{\varvec{\theta }}\\= & {} \int _{-\infty }^{\infty }\left[ 1+ \sum _{i=1}^{k}{n_i\over n} \left( {\mu _0-\mu \over \sigma _i}\right) ^2\right] ^{-q}\hbox {d}\mu \\\le & {} \int _{-\infty }^{\infty } (c_{11}\theta _1^2+2c_{12}\theta _1+c_{22})^{-1} d\theta _1 < \infty , \end{aligned}$$

if $q> {1\over 2}$. Thus, the conditional sum DB prior with $q_*^S=1$ is given by

$$\begin{aligned} \pi ^D(\mu \vert \sigma _1,\ldots ,\sigma _k) =c_S^{-1} \left[ 1+\sum _{i=1}^{k}{n_i\over n}\left( {\mu _0-\mu \over \sigma _i}\right) ^2\right] ^{-1}, \end{aligned}$$

where $c_S\equiv c_S (q_*^S,\sigma _1,\ldots ,\sigma _k) =\int _{-\infty }^{\infty } [1+\sum _{i=1}^{k}{n_i\over n}({\mu _0-\mu \over \sigma _i})^2]^{-1}\hbox {d}\mu .$ This proves Theorem 1. $\square$

Appendix 2: Minimal training sample size

Let $x_m(l)=(x_{11},x_{21},x_{22},\ldots , x_{k1},x_{k2})$ denote minimal training sample. Under model $M_1$, $x_{ij}$ is independent and identically distributed $N(\mu _0,\tau _i^2)$, and under $M_2$, they are independent and identically distributed $N(\mu ,\sigma _i^2)$. Thus, under model $M_1$, the marginal density is given by

$$\begin{aligned} m_1(\mathbf{z})= & {} \int _0^{\infty }\ldots \int _0^{\infty } \left( \sqrt{2\pi }\right) ^{-2k+1}\sigma _1^{-2} \exp \left\{ -{(x_{11}-\mu _0)^2 \over 2\sigma _1^2} \right\} \\&\times \left( \prod _{i=2}^{k}\sigma _i^{-3}\right) \exp \left\{ -\sum _{i=2}^{k} \sum _{j=1}^{2} {(x_{ij}-\mu _0)^2 \over 2\sigma _i^2} \right\} \hbox {d}\sigma _1\ldots \hbox {d}\sigma _k\\= & {} \left( \sqrt{2\pi }\right) ^{-2k+1} 2^{-k}\left[ {(x_{11}-\mu _0)^2\over 2}\right] ^{-{1\over 2}} \prod _{i=2}^{k} \left[ {\sum _{j=1}^2(x_{ij}-\mu _0)^2 \over 2}\right] ^{-1}. \end{aligned}$$

Therefore, the $m_1(\mathbf{z})$ goes to infinity if $x_{11}$ is equal to $\mu _0$.

Let $x_m(l)=(x_{11},x_{12},\ldots , x_{k1},x_{k2})$ denote the minimal training sample. Under model $M_1$, $x_{ij}$ is independent and identically distributed $N(\mu _0,\tau _i^2)$, and under $M_2$, they are independent and identically distributed $N(\mu ,\sigma _i^2)$. Thus, under model $M_1$,

$$\begin{aligned} m_1(\mathbf{z})= \left( \sqrt{2\pi }\right) ^{-2k} \prod _{i=1}^{k} \left[ {\sum _{j=1}^2(x_{ij}-\mu _0)^2}\right] ^{-1}<\infty . \end{aligned}$$

Further, under model $M_2$,

$$\begin{aligned} m_2(\mathbf{z})= & {} \left( \sqrt{2\pi }\right) ^{-2k} \int _{-\infty }^{\infty }\prod _{i=1}^{k} \left[ {S_i^2+2({\bar{x}}_i-\mu _0)^2 }\right] ^{-1}\hbox {d}\mu \\= & {} \left( \sqrt{2\pi }\right) ^{-2k} \left( \prod _{i=1}^{k} S_i^2\right) \int _{-\infty }^{\infty }\prod _{i=1}^{k} \left[ {1+{2({\bar{x}}_i-\mu _0)^2\over S_i^2}}\right] ^{-1}\hbox {d}\mu \\\le & {} \left( \sqrt{2\pi }\right) ^{-2k} \left( \prod _{i=1}^{k} S_i^2 \right) \int _{-\infty }^{\infty } \left[ {1+{2({\bar{x}}_i-\mu _0)^2\over S_i^2}}\right] ^{-1}\hbox {d}\mu . \end{aligned}$$

Therefore, this integral is finite because it is a form of the Student’s t density. The minimal training sample size 2k is proved. $\square$

Appendix 3: Proof of Theorem 2

The likelihood function under model $M_1$ is

$$\begin{aligned} L_1(\sigma _1,\ldots ,\sigma _k\vert \mathbf {x} ) = \left( \sqrt{2\pi }\right) ^{-n} \left( \prod _{i=1}^{k}\sigma _i^{-n_i}\right) \exp \left\{ -\sum _{i=1}^k {1\over 2\sigma _i^2}\left[ S_i^2+n_i({\bar{x}}_i-\mu _0)^2\right] \right\} , \end{aligned}$$

(23)

where $S_i^2=\sum _{j=1}^{n_i} (x_{ij}-{\bar{x}}_i)^2$ and ${\bar{x}}_i =\sum _{j=1}^{n_i} x_{ij}/n_i, i=1,\ldots ,k$. Additionally, under model $M_1$, the reference prior for $(\sigma _1,\ldots ,\sigma _k)$ is

$$\begin{aligned} \pi _1^N(\sigma _1,\ldots ,\sigma _k) \propto \prod _{i=1}^k \sigma _i^{-1}. \end{aligned}$$

(24)

Then from the likelihood (23) and the reference prior (24), $m_1(\mathbf {x}$ under model $M_1$ is given by

$$\begin{aligned} m_1(\mathbf {x})= & {} \int _{0}^{\infty }\ldots \int _{0}^{\infty } L_1(\sigma _1,\ldots ,\sigma _k\vert \mathbf {x}) \pi _{1}^{N}(\sigma _1,\ldots ,\sigma _k)\hbox {d}\sigma _1 \ldots \hbox {d}\sigma _k \nonumber \\= & {} \left( \sqrt{2\pi }\right) ^{-n}2^{-k} \prod _{i=1}^k {\varGamma }\left[ n_i \over 2\right] \left\{ {S_i^2+n_i({\bar{x}}_i-\mu _0)^2\over 2}\right\} ^{-{n_i\over 2}}. \end{aligned}$$

(25)

For model $M_2$, the reference prior for $(\mu ,\sigma _1,\ldots ,\sigma _k)$ is

$$\begin{aligned} \pi ^N (\mu ,\sigma _1,\ldots ,\sigma _k) \propto \prod _{i=1}^k \sigma _i^{-1}. \end{aligned}$$

(26)

The likelihood function under model $M_2$ is

$$\begin{aligned} L_2(\mu ,\sigma _1,\ldots ,\sigma _k \vert \mathbf {x}) \!=\!\left( \sqrt{2\pi }\right) ^{-n} \left( \prod _{i=1}^{k}\sigma _i^{-n_i}\right) \exp \left\{ -\sum _{i=1}^k {1\over 2\sigma _i^2}\left[ S_i^2+n_i({\bar{x}}_i-\mu )^2\right] \right\} \end{aligned}$$

(27)

Thus, from the likelihood (27) and the reference prior (26), the $m_2(\mathbf {x})$ under model $M_2$ is given as follows.

$$\begin{aligned} m_2(\mathbf {x})= & {} \int _{-\infty }^{\infty }\int _{0}^{\infty }\ldots \int _{0}^{\infty } L_2(\mu ,\sigma _1,\ldots ,\sigma _k\vert \mathbf {x}) \pi _{2}^{N}(\mu ,\sigma _1,\ldots ,\sigma _k) \hbox {d}\sigma _1 \ldots \hbox {d}\sigma _k \hbox {d}\mu \nonumber \\= & {} \left( \sqrt{2\pi }\right) ^{-n}2^{-k} \prod _{i=1}^k {\varGamma }\left[ n_i \over 2\right] \int _{-\infty }^{\infty } \prod _{i=1}^k \left\{ {S_i^2+n_i({\bar{x}}_i-\mu )^2\over 2}\right\} ^{-{n_i\over 2}}\hbox {d}\mu . \end{aligned}$$

(28)

Therefore, $B_{21}^N$ is given by

$$\begin{aligned} B_{21} ^{N}(\mathbf {x}) = \int _{-\infty }^{\infty } \prod _{i=1}^k \left\{ {S_i^2+n_i({\bar{x}}_i-\mu )^2\over S_i^2+n_i({\bar{x}}_i-\mu _0)^2}\right\} ^{-{n_i\over 2}}\hbox {d}\mu . \end{aligned}$$

(29)

Further, $\pi ^N(\mu \vert \sigma _1,\ldots ,\sigma _k)=1$. Hence, Theorem 2 is proved. $\square$