Bayesian inference using a noninformative prior for linear Gaussian random coefficient regression with inhomogeneous within-class variances

Abstract

A Bayesian inference for a linear Gaussian random coefficient regression model with inhomogeneous within-class variances is presented. The model is motivated by an application in metrology, but it may well find interest in other fields. We consider the selection of a noninformative prior for the Bayesian inference to address applications where the available prior knowledge is either vague or shall be ignored. The noninformative prior is derived by applying the Berger and Bernardo reference prior principle with the means of the random coefficients forming the parameters of interest. We show that the resulting posterior is proper and specify conditions for the existence of first and second moments of the marginal posterior. Simulation results are presented which suggest good frequentist properties of the proposed inference. The calibration of sonic nozzle data is considered as an application from metrology. The proposed inference is applied to these data and the results are compared to those obtained by alternative approaches.

Appendix

Subsequently we prove Theorem 2, i.e., propriety of the reference posterior. To this end, and similarly to Yang and Chen (1995), the additional interim variables ${\varvec{\beta }}_i, i=1, \ldots , K$ are used as follows. We employ the relation

$$\begin{aligned}&l({\varvec{\theta }},{\varvec{\gamma }^2},{\varvec{\sigma }^2}; {\varvec{y}}_1, \ldots ,{\varvec{y}}_K)= \prod _{i=1}^K \frac{e^{-\frac{1}{2} \left( {\varvec{y}}_i - {\varvec{X}}_i {\varvec{\theta }}\right) ^T {\varvec{V}}_i^{-1} \left( {\varvec{y}}_i - {\varvec{X}}_i {\varvec{\theta }}\right) }}{\sqrt{ (2 \pi )^{n_i} det\left( {\varvec{V}}_i \right) }} \nonumber \\&\quad \propto \int \psi ({\varvec{\theta }},{\varvec{\beta }}_1, \ldots , {\varvec{\beta }}_K,{\varvec{\gamma }^2},{\varvec{\sigma }^2}; {\varvec{y}}_1, \ldots ,{\varvec{y}}_K) \text {d}{\varvec{\beta }}_1 \cdots \text {d}{\varvec{\beta }}_K , \end{aligned}$$

(26)

where

$$\begin{aligned}&\psi ({\varvec{\theta }},{\varvec{\beta }}_1, \ldots ,{\varvec{\beta }}_K,{\varvec{\gamma }^2},{\varvec{\sigma }^2}; {\varvec{y}}_1, \ldots ,{\varvec{y}}_K) \nonumber \\&\quad =\left( \prod _{i=1}^{K} \sigma _i^{-n_i} e^{- \frac{1}{2 \sigma _i^2} \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) ^T \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) } \right) \nonumber \\&\qquad \times \,\left( \prod _{l=1}^{p} \gamma _l^{-K} \right) \left( \prod _{i=1}^{K} e^{- \frac{1}{2} \left( {\varvec{\beta }}_i - {\varvec{\theta }}\right) ^T {\varvec{\varLambda }}^{-1} \left( {\varvec{\beta }}_i - {\varvec{\theta }}\right) } \right) , \end{aligned}$$

(27)

and ${\varvec{V}}_i=\sigma _i^2 {\varvec{I}}+ {\varvec{X}}_i {\varvec{\varLambda }}{\varvec{X}}_i^T$. In the following, we consider

$$\begin{aligned}&f\left( {\varvec{\theta }},{\varvec{\beta }}_1, \ldots ,{\varvec{\beta }}_K,{\varvec{\gamma }^2},{\varvec{\sigma }^2}; {\varvec{y}}_1, \ldots ,{\varvec{y}}_K\right) \nonumber \\&\quad =\pi \left( {\varvec{\theta }},{\varvec{\gamma }^2},{\varvec{\sigma }^2}\right) \psi \left( {\varvec{\theta }},{\varvec{\beta }}_1, \ldots ,{\varvec{\beta }}_K,{\varvec{\gamma }^2},{\varvec{\sigma }^2}; {\varvec{y}}_1, \ldots ,{\varvec{y}}_K\right) , \end{aligned}$$

(28)

where $\pi ({\varvec{\theta }},{\varvec{\gamma }^2},{\varvec{\sigma }^2})$ denotes the reference prior (18).

We will show that the function $f({\varvec{\theta }},{\varvec{\beta }}_1, \ldots ,{\varvec{\beta }}_K,{\varvec{\gamma }^2}, {\varvec{\sigma }^2}; {\varvec{y}}_1, \ldots ,{\varvec{y}}_K)$ is proper, which implies the theorem. For ease of notation $f({\varvec{\theta }}; {\varvec{y}}_1, \ldots ,{\varvec{y}}_K)$, for example, will denote $f({\varvec{\theta }},{\varvec{\beta }}_1, \ldots ,{\varvec{\beta }}_K,{\varvec{\gamma }^2},{\varvec{\sigma }^2}; {\varvec{y}}_1, \ldots ,{\varvec{y}}_K)$ after integrating out all variables but ${\varvec{\theta }}$. We start by integrating out ${\varvec{\gamma }^2}$ and, in using Corollary 1, we get

$$\begin{aligned}&f\left( {\varvec{\theta }},{\varvec{\beta }}_1, \ldots ,{\varvec{\beta }}_K,{\varvec{\sigma }^2}; {\varvec{y}}_1, \ldots ,{\varvec{y}}_K\right) \nonumber \\&\quad \le c \int \psi \left( {\varvec{\theta }},{\varvec{\beta }}_1, \ldots ,{\varvec{\beta }}_K,{\varvec{\gamma }^2},{\varvec{\sigma }^2}; {\varvec{y}}_1, \ldots ,{\varvec{y}}_K\right) \nonumber \\&\qquad \times \sqrt{\varPhi ({\varvec{\sigma }^2})} \left( \prod _{l=1}^{p} \gamma _l^{-3/2} \right) \left( \prod _{i=1}^{K} \sigma _i^{-2} \right) \text {d}{\varvec{\gamma }^2}\nonumber \\&\quad = c \sqrt{\varPhi ({\varvec{\sigma }^2})} \left( \prod _{i=1}^K \sigma _i^{-(n_i+2)} e^{- \frac{1}{2 \sigma _i^2} \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) ^T \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) } \right) \nonumber \\&\qquad \times \int \left( \prod _{l=1}^{p} \gamma _l^{-(K+3/2)} \right) \left( \prod _{i=1}^{K} e^{- \frac{1}{2} \left( {\varvec{\beta }}_i - {\varvec{\theta }}\right) ^T {\varvec{\varLambda }}^{-1} \left( {\varvec{\beta }}_i - {\varvec{\theta }}\right) } \right) \text {d}{\varvec{\gamma }^2}\nonumber \\&\quad \propto \sqrt{\varPhi ({\varvec{\sigma }^2})} \left( \prod _{i=1}^{K} \sigma _i^{-(n_i+2)} e^{- \frac{1}{2 \sigma _i^2} \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) ^T \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) } \right) \nonumber \\&\qquad \times \prod _{l=1}^{p} \left( \sum _{i=1}^K ({\varvec{\beta }}_i-{\varvec{\theta }})_l^2 \right) ^{-(K-1/2)/2} , \end{aligned}$$

(29)

where for evaluating the integral in the last step $K \ge 1$ is required which is satisfied due to our assumption $K\ge 2$. In the next step we integrate out ${\varvec{\sigma }^2}$, yielding

$$\begin{aligned}&f\left( {\varvec{\theta }},{\varvec{\beta }}_1, \ldots ,{\varvec{\beta }}_K; {\varvec{y}}_1, \ldots ,{\varvec{y}}_K\right) \nonumber \\&\quad \le c_1 \left( \prod _{l=1}^{p} \left( \sum _{i=1}^K ({\varvec{\beta }}_i-{\varvec{\theta }})_l^2 \right) ^{-(K-1/2)/2} \right) \nonumber \\&\qquad \times \int \sqrt{\varPhi ({\varvec{\sigma }^2})} \left( \prod _{i=1}^{K} \sigma _i^{-(n_i+2)} e^{- \frac{1}{2 \sigma _i^2} \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) ^T \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) } \right) \text {d}{\varvec{\sigma }^2}\nonumber \\&\quad \le c_1 \left( \prod _{l=1}^{p} \left( \sum _{i=1}^K ({\varvec{\beta }}_i-{\varvec{\theta }})_l^2 \right) ^{-(K-1/2)/2} \right) \nonumber \\&\qquad \times \int \left( 1 + \varPhi ({\varvec{\sigma }^2}) \right) \left( \prod _{i=1}^{K} \sigma _i^{-(n_i+2)} e^{- \frac{1}{2 \sigma _i^2} \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) ^T \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) } \right) \text {d}{\varvec{\sigma }^2}\nonumber \\&\quad = c_1 \left( \prod _{l=1}^{p} \left( \sum _{i=1}^K ({\varvec{\beta }}_i-{\varvec{\theta }})_l^2 \right) ^{-(K-1/2)/2} \right) \nonumber \\&\qquad \times \int \left( 1 + \sum _{i_1=1}^K \cdots \sum _{i_p=1}^K \prod _{l=1}^p \sigma _{i_l}^{-1} \right) \nonumber \\&\qquad \times \,\left( \prod _{i=1}^{K} \frac{ e^{- \frac{1}{2 \sigma _i^2} \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) ^T \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) } }{\sigma _i^{n_i+2}} \right) \text {d}{\varvec{\sigma }^2}\end{aligned}$$

(30)

for some $c_1>0$. For the integral in the last two lines of (30) we get

$$\begin{aligned}&\int \left( 1 + \sum _{i_1=1}^K \cdots \sum _{i_p=1}^K \prod _{l=1}^p \sigma _{i_l}^{-1} \right) \left( \prod _{i=1}^{K} \frac{ e^{- \frac{1}{2 \sigma _i^2} \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) ^T \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) } }{\sigma _i^{n_i+2}} \right) \text {d}{\varvec{\sigma }^2}\nonumber \\&\quad = \prod _{i=1}^{K} \left( \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) ^T \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) \right) ^{-n_i/2} \nonumber \\&\qquad + \sum _{i_1=1}^K \cdots \sum _{i_p=1}^K \prod _{i=1}^{K} \left( \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) ^T \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) \right) ^{-(n_i+m_i(i_1,\ldots ,i_p))/2} , \end{aligned}$$

(31)

where $0 \le m_i(i_1,\ldots ,i_p) \le p$. Since for $i=1, \ldots , K$

$$\begin{aligned} \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) ^T \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) = \left( {\varvec{\beta }}_i-\widehat{{\varvec{\beta }}}_i \right) ^T {\varvec{X}}_i^T {\varvec{X}}_i \left( {\varvec{\beta }}_i-\widehat{{\varvec{\beta }}}_i \right) + \delta _i^2 \end{aligned}$$

(32)

holds with $\delta _i\ne 0$, where both $\widehat{{\varvec{\beta }}}_i$ and $\delta _i$ depend only on ${\varvec{X}}_i$ and ${\varvec{y}}_i$, a constant $c_2>0$ exists, so that

$$\begin{aligned}&\left( \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) ^T \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) \right) ^{-(n_i+m_i(i_1,\ldots ,i_p))/2} \nonumber \\&\quad \le c_2 \left( \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) ^T \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) \right) ^{-n_i/2} \end{aligned}$$

(33)

holds for all such terms entering in (31). Using (30), we hence get

$$\begin{aligned}&f\left( {\varvec{\theta }},{\varvec{\beta }}_1, \ldots ,{\varvec{\beta }}_K; {\varvec{y}}_1, \ldots ,{\varvec{y}}_K\right) \nonumber \\&\quad \le c_3 \left( \prod _{l=1}^{p} \left( \sum _{i=1}^K ({\varvec{\beta }}_i-{\varvec{\theta }})_l^2 \right) ^{-(K-1/2)/2} \right) \nonumber \\&\qquad \times \,\left( \prod _{i=1}^{K} \left( \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) ^T \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) \right) ^{-n_i/2} \right) \end{aligned}$$

(34)

for some $c_3 >0$. Next we introduce the notation

$$\begin{aligned} \sum _{i=1}^K ({\varvec{\beta }}_i-{\varvec{\theta }})_l^2 = K (\zeta _l-\theta _l)^2 +(K-1) s_l^2 , \end{aligned}$$

(35)

where $\zeta _l=\sum _{i=1}^K ({\varvec{\beta }}_i)_l /K$, $\theta _l=({\varvec{\theta }})_l$, and $s_l^2=\sum _{i=1}^K (({\varvec{\beta }}_i)_l-\zeta _l)^2 /(K-1)$. Integrating (34) over ${\varvec{\theta }}$ yields

$$\begin{aligned}&f\left( {\varvec{\beta }}_1 ,\ldots ,{\varvec{\beta }}_K; {\varvec{y}}_1, \ldots ,{\varvec{y}}_K\right) \nonumber \\&\quad \le c_4 \left( \prod _{l=1}^{p} \frac{\int \left( 1 + u^2 \right) ^{-(K-1/2)/2} \text {d}u}{s_l^{(K-3/2)}} \right) \nonumber \\&\qquad \,\times \left( \prod _{i=1}^{K} \left( \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) ^T \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) \right) ^{-n_i/2} \right) \nonumber \\&\quad \propto \left( \prod _{l=1}^{p} s_l^{-(K-3/2)} \right) \left( \prod _{i=1}^{K} \left( \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) ^T \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) \right) ^{-n_i/2} \right) \end{aligned}$$

(36)

for some $c_4>0$, where the relation $K>3/2$ is required so that the integrals remain finite. In order to show that (36) is proper we proceed as follows. We rearrange $({\varvec{\beta }}_1^T, \ldots , {\varvec{\beta }}_K^T)^T$ as $(({\varvec{\beta }}_1)_1,({\varvec{\beta }}_2)_1 \ldots , ({\varvec{\beta }}_K)_1, ({\varvec{\beta }}_1)_2, ({\varvec{\beta }}_2)_2\ldots )^T$, and in writing the latter as $({\varvec{v}}_1^T, \ldots , {\varvec{v}}_p^T)^T$ we decompose each ${\varvec{v}}_l$ according to ${\varvec{v}}_l = \zeta _l {\varvec{1}}+ {\varvec{z}}_l$ with ${\varvec{1}}=(1, \ldots ,1)^T$, where ${\varvec{z}}_l^T {\varvec{1}}=0$ and $\zeta _l={\varvec{v}}_l^T {\varvec{1}}/K$. Hence $s_l^2={\varvec{z}}_l^T {\varvec{z}}_l /(K-1)$ holds. The inequality

$$\begin{aligned} \int _{{\varvec{z}}_l^T {\varvec{z}}_l \le 1} s_l^{-(K-3/2)} ~ \text {d}{\varvec{z}}_l < \infty \end{aligned}$$

(37)

follows immediately from writing ${\varvec{z}}_l=z_1 {\varvec{b}}_1 + \cdots + z_{K-1} {\varvec{b}}_{K-1}$ for some orthonormal basis ${\varvec{b}}_1, \ldots {\varvec{b}}_{K-1}$ of the subspace in $\mathbb {R}^K$ orthogonal to ${\varvec{1}}\in \mathbb {R}^K$, and changing to spherical coordinates with $\text {d}z_1 \cdots \text {d}z_{K-1} = r^{K-2} \text {d}r \text {d}\varOmega _{K-1}$, where $\text {d}\varOmega _{K-1}$ denotes the volume element on the unit sphere in dimension $K-1$. Since $s_l=r$, (37) remains finite because $\int _{0 \le \widetilde{r} \le 1} \widetilde{r}^{-(K- 3/2-(K-2))} \text {d}\widetilde{r} < \infty$.

Furthermore, we observe that

$$\begin{aligned} s_l^{-(K-3/2)} \le (K-1)^{(K/2-3/4)} \quad \text {for all} ~~ {\varvec{z}}_l^T {\varvec{z}}_l >1 \end{aligned}$$

(38)

holds and, by using

$$\begin{aligned} \left( \prod _{i=1}^{K} \left( \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) ^T \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) \right) ^{-n_i/2} \right) \le c_5 \end{aligned}$$

(39)

for some $c_5>0$ [cf. (32)], we conclude that (36) is proper, as

$$\begin{aligned} \int \prod _{i=1}^{K} \left( \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) ^T \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) \right) ^{-n_i/2} \text {d}{\varvec{\beta }}_1 \cdots \text {d}{\varvec{\beta }}_K < \infty \end{aligned}$$

(40)

holds. The latter condition is satisfied since, in again using (32) and utilizing the fact that ${\varvec{X}}_i^T {\varvec{X}}_i$ is positive definite, we can write

$$\begin{aligned}&\int \left( \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) ^T \left( {\varvec{y}}_i-{\varvec{X}}_i {\varvec{\beta }}_i \right) \right) ^{-n_i/2} \text {d}{\varvec{\beta }}_i \nonumber \\&\quad = \int \left( \left( {\varvec{\beta }}_i-\widehat{{\varvec{\beta }}}_i \right) ^T {\varvec{X}}_i^T {\varvec{X}}_i \left( {\varvec{\beta }}_i-\widehat{{\varvec{\beta }}}_i \right) + \delta _i^2 \right) ^{-n_i/2} \text {d}{\varvec{\beta }}_i \nonumber \\&\quad \propto \int \left( \widetilde{{\varvec{\beta }}}_i^T \widetilde{{\varvec{\beta }}}_i + \delta _i^2 \right) ^{-n_i/2} \text {d}\widetilde{{\varvec{\beta }}}_i \nonumber \\&\quad \propto \int \left( r^2 + \delta _i^2 \right) ^{-n_i/2} r^{(p-1)} \text {d}r \text {d}\varOmega _p , \end{aligned}$$

(41)

which remains finite for $n_i >p$ and hence concludes the proof of Theorem 2.