Fitting a Mixture Distribution to a Variable Subject to Heteroscedastie Measurement Errors

Summary

In a structural errors-in-variables model the true regressors are treated as stochastic variables that can only be measured with an additional error. Therefore the distribution of the latent predictor variables and the distribution of the measurement errors play an important role in the analysis of such models. In this article the conventional assumptions of normality for these distributions are extended in two directions. The distribution of the true regressor variable is assumed to be a mixture of normal distributions and the measurement errors are again taken to be normally distributed but the error variances are allowed to be heteroscedastie. It is shown how an EM algorithm solely based on the error-prone observations of the latent variable can be used to find approximate ML estimates of the distribution parameters of the mixture. The procedure is illustrated by a Swiss data set that consists of regional radon measurements. The mean concentrations of the regions serve as proxies for the true regional averages of radon. The different variability of the measurements within the regions motivated this approach.

Appendix

Al Maximization of (7) with respect to α₁,…,α_k: To maximize (7) with respect to the proportion parameters α₁,…, α_m under the restriction $\sum\nolimits_{k = 1}^m {} {\alpha _k} = 1$ we maximize the function

$$La({\alpha _1}, \ldots ,{\alpha _m},{\lambda _L}) = \sum\limits_{k = 1}^m {} \sum\limits_{i = 1}^n {} p_k^{(c)}({W_i})\log {\alpha _k} + {\lambda _L}\left( {\sum\limits_{k = 1}^m {} {\alpha _k} - 1} \right).$$

Partial differentiation yields the conditions

• i) $- \lambda _L^{ - 1}\sum\limits_{i = 1}^n {} p_k^{(c)}({W_i}) = \alpha _k^{(n)}\;\;\;{\rm{for}}\;\;\;k = 1, \ldots ,m,$ which inserted into the restriction lead to the equation

• ii)
  $$\sum\limits_{i = 1}^n {} \sum\limits_{k = 1}^m {} p_k^{(c)}({W_i}) = - {\lambda _L}.$$

If we replace the weights $p_k^{(c)}({W_i})$ in ii) with their original expressions (6) it is easily seen that −λ_L = n and from there with this result plugged into i) the solutions (8) follow.

A2 Maximization of (7) with respect to θ_k and ς_k under a homoscedastic measurement error model: In the case of a homoscedastic measurement error model, that is ${U_i} \sim N(0,\sigma _U^2)$ for i = 1,…, n, the solutions $\theta _k^{(n)}$ and $\varsigma _k^{(n)}$ of the maximization problem (9) follow directly from the equations (10). For k = 1,…,m the next approximations $\theta _k^{(n)}$ and $\varsigma _k^{(n)}$, respectively $\varsigma _k^{2(n)}$, are given by

$$\begin{array}{*{20}{c}} {\theta _k^{(n)}}& = &{\frac{{\sum\nolimits_{i = 1}^n {{W_i}p_k^{(c)}({W_i})} }}{{\sum\nolimits_{i = 1}^n {p_k^{(c)}({W_i})} }}\;and\;\;\;\;\;\;\;\;\;\;\;} \\ {\zeta _k^{2(n)}}& = &{\frac{{\sum\nolimits_{i = 1}^n {{{({W_i} - \theta _k^{(n)})}^2}p_k^{(c)}({W_i})} }}{{\sum\nolimits_{i = 1}^n {p_k^{(c)}({W_i})} }} - \sigma _U^2.} \end{array}$$

A3 Computation of the matrices H_k(θ_k, ς_k): The matrices H_k(θ_k, ς_k) of the second derivatives of qk(θ_k, ς_k) used in the Newton approximation of the M Step of the algorithm are given by

$${H_k}({\theta _k},{\zeta _k}) = \left( {\begin{array}{*{20}{c}} {\tfrac{{{\partial ^2}{q_k}}}{{\partial \theta _k^2}}}&{\tfrac{{{\partial ^2}{q_k}}}{{\partial {\theta _k}{\partial _{\zeta k}}}}} \\ {\tfrac{{{\partial ^2}{q_k}}}{{{\partial _{\zeta k}}\partial {\theta _k}}}}&{\tfrac{{{\partial ^2}{q_k}}}{{\partial \zeta _k^2}}} \end{array}} \right)$$

with the elements

$$\begin{array}{lll} \frac{\partial^{2} q_{k}}{\partial \theta_{k}^{2}} &=&-\sum\limits_{i=1}^{n} \frac{1}{\varsigma_{k}^{2}+\sigma_{i}^{2}} p_{k}^{(c)}(W_{i}), \\ \frac{\partial^{2} q_{k}}{\partial\varsigma_{k} \partial\theta_{k}} &=&-2 \sum\limits_{i=1}^{n} \frac{\varsigma_{k}(W_{i}-\theta_{k})}{(\varsigma_{k}^{2}+\sigma_{i}^{2})^{2}} p_{k}^{(c)}(W_{i})\quad\text{and} \\ \frac{\partial^{2} q_{k}}{\partial \varsigma_{k}^{2}} &=&\sum\limits_{i=1}^{n} \frac{1}{(\varsigma_{k}^{2}+\sigma_{i}^{2})^{3}}(\varsigma_{k}^{4}-\sigma_{i}^{4}+(\sigma_{i}^{2}-3 \varsigma_{k}^{2})(W_{i}-\theta_{k})^{2}) p_{k}^{(c)}(W_{i}).\end{array}$$