Abstract
The problem of inference in Bayesian Normal mixture models is known to be difficult. In particular, direct Bayesian inference (via quadrature) suffers from a combinatorial explosion in having to consider every possible partition of n observations into k mixture components, resulting in a computation time which is O(k n). This paper explores the use of discretised parameters and shows that for equal-variance mixture models, direct computation time can be reduced to O(D k n k), where relevant continuous parameters are each divided into D regions. As a consequence, direct inference is now possible on genuine data sets for small k, where the quality of approximation is determined by the level of discretisation. For large problems, where the computational complexity is still too great in O(D k n k) time, discretisation can provide a convergence diagnostic for a Markov chain Monte Carlo analysis.
Similar content being viewed by others
References
Basford K.E., McLachlan G.M., and York M.G. 1997. Modelling the distribution of stamp paper thickness via finite normal mixtures: The 1872 Hidalgo stamp issue of Mexico revisited. Journal of Applied Statistics 24: 169–179.
Bowmaker J.K., Jacobs G.H., Spiegelhalter D.J., and Mollon J.D. 1985. Two types of trichromatic squirrel monkey share a pigment in the red-green region. Vision Research 25: 1937–1946.
Casella G., Mengersen K.L., Robert C.P., and Titterington D.M. 2000 Perfect slice samplers for mixtures of distributions. Journal of the Royal Statistical Society B, 64: 777–790.
Dempster A.P., Laird N.M., and Rubin D. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B 39: 1–38.
Diebolt J. and Robert C.P. 1994. Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society B 56: 363–375.
Hall P. and Wand M.P 1996. On the accuracy of Binned kernel density estimators. Journal of Multivariate Analysis 56: 165–184.
Hobert J.P., Robert C.P., and Titterington D.M. 1999. On perfect simulation for some mixtures of distributions. Statistics and Computing 9: 287–298.
Jain S. and Neal R.M. 2000. A split-merge Markov Chain Monte Carlo procedure for the Dirichlet process mixture model. Technical Report 2003, Department of Statistics, University of Toronto.
Lehmann E.L. 1983. Theory of Point Estimation. Wiley, New York
Lunn D.J, Thomas A., Best N., and Spiegelhalter D. 2000. WinBUGS?A Bayesian modelling framework: Concepts, structure and extensibility. Statistics and Computing 10: 325–337.
McLachlan G.J. and Peel D. 2000. Finite Mixture Models. Wiley, New York.
Naylor J.C. and Smith A.F.M. 1982. Applications of a method for the efficient computation of posterior distributions. Applied Statistics 31: 214–225.
Reilly P.M. 1976. The numerical computation of posterior distributions in Bayesian statistical inference. Applied Statistics 25: 201–209.
Robert C.P. 1996. Mixtures of distributions: Inference and estimation. In: Gilks W.R., Richardson S., and Spiegelhalter D.J. (Eds.), Markov Chain Monte Carlo in Practice. Chapman and Hall, London, pp. 441–464.
Roeder K. 1990. Density estimation with confidence sets exemplified by superclusters and voids in the galaxies. Journal of the American Statistical Association 85: 617–624
Titterington D.M., Smith A.F.M., and Makov U.E. 1985. Statistical Analysis of Finite Mixture Distributions. Wiley, Chichester.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Brewer, M.J. Discretisation for inference on normal mixture models. Statistics and Computing 13, 209–219 (2003). https://doi.org/10.1023/A:1024214615828
Issue Date:
DOI: https://doi.org/10.1023/A:1024214615828