Abstract
Non-Gaussian factor analysis differs from ordinary factor analysis because of the distribution assumption on the factors which are modelled by univariate mixtures of Gaussians thus relaxing the classical normal hypothesis. From this point of view, the model can be thought of as a generalization of ordinary factor analysis and its estimation problem can still be solved via the maximum likelihood method. The focus of this work is to introduce, develop and explore a Bayesian analysis of the model in order to provide an answer to unresolved questions about the number of latent factors and simultaneously the number of mixture components to model each factor. The effectiveness of the proposed method is explored in a simulation study and in a real example of international exchange rates.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Arminger, G., Muthén, B.O.: A Bayesian approach to nonlinear latent variable models using the Gibbs sampler and the Metropolis-Hastings algorithm. Psychometrika 63, 271–300 (1998)
Attias, H.: Independent factor analysis. Neural Comput. 11, 803–851 (1999)
Besag, J.: A candidate’s formula: A curious result in Bayesian prediction. Biometrika 76, 183–183 (1989)
Browne, M.W.: Asymptotically distribution-free methods for the analysis covariance structures. British J. Math. Stat. Psychol. 37, 62–83 (1984)
Calò, D.G., Montanari, A., Viroli, C.: Model-based density estimation by independent factor analysis. In: Spiliopoulou, M., Kruse, R., Bolgelt, C., Nürnberger, A., Gaul, W. (eds.) From Data and Information Analysis to Knowledge Engineering. Studies in Classification, Data Analysis, and Knowledge Organization, pp. 166–173. Springer, Berlin (2006)
Comon, P.: Independent component analysis, a new concept? Signal Process. 36, 287–314 (1994)
Dellaportas, P., Papageorgiou, I.: Multivariate mixtures of normals with unknown number of components. Stat. Comput. 16, 57–68 (2006)
Fokoué, E., Titterington, D.M.: Mixtures of factor analysers. Bayesian estimation and inference in stochastic simulation. Mach. Learn. 50, 73–94 (2003)
Gelfand, A.E., Dey, D.: Bayesian model choice: Asymptotics and exact calculations. J. R. Stat. Soc. Ser. B 56, 501–514 (1994)
Ghahramani, Z., Hilton, G.E.: The EM algorithm for mixture of factor analyzers. Technical report CRG-TR-96-1, Department of Computer Science, University of Toronto, Canada (1997)
Giudici, P., Stanghellini, E.: Bayesian inference for graphical factor analysis models. Psychometrika 66(4), 577–592 (2001)
Green, P.J.: Contribution to the discussion of paper by Grenander and Miller (1994). J. R. Stat. Soc. B 56, 589–590 (1994)
Green, P.J.: Reversible jump Markov chain, Monte Carlo computation and Bayesian model determination. Biometrika 82, 711–732 (1995)
Harman, H.: Modern Factor Analysis. University of Chicago Press, Chicago (1960)
Hyvärinen, A.: Fast independent component analysis with noisy data using Gaussian moments. In: Proceedings of the International Symposium on Circuits and Systems, vol. V, pp. 57–51. Orlando, Florida (1999)
Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. Wiley, New York (2001)
Lee, S.Y.: A Bayesian approach to confirmatory factor analysis. Psychometrika 46, 153–160 (1981)
Lewis, S.M., Raftery, A.E.: Estimating Bayes’ factors via posterior simulation with the Laplace-Metropolis estimator. J. Am. Stat. Assoc. 92, 648–655 (1997)
Lopes, H.F., West, M.: Bayesian model assessment in factor analysis. Stat. Sinica 14, 41–67 (2004)
McLachlan, G.J., Peel, D.: Mixtures of factor analyzers. In: Langley, P. (ed.) Proceedings of the Seventeenth International Conference on Machine Learning, pp. 599–606. Morgan Kaufmann, San Francisco (2000)
Meng, X.L., Wong, W.H.: Simulating ratios of normalizing constants via a simple identity: A theoretical exploration. Stat. Sinica 6, 831–860 (1996)
Mooijaart, A.: Factor analysis for non-normal variables. Psychometrika 50, 323–342 (1985)
Moulines, E., Cardoso, J.P., Gassiat, E.: Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models. In: Proceedings of IEEE Conference on Acoustic, Speech and Signal Processing, vol. 5, pp. 3617–3620 (1997)
Newton, M.A., Raftery, A.E.: Approximate Bayesian inference with the weighted likelihood bootstrap. J. R. Stat. Soc. Ser. B 56, 3–48 (1994)
Richardson, S., Green, P.J.: On Bayesian analysis of mixtures with an unknown number of components (with discussion). J. R. Stat. Soc. Ser. B 59, 731–792 (1996)
Yalcin, I., Anemiya, Y.: Nonlinear factor analysis as a statistical method. Stat. Sci. 16(3), 275–294 (2001)
Yu, B., Mykland, P.: Looking at Markov samplers through CUSUM path plots: a simple diagnostic idea. Stat. Comput. 8, 275–286 (1998)
Viroli, C.: Choosing the number of factors in independent factor analysis model. Metodološki Zvezki, Advances in Methodology and Statistics 2, 219–229 (2005)
Viroli, C.: Fitting the independent factor analysis model using the MCMC algorithm. J. Stat. Comput. Simul. 77(9), 725–737 (2007)
West, M., Harrison, P.J.: Bayesian Forecasting and Dynamic Model. Springer, New York (1997)
Zhang, Z., Chan, K.L., Wu, Y., Chen, C.: Learning a multivariate Gaussian mixture models with the reversible jump MCMC algorithm. Stat. Comput. 14, 343–355 (2004)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Viroli, C. Bayesian inference in non-Gaussian factor analysis. Stat Comput 19, 451 (2009). https://doi.org/10.1007/s11222-008-9105-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11222-008-9105-8