Skip to main content
Springer Nature Link
Log in

Post-nonlinear Independent Component Analysis by Variational Bayesian Learning

  • Conference paper
  • First Online:
Independent Component Analysis and Blind Signal Separation (ICA 2004)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 3195))

Abstract

Post-nonlinear (PNL) independent component analysis(ICA) is a generalisation of ICA where the observations are assumed to have been generated from independent sources by linear mixing followed by component-wise scalar nonlinearities. Most previous PNL ICA algorithms require the post-nonlinearities to be invertible functions. In this paper, we present a variational Bayesian approach to PNL ICA that also works for non-invertible post-nonlinearities. The method is based on a generative model with multi-layer perceptron (MLP) networks to model the post-nonlinearities. Preliminary results with a difficult artificial example are encouraging.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. J. Wiley, Chichester (2001)

    Book  Google Scholar 

  2. Hyvärinen, A., Pajunen, P.: Nonlinear independent component analysis: Existence and uniqueness results. Neural Networks 12(3), 429–439 (1999)

    Article  Google Scholar 

  3. Taleb, A., Jutten, C.: Source separation in post-nonlinear mixtures. IEEE Trans. on Signal Processing 47(10), 2807–2820 (1999)

    Article  Google Scholar 

  4. Jutten, C., Karhunen, J.: Advances in nonlinear blind source separation. In: Proc. of the 4th Int. Symp. on Independent Component Analysis and Blind Signal Separation (ICA 2003), pp. 245–256 (2003) (invited paper in the special session on nonlinear ICA and BSS)

    Google Scholar 

  5. Ilin, A., Achard, S., Jutten, C.: Bayesian versus constrained structure approaches for source separation in post-nonlinear mixtures. In: Proc. International Joint Conference on Neural Networks, IJCNN 2004 (2004) (to appear)

    Google Scholar 

  6. Lappalainen, H., Honkela, A.: Bayesian nonlinear independent component analysis by multi-layer perceptrons. In: Girolami, M. (ed.) Advances in Independent Component Analysis, pp. 93–121. Springer, Berlin (2000)

    Chapter  Google Scholar 

  7. Valpola, H., Oja, E., Ilin, A., Honkela, A., Karhunen, J.: Nonlinear blind source separation by variational Bayesian learning. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences E86-A(3), 532–541 (2003)

    Google Scholar 

  8. Taleb, A., Jutten, C.: Batch algorithm for source separation in postnonlinear mixtures. In: Proc. Int. Workshop on Independent Component Analysis and Signal Separation (ICA 1999) (Aussois, France), pp. 155–160 (1999)

    Google Scholar 

  9. Hinton, G.E., van Camp, D.: Keeping neural networks simple by minimizing the description length of the weights. In: Proc. of the 6th Ann. ACM Conf. on Computational Learning Theory, Santa Cruz, CA, USA, pp. 5–13 (1993)

    Google Scholar 

  10. MacKay, D.J.C.: Developments in probabilistic modelling with neural networks – ensemble learning. In: Neural Networks: Artificial Intelligence and Industrial Applications. Proc. of the 3rd Annual Symposium on Neural Networks, pp. 191–198 (1995)

    Google Scholar 

  11. Lappalainen, H., Miskin, J.: Ensemble learning. In: Girolami, M. (ed.) Advances in Independent Component Analysis, pp. 75–92. Springer, Berlin (2000)

    Chapter  Google Scholar 

  12. Attias, H.: Independent factor analysis. Neural Computation 11(4), 803–851 (1999)

    Article  Google Scholar 

  13. Lappalainen, H.: Ensemble learning for independent component analysis. In: Proc. Int. Workshop on Independent Component Analysis and Signal Separation (ICA 1999), Aussois, France, pp. 7–12 (1999)

    Google Scholar 

  14. Miskin, J., MacKay, D.J.C.: Ensemble learning for blind source separation. In: Roberts, S., Everson, R. (eds.) Independent Component Analysis: Principles and Practice, pp. 209–233. Cambridge University Press, Cambridge (2001)

    Chapter  Google Scholar 

  15. Penny, W., Everson, R., Roberts, S.: ICA: model order selection and dynamic source models. In: Roberts, S., Everson, R. (eds.) Independent Component Analysis: Principles and Practice, pp. 299–314. Cambridge University Press, Cambridge (2001)

    Chapter  Google Scholar 

  16. Valpola, H., Karhunen, J.: An unsupervised ensemble learning method for nonlinear dynamic state-space models. Neural Computation 14(11), 2647–2692 (2002)

    Article  Google Scholar 

  17. Honkela, A.: Approximating nonlinear transformations of probability distributions for nonlinear independent component analysis. In: Proc. International Joint Conference on Neural Networks, IJCNN 2004 (2004) (to appear)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Neural Networks Research Centre, Helsinki University of Technology, P.O. Box 5400, FI-02015 HUT, Espoo, Finland

    Alexander Ilin & Antti Honkela

Authors
  1. Alexander Ilin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antti Honkela
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dept. of Architecture and Computer Technology, University of Granada, Spain

    Carlos G. Puntonet  & Alberto Prieto  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2004 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Ilin, A., Honkela, A. (2004). Post-nonlinear Independent Component Analysis by Variational Bayesian Learning. In: Puntonet, C.G., Prieto, A. (eds) Independent Component Analysis and Blind Signal Separation. ICA 2004. Lecture Notes in Computer Science, vol 3195. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30110-3_97

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-30110-3_97

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-23056-4

  • Online ISBN: 978-3-540-30110-3

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics