Abstract
The variational Bayesian nonlinear blind source separation method introduced by Lappalainen and Honkela in 2000 is initialised with linear principal component analysis (PCA). Because of the multilayer perceptron (MLP) network used to model the nonlinearity, the method is susceptible to local minima and therefore sensitive to the initialisation used. As the method is used for nonlinear separation, the linear initialisation may in some cases lead it astray. In this paper we study the use of kernel PCA (KPCA) in the initialisation. KPCA is a rather straightforward generalisation of linear PCA and it is much faster to compute than the variational Bayesian method. The experiments show that it can produce significantly better initialisations than linear PCA. Additionally, the model comparison methods provided by the variational Bayesian framework can be easily applied to compare different kernels.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. J. Wiley, Chichester (2001)
Jutten, C., Karhunen, J.: Advances in nonlinear blind source separation. In: Proc. of the 4th Int. Symp. on Independent Component Analysis and Blind Signal Separation (ICA 2003), pp. 245–256 (2003); Invited paper in the special session on nonlinear ICA and BSS
Schölkopf, B., Smola, A., Müller, K.-R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10(5), 1299–1319 (1998)
Harmeling, S., Ziehe, A., Kawanabe, M., Müller, K.-R.: Kernel-based nonlinear blind source separation. Neural Computation 15(5), 1089–1124 (2003)
Lappalainen, H., Honkela, A.: Bayesian nonlinear independent component analysis by multi-layer perceptrons. In: Girolami, M. (ed.) Advances in Independent Component Analysis, pp. 93–121. Springer, Berlin (2000)
Valpola, H., Oja, E., Ilin, A., Honkela, A., Karhunen, J.: Nonlinear blind source separation by variational Bayesian learning. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences E86-A(3), 532–541 (2003)
Hinton, G.E., van Camp, D.: Keeping neural networks simple by minimizing the description length of the weights. In: Proc. of the 6th Ann. ACM Conf. on Computational Learning Theory, Santa Cruz, CA, USA, pp. 5–13 (1993)
MacKay, D.J.C.: Developments in probabilistic modelling with neural networks – ensemble learning. In: Neural Networks: Artificial Intelligence and Industrial Applications. Proc. of the 3rd Annual Symposium on Neural Networks, pp. 191–198 (1995)
Valpola, H., Östman, T., Karhunen, J.: Nonlinear independent factor analysis by hierarchical models. In: Proc. 4th Int. Symp. on Independent Component Analysis and Blind Signal Separation (ICA2003), Nara, Japan, pp. 257–262 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Honkela, A., Harmeling, S., Lundqvist, L., Valpola, H. (2004). Using Kernel PCA for Initialisation of Variational Bayesian Nonlinear Blind Source Separation Method. In: Puntonet, C.G., Prieto, A. (eds) Independent Component Analysis and Blind Signal Separation. ICA 2004. Lecture Notes in Computer Science, vol 3195. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30110-3_100
Download citation
DOI: https://doi.org/10.1007/978-3-540-30110-3_100
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23056-4
Online ISBN: 978-3-540-30110-3
eBook Packages: Springer Book Archive