A fixed-point nonlinear PCA algorithm for blind source separation

doi:10.1016/j.neucom.2005.05.009

Neurocomputing

Volume 69, Issues 1–3, December 2005, Pages 264-272

https://doi.org/10.1016/j.neucom.2005.05.009 Get rights and content

Abstract

This paper addresses the problem of blind source separation and presents a fixed-point nonlinear principal component analysis (NPCA) algorithm. It is a block-wise batch algorithm and gives an alternative perspective on existing adaptive online NPCA algorithms. Utilizing new activation functions that automatically satisfy a stability condition, the proposed algorithm can separate mixed signals with sub- and super-Gaussian source distributions. The efficiency is confirmed by extensive computer simulations on man-made sources as well as practical speech signals.

Introduction

The problem of blind source separation (BSS) has been studied by many authors in recent years (for a review, see e.g. [5]). In the noise-free instantaneous case, we assume that we have a sequence of m-dimensional measured vectors $x_{t} = [x_{1} (t), \dots, x_{m} (t)]^{T}$ , which are generated according to $x_{t} = {As}_{t}, t = 1, 2, \dots,$ where $A$ is an unknown $m \times n$ mixing matrix with full column rank ( $m ⩾ n$ ), $s_{t} = [s_{1} (t), \dots, s_{n} (t)]^{T}$ is a vector of independent source signals, all but perhaps one of them non-Gaussian. The task of BSS is to construct an $n \times m$ separating matrix $B$ given just the observation sequence, using the independence assumption of the source signals, such that the output vector $y_{t} = {Bx}_{t}$ recovers the n source signals up to scaling and permutation [3].

A variety of algorithms have been proposed for BSS [5], and they can be roughly divided into two major categories: adaptive online algorithms and batch algorithms. The former perform in a sample-by-sample mode, such as the natural/relative gradient algorithms [1], [2]. Batch algorithms are block-wise and they will not work until a block of data samples is received, for example, the fast fixed-point algorithms [4].

This paper first provides a unifying framework for the neural online nonlinear principal component analysis (NPCA) algorithms [6], [7], [10] and then presents a block fixed-point NPCA algorithm. This new algorithm utilizes nonlinear functions that satisfy a stability condition automatically; hence, it can separate mixed signals with sub- and super-Gaussian source distributions. Computer simulations on man-made sources as well as practical speech signals are reported to illustrate the effectiveness.

Section snippets

A fixed-point NPCA algorithm

Provided that the measured vector $x_{t}$ has already been followed by an $n \times m$ whitening filter $U$ such that the components of $v_{t} = {Ux}_{t}$ are unit variance and uncorrelated, the BSS problem is reduced to the search for an $n \times n$ orthonormal matrix $W$ and the total separating matrix is given as $B = WU$ . There exist in the literature several contrast functions for $W$ , such as the maximum likelihood contrast [8], the negentropy contrast [4] and the high-order statistics contrasts [2], [3]. This paper focuses on the

Computer simulations

In order to verify the effectiveness of the fixed-point NPCA algorithm (16), we consider the separation of the following four sets of source signals:

Case 1: five sub-Gaussian source signals (taken from [1] and [10]): $s_{t} = [sign (\cos (2 π 155 t))$ , $\sin (2 π 800 t),$ $\sin (2 π 90 t)$ , $\sin (2 π 300 t + 6 \cos (2 π 60 t))$ , $r (t)]^{T}$ , where $r (t)$ is a noise source uniformly distributed in $[- 1, + 1]$ .

Case 2: four super-Gaussian speech signals: two male speakers and two female speakers (available at //www.kecl.ntt.co.jp/icl/signal/sawada/webdemo/bssdemo.html

Conclusions

This paper provides a unifying framework for adaptive online NPCA learning rules, and presents a block fixed-point NPCA algorithm. The new algorithm applies the activation functions that automatically satisfy the stability condition; hence it can separate mixed signals with sub- and super-Gaussian source distributions. Experiment results on the man-made sources as well as the practical speech signals show that the fixed-point NPCA algorithm works more efficiently than the existing fixed-point

Acknowledgements

This work was supported by the major program of the National Natural Science Foundation of China under Grant 60496311 and by the Chinese Postdoctoral Science Foundation under Grant 2004035061.

Xiaolong Zhu received his B.S. degree in Measurement and Control Engineering in 1998, and his Ph.D. degree in Information and Communication Engineering in 2003, respectively, both from the Xidian University, Xi’an, China. From August 2003 to July 2005, he was a Postdoctoral Researcher in the Department of Automation, Tsinghua University, Beijing, China. Currently, he is with the Alcatel Shanghai Bell, Co., Ltd., Shanghai, China. His research interests include blind signal processsing, and

References (10)

P. Comon
Independent component analysis, a new concept?
Signal Process.
(1994)
Z. Shi et al.
A new fixed-point algorithm for independent component analysis
Neurocomputing
(2004)
L. Xu
Least mean square error reconstruction principle for self-organizing neural-nets
Neural Networks
(1993)
S. Amari, A. Cichocki, H. Yang, A new algorithm for blind signal separation, Advances in Neural Information Processing...
J.F. Cardoso et al.
Equivariant adaptive source separation
IEEE Trans. Signal Process.
(1996)

There are more references available in the full text version of this article.

Cited by (7)

An optimized EASI algorithm
2009, Signal Processing
This paper addresses the problem of blind source separation and presents a kind of optimized equivariant adaptive separation via independence (EASI) algorithms. According to the cumulant based approximation to the mutual information contrast function, the EASI learning rule is optimized by multiplying the symmetric part with an optimal time variant weight coefficient. Simulation results show the proposed optimized EASI algorithms outperform the existing algorithms in convergent speed and steady-state accuracy.
Extracting underlying meaningful features and canceling noise using independent component analysis for direct marketing
2007, Expert Systems with Applications
As the Internet spreads widely, it has become easier for companies to obtain and utilize valuable information on their customers. Nevertheless, many of them have difficulty in using the information effectively because of the huge amount of data from their customers that must to be analyzed. In addition, the data usually contains much noise due to anonymity of the Internet. Consequently, extracting the underlying meanings and canceling the noise of the collected customer data are crucial for the companies to implement their strategies for customer relationship management. As a novel solution, we propose the use of independent component analysis (ICA). ICA is a multivariate statistical tool which extracts independent components or sources of information, given only observed data that are assumed to be linear mixtures of some unknown sources. Moreover, ICA is able to reduce the dimension of the observed data, especially noisy variables. To validate the usefulness of ICA, we applied it to a real-world one-to-one marketing case. In this study, we used ICA as a preprocessing tool, and made a prediction for potential buyers using artificial neural networks (ANNs). We also applied PCA as a comparative model for ICA. The experimental results showed that ICA-preprocessed ANN outperformed all the comparative classifiers without preprocessing as well as PCA-preprocessed ANN.
A fast NPCA algorithm for online blind source separation
2006, Neurocomputing
This paper addresses the problem of blind source separation (BSS) and presents an optimum step size which makes the nonlinear principal component analysis (NPCA) cost function descend in the fastest way. By applying this step size in the self-stabilized NPCA algorithm, a fast NPCA algorithm is obtained. Computer simulations of online BSS show that the new algorithm works more efficiently than the existing least-mean-square (LMS)-type and recursive least-squares (RLS)-type NPCA algorithms.
Spacecraft electrical characteristics identification method based on PCA feature extraction and WPSVM
2015, Beijing Hangkong Hangtian Daxue Xuebao/Journal of Beijing University of Aeronautics and Astronautics
Application of PCA and ICA as preprocessing tools for eCRM
2014, Information (Japan)
Pseudo online independent component analysis for dynamical mixing using gradient optimization
2009, Journal of Advanced Computational Intelligence and Intelligent Informatics

View all citing articles on Scopus

Jimin Ye received his B.S. and M.S. degrees in Mathematics from the Shanxi Normal University, Xi’an, China, in 1990 and 1993, respectively. Since 1993, he has been with the School of Science, Xidian University, Xi’an, China, where he became an Associate Professor in 2002. He is currently working towards his Ph.D in Electronic Engineering at the Key Laboratory for Radar Signal Processing, Xidian University. His research interests include statistical signal processing, array signal processing and neural networks.

Xianda Zhang received his B.S. degree in Radar Engineering from Xidian University, Xi’an, China, in 1969 and his Ph.D. degree in Electrical Engineering from the Tohoku University, Sendai, Japan, in 1987. From August 1990 to August 1991 he was a Postdoctoral Researcher with the Department of Electrical and Computer Engineering, University of California at San Diego. Since 1992, he has been with the Department of Automation, Tsinghua University, Beijing, China, as a Professor. From April 1999 to March 2002, he was with the Key Laboratory for Radar Signal Processing, Xidian University, Xi’an, China, as a Specially Appointed Professor awarded by the Ministry of Education of China and the Cheung Kong Scholars Programme. His current research interests are signal processing and intelligent signal processing with applications in radar and communications.

View full text

LettersA fixed-point nonlinear PCA algorithm for blind source separation

Abstract

Introduction

Section snippets

A fixed-point NPCA algorithm

Computer simulations

Conclusions

Acknowledgements

Signal Process.

Neurocomputing

Neural Networks

Equivariant adaptive source separation

IEEE Trans. Signal Process.

Letters
A fixed-point nonlinear PCA algorithm for blind source separation