Blind separation of non-stationary sources using continuous density hidden Markov models

doi:10.1016/j.dsp.2013.03.012

Digital Signal Processing

Volume 23, Issue 5, September 2013, Pages 1549-1564

https://doi.org/10.1016/j.dsp.2013.03.012 Get rights and content

Abstract

Blind source separation (BSS) has attained much attention in signal processing society due to its ‘blind’ property and wide applications. However, there are still some open problems, such as underdetermined BSS, noise BSS. In this paper, we propose a Bayesian approach to improve the separation performance of instantaneous mixtures with non-stationary sources by taking into account the internal organization of the non-stationary sources. Gaussian mixture model (GMM) is used to model the distribution of source signals and the continuous density hidden Markov model (CDHMM) is derived to track the non-stationarity inside the source signals. Source signals can switch between several states such that the separation performance can be significantly improved. An expectation-maximization (EM) algorithm is derived to estimate the mixing coefficients, the CDHMM parameters and the noise covariance. The source signals are recovered via maximum a posteriori (MAP) approach. To ensure the convergence of the proposed algorithm, the proper prior densities, conjugate prior densities, are assigned to estimation coefficients for incorporating the prior information. The initialization scheme for the estimates is also discussed. Systematic simulations are used to illustrate the performance of the proposed algorithm. Simulation results show that the proposed algorithm has more robust separation performance in terms of similarity score in noise environments in comparison with the classical BSS algorithms in determined mixture case. Additionally, since the mixing matrix and the sources are estimated jointly, the proposed EM algorithm also works well in underdetermined case. Furthermore, the proposed algorithm converges quickly with proper initialization.

Section snippets

Fanglin Gu received the B.Sc. in communication engineering in 2008 from the PLA University of Science & Technology, Nanjing, China. He is currently pursuing the Ph.D. degree at the PLA University of Science & Technology. His research interests include blind source separation, tensor decomposition, and satellite communication.

References (39)

Y. Zhang et al.
A Gaussian mixture model for underdetermined independent component analysis
Signal Process.
(2006)
S. Sun et al.
Blind source separation with time series variational Bayes expectation maximization algorithm
Digital Signal Process.
(2012)
Q. Huang et al.
Temporally correlated source separation using variational Bayesian learning approach
Digital Signal Process.
(2007)
A.J. Bell et al.
An information-maximization approach to blind separation and blind deconvolution
Neural Comput.
(1995)
J.F. Cardoso et al.
Equivariant adaptive source separation
IEEE Trans. Signal Process.
(1996)
J.F. Cardoso et al.
Blind beamforming for non-Gaussian signals
IEE Proc. F
(1993)
A. Hyvärinen
Fast and robust fixed-point algorithms for independent component analysis
IEEE Trans. Neural Netw.
(1999)
S. Amari et al.
Adaptive blind signal processing—Neural network approaches
Proc. IEEE
(1998)
A. Hyvärinen
Gaussian moments for noisy independent component analysis
IEEE Signal Process. Lett.
(1999)
L.D. Lathauwer et al.
Fourth-order cumulant-based blind identification of underdetermined mixtures
IEEE Trans. Signal Process.
(2007)

X. Luciani et al.

Blind identification of underdetermined mixtures based on the characteristic function: The complex case

IEEE Trans. Signal Process.

(2011)

K.H. Knuth, A Bayesian approach to source separation, in: Proceedings of the 1st International Conference on...

K.H. Knuth, Informed source separation: A Bayesian tutorial, in: Proceedings of the 13th European Signal Processing...

É. Moulines, J.F. Cardoso, E. Gassiat, Maximum likelihood for blind separation and deconvolution of noisy signals using...

H. Snoussi, A.M. Djafari, Bayesian source separation with mixture of Gaussians prior for sources and Gaussian prior for...

L.C. Khor, W.L. Woo, S.S. Dlay, Non-sparse approach to underdetermined blind signal estimation, in: 2005 IEEE...

H. Snoussi et al.

Bayesian blind separation of generalized hyperbolic process in noisy and underdetermined mixture

IEEE Trans. Signal Process.

(2006)

S. Kim, C.D. Yoo, Underdetermined blind source separation based on generalized Gaussian distribution, in: Proceedings...

A. Belouchrani et al.

Maximum likelihood source separation for discrete sources

Cited by (11)

Blind separation of sparse sources from nonlinear mixtures
2021, Digital Signal Processing: A Review Journal
Citation Excerpt :
Consider a system that gets the inputs or sources, and instantaneously generates the outputs or observations using some mixture functions. The goal of blind source separation (BSS) is retrieving the sources from the observations in the considered system when: 1) the mixture functions are unknown; and 2) some basic information such as independency, uncorrelatedness, or sparseness are available about the sources [25,18,22,42]. In general, BSS can be categorized into linear and nonlinear depending on the mixture functions.
Blind source separation (BSS) has been studied well when the sources are sparse signals and their combinations are linear. Sparse component analysis (SCA) or dictionary learning algorithms propose well-known approaches for performing BSS in this model. In this study, we explore BSS when the sources are sparse signals, however, their combinations are nonlinear. The considered scenario can be adapted to several signal processing applications such as machine learning and system identification. To perform BSS, we first approximate the nonlinear mixture functions with the polynomial functions. Then, using an alternating approach, we estimate the sources and the coefficients of polynomial functions. The considered strategy is similar to one employed in dictionary learning algorithms for sparse representation of signals. In fact, in iterations where the sources are estimated, we cluster the signals, and in iterations where the coefficients of polynomial functions are estimated, we assign a polynomial manifold to each cluster. The identifiability issues of the considered nonlinear model are also discussed. Experimental results demonstrate the effectiveness of the proposed method relative to state-of-the-art methods.
High resolution sub-band decomposition underdetermined blind signal separation using virtual sensor based ICA method for low latency applications
2021, Digital Signal Processing: A Review Journal
Citation Excerpt :
Also, some algorithms have been introduced so far to solve the BSS problem which extremely depend on the training data of source signals [72]. The training data can be modeled using probabilistic models like Gaussian Mixture Model (GMM) [73,74], Hidden Markov Model (HMM) or factorial HMM [75,76] and Deep Neural Networks (DNNs) [77–80]. The aforementioned models are learned from the training data under the assumption that the sources appear in the mixture signals with the same energy level as they appear in the training dataset.
Underdetermined blind source separation (UBSS) objective is to recover the source signals from a number of mixtures without any information about the mixing system. Technologies such as teleconferencing, hearing aids and hands-free telephony require real time processing, as long delays are considered intolerable in interactive two-way communication. This is a major challenge in BSS, as algorithms generally require significant amounts of data (several seconds of data or more) to generate sufficient statistics for separation. In this paper, we propose a hybrid algorithm which combines sub-band decomposition and well-known Independent Component Analysis (ICA) based algorithm, Extended-Infomax. We first employ sub-band decomposition algorithm in sparse time domain to compensate low data efficiency of short time block lengths and estimate the mixing matrix. Subsequently, the proposed virtual sensor based underdetermined Extended-Infomax source model is used to estimate the source signals. As the separation evaluation results are so sensitive to the mixing matrix estimation outcome, achieving nearly optimum results in the first phase of the proposed algorithm significantly improves the results of source separation performance. A weighted version of k-plane clustering algorithm is derived to obtain the mixing coefficients. Experimental evaluations reveal the effectiveness of our proposed method over the state-of-the-art techniques.
Non-stationary sources separation based on maximum likelihood criterion using source temporal–spatial model
2018, Neurocomputing
Citation Excerpt :
However, those models are used to represent stationary signals. Meanwhile, authors of [26] combined HMM with GMM to present the non-stationary characteristic and statistical characteristic of non-stationary signals respectively, which obtains a good separation result. However, the complexity of the algorithm grows exponentially with a linear increase in model order.
In this paper, we propose a new non-stationary sources separation algorithm, which is referred to as autoregressive hidden Markov Gaussian process (AR-HMGP), in which the sources are non-stationary and temporally correlated. For the proposed algorithm, a generative model is employed to track the non-stationarity of the source where the temporal dependencies of sources are represented by autoregressive model (AR) and the distribution of the associated innovation process is described using non-stationary Gaussian process with hidden Markov model (HMM). We further explore the maximum likelihood (ML) method to estimate the parameters of the source model by using the expectation maximum (EM) algorithm. Our important findings reveal that (a) AR-HMGP algorithm outperforms the other three BSS algorithms for non-stationary sources separation, the instantaneous mixture system is also well corroborated with the effectiveness of our algorithm; (b) both independent and dependent non-stationary sources have been successfully separated; (c) the proposed algorithm is robust with respect to noise, while the other three algorithms are not.
A new method for Markovian adaptation of the non-Markovian queueing system using the hidden Markov model
2019, Algorithms
Frame-Based Variational Bayesian Learning for Independent or Dependent Source Separation
2018, IEEE Transactions on Neural Networks and Learning Systems
Super-Gaussian BSS Using Fast-ICA with Chebyshev–Pade Approximant
2018, Circuits, Systems, and Signal Processing

View all citing articles on Scopus

Hang Zhang received the B.Sc. in satellite communication from Institute of Communication Engineering Nanjing, and the M.Sc. in communication engineering from Southeast University, China. She is now a professor of the PLA University of Science & Technology. Her main research interests include blind source separation, cognitive radio network, and satellite communication.

Desheng Zhu received B.Sc. in communication engineering from Institute of Communication Engineering Nanjing, China. He is now a researcher and professor in communication ministry of the Chinese Peopleʼs Liberation Army, has published about 30 discourse, and 4 teaching materials, and gained two national awards of science and technology. His research interests include radio communication, satellite communication and network, etc.

View full text

Blind separation of non-stationary sources using continuous density hidden Markov models

Abstract

Section snippets

Signal Process.

Digital Signal Process.

Digital Signal Process.

An information-maximization approach to blind separation and blind deconvolution

Neural Comput.

Equivariant adaptive source separation

IEEE Trans. Signal Process.

Blind beamforming for non-Gaussian signals

IEE Proc. F

Fast and robust fixed-point algorithms for independent component analysis

IEEE Trans. Neural Netw.

Adaptive blind signal processing—Neural network approaches

Proc. IEEE

Gaussian moments for noisy independent component analysis

IEEE Signal Process. Lett.

Fourth-order cumulant-based blind identification of underdetermined mixtures

IEEE Trans. Signal Process.

Blind identification of underdetermined mixtures based on the characteristic function: The complex case

IEEE Trans. Signal Process.

Bayesian blind separation of generalized hyperbolic process in noisy and underdetermined mixture

IEEE Trans. Signal Process.

Maximum likelihood source separation for discrete sources