Multi-user pdf estimation based criteria for adaptive blind separation of discrete sources

doi:10.1016/j.sigpro.2004.11.023

Signal Processing

Volume 85, Issue 5, May 2005, Pages 1059-1072

https://doi.org/10.1016/j.sigpro.2004.11.023 Get rights and content

Abstract

This paper deals with criteria for adaptive blind separation of discrete sources. The criteria are based on the estimation of the probability density function (pdf) of the recovered signal using a parametric model and the divergence of Kullback–Leibler to measure the similarities between the involved signals. Two strategies that guarantee the recovering of all sources are employed: the first one introduces a penalty when the sources are correlated and the second one constrains the filtering to an orthogonal global system response. Simulations are carried out to evaluate the performance of the criteria compared with existing blind methods in typical multi-user environments such as spatial and space-time processing.

Introduction

Blind source separation (BSS) has been gaining increasing attention in the signal processing community due to its wide applicability in many fields such as digital communications, biomedical engineering and financial data analysis among others [14].

Since the milestone work by Hérault et al. in 1985 [13], much effort has been done in order to construct suitable statistical criteria that reflect some known structural properties of the sources [19]. A common characteristic of many criteria is the use of higher order statistics (HOS), since second order statistics (SOS) are not sufficient to solve the source separation problem.

The information-theoretic approach has been introduced by Donoho in [11], who has treated the BSS problem from an entropy minimization viewpoint. Another well-known method to solve BSS problems is the use of contrast functions introduced by Comon [8], is any non-linear function which is invariant to permutation and scaling matrices, and attains its minimum value in correspondence of the mutual independence among the output components [19]. Clearly, the methods are based in some existing requirements for BSS such as linear and time-invariant mixture, as many sensors as sources and mutually independent sources.

These works have provided important results on the issue of necessary and sufficient conditions to blindly provide source separation. Despite the development of techniques that rely directly on HOS, some single user techniques, such as constant modulus (CM) and Shalvi–Weinstein criteria, have been proposed for BSS in a single-stage and multi-stage context [16], [20].

Multiple-input multiple-output (MIMO) systems have received a lot of attention during the last decade, with several multiple access systems emerging during that time. Through BSS techniques, several strategies have shown promising results in the context of multiuser detection. To cite a few references, [9], [17], [21] reveal the potential of equalization strategies to source separation.

Papadias proposed in [19], [20] a source separation approach that is based on the Shalvi–Weinstein criterion. The proposal is called multiuser kurtosis and consists of the kurtosis maximization, constrained to an orthogonal global response. It has been a great advance on the field of BSS because it has proved global convergence for an arbitrary number of users, which had not been done before.

We have previously proposed a source separation criterion based on the estimation of the probability density function (pdf) of the ideally recovered signals [5]. The criterion uses the knowledge of the pdf of transmitted signals, as in [25], to construct a parametric model that retains the statistical properties of the source signals. Thus, the separation matrix is optimized in order to maximize the similarities between the pdf of the recovered signal and the parametric model. For this sake, we use the Kullback–Leibler divergence to measure these similarities. The criterion is then similar to the one presented in [22]; however we consider a much simpler stochastic approximation to derive an adaptive algorithm.

Our objective in this work is to propose multi-user methods based on the pdf estimation based criterion and use different strategies to ensure that all sources are recovered given some requirements (e.g., linear mixture, discrete sources). Two strategies are used for this task: (i) an auxiliary criterion that penalizes the estimated sources (outputs) that are correlated and (ii) a filtering that constrains the obtained global response to be orthogonal. Also, applications in the context of multi-user processing are presented in order to evaluate the performance of the proposals comparing them with existing blind criteria.

The rest of the paper is organized as follows. The system model for memoryless and convolutive systems is described in details in Section 2. In Section 3, the pdf estimation based blind criterion is revisited in the context of SISO systems. In Section 4, the criteria for adaptive blind source separation are presented in detail, showing the different strategies for multi-user consideration. Simulation results that illustrate the performance of the proposals in some typical situations are presented in Section 5. Finally, our conclusions are stated in Section 6.

Section snippets

Blind source separation: general assumptions

In the model we made the following assumptions [26]:

(AS1)
K sources with independent and identically distributed (i.i.d.) and mutually independent zero mean discrete sequences $a_{k} (n)$ , $k = 1, \dots, K$ .
(AS2)
MIMO linear channel.
(AS3)
At least as many sensors as sources.
(AS4)
Noise is zero-mean, ergodic, stationary Gaussian sequence independent of $a_{k} (n)$ .

If we consider M sensors in the receiver we can represent, for instantaneous mixtures, the received signal at time instant

n

x (n) = Ha (n) + v (n),

where

a (n) = [\begin{matrix} a_{1} (n) & \dots & a_{K} (n) \end{matrix}]^{K}

is the

Review

In [5] was proposed a blind criterion in the context of SISO systems (blind deconvolution). This method is revisited here.

Let $w_{ideal}$ be an ideal zero-forcing linear equalizer, its output can be written as $y (n) = w_{ideal}^{H} x (n),$ where $x (n) = H a (n) + v (n),$ $H$ is the $N \times (N + L - 1)$ convolution matrix of the single-user channel with impulse response of length $L$ and $N$ is the length of the impulse response of the equalizer [2].

Then, using Eq. (20) in (19), it is possible to write: $y (n) = (H a (n) + v (n))^{H} w_{ideal} = a^{H} (n) H^{H} w_{ideal} + v$

Multi-user pdf estimation based criteria

In order to generalize the criterion to the multiple sources case, we have used the conditions for signal recovering, that are directly stated from the Shalvi–Weinstein (SW) criterion. These conditions can be written as [19]:

(C1)
$a_{k} (n)$ is i.i.d. and zero mean $(k = 1, \dots, K)$ ,
(C2)
$a_{k} (n)$ and $a_{q} (n)$ are statistically independent for $k \neq q$ and have the same pdf,
(C3)
$| κ [y_{k} (n)] | = | κ_{a} | (k = 1, \dots, K)$ ,
(C4)
$E {| y_{k} (n) |^{2}} = σ_{a}^{2} (k = 1, \dots, K)$ ,
(C5)
$E {y_{k} (n) y_{q}^{*} (n)} = 0, k \neq q$ ,

where

a_{k} (n)

is the sequence transmitted by the

k

th source,

κ_{a}

is the kurtosis,

σ_{a}^{2}

Computational simulations

This section demonstrates the performance of the proposed multi-user criteria, which is compared to blind separation criteria found in the literature.

We have chosen three common situations in multi-user processing in order to evaluate the performance over different environments. They are described in the sequel.

Conclusions and perspectives

In this paper we have presented a two blind criteria based on the estimation of the probability density function of the recovered signals for adaptive blind source separation.

The criterion of interference removal is based on the maximization of the similarity between the recovered signal probability density function and a parametric model that fits the system order and transmitted signal statistical characteristics. The Kullback–Leibler divergence is then used to derive the algorithm. Two

Acknowledgements

The authors would like to deeply thank the anonymous reviewers for their comments and suggestions that contributed a lot to the manuscript. They also thank Dr. Renato R. Lopes by his careful proofreading and comments.

References (26)

P. Comon
Independent component analysisa new concept?
Signal Process.
(1994)
N. Delfosse et al.
Adaptive blind separation of independent sourcesa deflation approach
Signal Process.
(1995)
J.K. Tugnait
Adaptive blind separation of convolutive mixtures of independent linear signals
Signal Process.
(1999)
C.M. Bishop
Neural Networks for Pattern Recognition
(1995)
C.C. Cavalcante, Neural prediction and probability density function estimation applied to blind equalization, Master's...
C.C. Cavalcante, On blind source separation: proposals and analysis of multi-user processing strategies, Ph.D. Thesis,...
C.C. Cavalcante et al.
Adaptive blind multiuser separation criterion based on log-likelihood maximisation
IEE Electron. Lett.
(2002)
C.C. Cavalcante et al.
A PDF estimation-based blind criterion for adaptive equalization
C.C. Cavalcante, J.C.M. Mota, J.M.T. Romano, Polynomial Expansion of the Probability Density Function About Gaussian...
F.R.P. Cavalcanti, J.M.T. Romano, Blind multiuser detection in space division multiple access systems, Ann. des...

Z. Ding et al.

Blind Equalization and Identification

(2001)

D. Donoho, On Minimum Entropy Deconvolution, Academic Press, New York, 1981, pp....

S. Haykin (Ed.), Unsupervised adaptive filtering, Source Separation, vol. I, Wiley, New York,...

Cited by (8)

A class of multi-modulus blind deconvolution algorithms using hyperbolic and Givens rotations for MIMO systems
2021, Signal Processing
Citation Excerpt :
However, in most practical cases, the channels are convolutive and the aforementioned methods [10,11,13] do not apply directly. The approach is also linked to the aspect of pdf-matching or alphabet matching [12] since the consideration of the multimodulus tries to capture more (statistical) information from the other aspects of the source distributions than the power (second-order) of constant modulus. Also, the Multi Modulus (MM) criterion does not take into account all statistics, it is only suitable for “small size” constellations.
This paper targets the blind deconvolution problem for multiple-input multiple-output communication systems, using small and moderate constellation’s size signals, i.e. PSK and QAM. We introduce four different blind deconvolution algorithms based on four different techniques. These algorithms come as a natural extension of the successful work done by Shah et al in 2018 for blind source separation (BSS). The first two methods are considered as two-step based methods, where the first one performs the BSS for the spatio-temporal system followed by a pairing and sorting phase. While the second is accomplished by performing a cascaded linear equalization, using one of the existing subspace-methods, followed by the BSS routine. The third method is based on the minimization of a hybrid cost function, and the last one is a deflation-based method. These solutions summarize the main possible paths that can be followed to extend any of the existing instantaneous de-mixing algorithms. Experimental results are provided to compare and highlight the unique characteristics of each of the four different methods.
Conditions for the existence of a generalization of Rényi divergence
2020, Physica A: Statistical Mechanics and its Applications
Citation Excerpt :
The interest on a different statistical divergence metric is motivated, among others, in applications related to optimization and statistical learning since more flexible functions and expressions may be suitable to larger classes of data and signals and lead to more efficient information recovery methods [16–18]. To cite a few, the usage of divergence metric has been considered in several domains such as statistics (including statistical physics) and learning [10,19–21], econometrics [22–26], digital communications [27–30], signal and image processing [31–33], biomedical processing [34]. Also, quantum versions of generalized divergences are of interest in the literature [35,36].
We give necessary and sufficient conditions for the existence of a generalization of Rényi divergence, which is defined in terms of a deformed exponential function. If the underlying measure $μ$ is non-atomic, we found that not all deformed exponential functions can be used in the generalization of Rényi divergence; a condition involving the deformed exponential function is provided. In the case $μ$ is purely atomic (the counting measure on the set of natural numbers), we show that any deformed exponential function can be used in the generalization.
Conditions for the existence of a generalization of Rényi divergence
2020, arXiv
Multiuser processing using blind source separation methods
2020, EW 2007 - 13th European Wireless Conference: Enabling Technologies for Wireless Multimedia Communications
Convex Combination of SISO Equalization and Blind Source Separation for MIMO Blind Equalization
2019, Wireless Personal Communications
A Bayesian approach to blind separation of mixed discrete sources by Gibbs sampling
2011, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

View all citing articles on Scopus

View full text

Multi-user pdf estimation based criteria for adaptive blind separation of discrete sources

Abstract

Introduction

Section snippets

Blind source separation: general assumptions

Review

Multi-user pdf estimation based criteria

Computational simulations

Conclusions and perspectives

Acknowledgements

Signal Process.

Signal Process.

Signal Process.

Neural Networks for Pattern Recognition

Adaptive blind multiuser separation criterion based on log-likelihood maximisation

IEE Electron. Lett.

A PDF estimation-based blind criterion for adaptive equalization

Blind Equalization and Identification