Statistical convergence behavior of affine projection algorithms
Introduction
In noise and echo cancellation, equalization, and beamforming, adaptive filtering techniques are widely used. A very popular algorithm is the normalized least mean square (NLMS) algorithm [1], which is computationally very simple and easy to implement. Unfortunately, for highly colored input signals – with a covariance matrix that exhibits a large dynamic range of eigenvalues – this algorithm suffers from slow convergence. Over the past three decades, computationally efficient, rapidly converging adaptive filtering algorithms [2], [3], [4], [5], [6], [7] have been proposed to ameliorate this problem, which use multiple input vectors to compute the iterated direction of the adaptive filter. For example, Ozeki and Umeda [2] discovered the affine projection algorithm (APA) from the geometric viewpoint of affine subspace projections. Kratzer and Morgan [3] developed the partial rank algorithm (PRA), which addresses numerical conditioning. Sankaran and Beex [4] proposed NLMS with orthogonal correction factors (NLMS-OCF) based on the idea that the best improvement in weights occurs if successive input vectors are orthogonal to each other. Morgan and Kratzer [5] pointed out that all these algorithms, which were independently developed as a result of various interpretations and from different perspectives, can be viewed as a generalization of the NLMS algorithm that updates on the basis of multiple input signal vectors. Zhi [6], [7] presented an affine projection algorithm with direction error (AP-DE) to solve the nonconformity between the iterated direction of the adaptive filter and the direction caused by the iteration error. Fast APA versions have been proposed as well [8], [9], [10]. We will refer to the entire class of algorithms as affine projection algorithms. In this paper, we will use the formulation by Ozeki and Umeda [2] to analyze the convergence behavior of the APA class.
Some work has been done to analyze the convergence behavior of the APA class. Reference [11] presented a definition of the APA based on the direction vector, but the error signal is still driven by the input vector. In this paper, on the other hand, the error signal is driven by the direction vector. The direction vector behavior was investigated by means of carefully designed comparative experiments [12]. A quantitative analysis of the APA was presented in [13], [14], which analyzes the mean weight error and the mean-square error (MSE) based on an independent and identically distributed input signal, which was proposed earlier [15] to analyze the convergence behavior of the NLMS algorithm. Subsequently, a unified treatment of the MSE, tracking, and transient performance was provided [16], based on energy conservation arguments and without restricting the regressors to specific models. A statistical analytical model for predicting the stochastic behavior of the APA class has been provided [17], [18], [19] for autoregressive (AR) inputs. A statistical analysis model was shown to analyze the AP-DE algorithm for AR input signals [6], [7]. However, generally the AR input signal characteristics are not known. The above results motivate us to continue work on analyzing the APA class of algorithms for various types of input signals.
In this discussion follows the works in [13], [14] and [17], but is not limited to AR process. The new model is applicable to input processes that are AR as well as autoregressive-moving average (ARMA). We analyze the quantitative statistical properties of the direction vector and propose some useful assumptions. We then study the convergence behavior of the weight error and a closed-form expression for the MSE of the APA class of algorithms, appropriate for ARMA input signal models, is obtained. The steady-state weight behavior is also determined. Finally, we show that our analytical results predict simulation results quite well.
Notations used in this paper are fairly standard. Scalars are denoted by plain lowercase or uppercase letters. Vector quantities are denoted by boldface lowercase letters and matrix quantities by boldface capital letters. Throughout the paper, the following notations are also adopted:
(•)T Transpose of a vector or matrix;
(•)H Hermitian transpose of a vector or matrix;
(•)* Complex conjugate for scalars;
E[•] Expectation;
tr(•) Trace of a matrix;
|•| Absolute value of a quantity;
real(•) Real part of the complex number.
Section snippets
The APA class of algorithms
Fig. 1 show an adaptive filter used in the system identification mode. The wide sense stationary input process {xn}, which is zero-mean, and the corresponding measured output {dn}, possibly contaminated with the measurement noise {ɛn}, is measurable. The measurement noise {ɛn} is zero mean white noise and is denoted by complex value. The input process is converted into input vectors {xn}, via a tapped delay line (TDL), and are defined as
The objective is to estimate an N
Statistical properties of the direction vector
The convergence analysis of the APA class is done based on the following assumptions on the underlying system:
A1. The direction vectors {φn} are zero mean with covariance matrix where and . The latter eigenvectors are orthonormal, i.e. . That is, V is a unitary matrix.
A2. Define the transformed direction vector as
From (6) and (7), the covariance matrix equals where the elements in are independent of each
Weight error behavior
Under the above four assumptions, as will be seen, the convergence analysis becomes tractable. To analyze the convergence behavior of (3) with the step-size μ set equal to one, first, substitute (16a) into (3c)
Combining (16b) and (18), the adaptation equation in error form is obtained as
Pre-multiplication of (19) by, and respectively [11], using (3d) as and (5), results in
Mean-square error behavior
Based on (37), the covariance of the transformed weight error vector is given by
The second and third terms in (56) are determined based on (40), (8) and (45), which yields and
Now, consider the fourth term
Steady-state behavior
Because the element in the transformed weight error is the scale in the pth dimension, it is independent of the other dimensions. For the off-diagonal elements in (73) where p ≠ q, 1 ≤ p ≤ N, 1 ≤ q ≤ N. Assuming convergence, based on the expression in (50a), as n → ∞, the transformed weight error . So the expression in (85) goes to zero. Consequently, the off-diagonal elements in (73) also go to zero. Thus,
Comparison with simulation results
In this section, we compare the MSE learning curves from simulation with the derived modeled result as given in (73) and (77). The modeled steady-state error expression is shown in (94). The initial estimate for the weights is . The simulation results shown are obtained by ensemble averaging over 100 independent trials for each experiment. And the driven signal {zn} is denoted by complex.
Case 1a. Consider an AR (1) input signal given by with {zn} zero-mean white Gaussian
Conclusion
The direction vector {φn} is the error vector in estimating the present input vector from the m most recent past input vectors. The direction vector is orthogonal to the m most recent past input vectors. This is the reason that the APA class of algorithms exhibits a faster convergence rate than NLMS, especially for highly colored input vectors. The direction vector {φn} exhibits nice properties for predicting the actual APA weight behavior. In this paper, four assumptions are used that give
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under Grant no. 61201321 and 61471300, in part by Natural Science Basic Research Plan in Shaanxi Province of China under Grant no. 2014JQ8356 and in part by the Fundamental Research Funds for the Central Universities under Grant no. 3102014JCQ01063.
References (21)
Adaptive Filter Theory
(1991)- et al.
An adaptive filtering algorithm using an orthogonal projection to an affine subspace and its properties
Electron. Commun. Japan
(1984) - et al.
The partial-rank algorithm for adaptive beamforming
- et al.
Normalized LMS algorithm with orthogonal correction factors
- et al.
On a class of computationally efficient, rapidly converging, generalized NLMS algorithms
IEEE Signal Proc. Lett.
(1996) - et al.
On the convergence behavior of affine projection algorithm with direction error
Asian J. Control
(2014) - et al.
A new affine projection algorithm and its statistical behavior
Chin. J. Electron.
(2013) - et al.
Fast generalized affine projection algorithm
Int. J. Adapt. Control Signal Process.
(2000) - et al.
The fast affine projection algorithm
- et al.
New affine projection algorithms based on Gauss-Seidel method