Elsevier

Applied Mathematics and Computation

Volume 270, 1 November 2015, Pages 511-526
Applied Mathematics and Computation

Statistical convergence behavior of affine projection algorithms

https://doi.org/10.1016/j.amc.2015.08.054Get rights and content

Abstract

Class of algorithms referring to the affine projection algorithms (APA) applies updates to the weights in a direction that is orthogonal to the most recent input vectors. This speeds up the convergence of the algorithm over that of the normalized least mean square (NLMS) algorithm, especially for highly colored input processes. In this paper a new statistical analysis model is used to analyze the APA class of algorithms with unity step size. Four assumptions are made, which are based on the direction vector for the APA class. Under these assumptions, deterministic recursive equations for the weight error and for the mean-square error are derived. We also analyze the steady-state behavior of the APA class. The new model is applicable to input processes that are autoregressive as well as autoregressive-moving average, and therefore is useful under more general conditions than previous models for prediction of the mean square error of the APA class. Simulation results are provided to corroborate the analytical results.

Introduction

In noise and echo cancellation, equalization, and beamforming, adaptive filtering techniques are widely used. A very popular algorithm is the normalized least mean square (NLMS) algorithm [1], which is computationally very simple and easy to implement. Unfortunately, for highly colored input signals – with a covariance matrix that exhibits a large dynamic range of eigenvalues – this algorithm suffers from slow convergence. Over the past three decades, computationally efficient, rapidly converging adaptive filtering algorithms [2], [3], [4], [5], [6], [7] have been proposed to ameliorate this problem, which use multiple input vectors to compute the iterated direction of the adaptive filter. For example, Ozeki and Umeda [2] discovered the affine projection algorithm (APA) from the geometric viewpoint of affine subspace projections. Kratzer and Morgan [3] developed the partial rank algorithm (PRA), which addresses numerical conditioning. Sankaran and Beex [4] proposed NLMS with orthogonal correction factors (NLMS-OCF) based on the idea that the best improvement in weights occurs if successive input vectors are orthogonal to each other. Morgan and Kratzer [5] pointed out that all these algorithms, which were independently developed as a result of various interpretations and from different perspectives, can be viewed as a generalization of the NLMS algorithm that updates on the basis of multiple input signal vectors. Zhi [6], [7] presented an affine projection algorithm with direction error (AP-DE) to solve the nonconformity between the iterated direction of the adaptive filter and the direction caused by the iteration error. Fast APA versions have been proposed as well [8], [9], [10]. We will refer to the entire class of algorithms as affine projection algorithms. In this paper, we will use the formulation by Ozeki and Umeda [2] to analyze the convergence behavior of the APA class.

Some work has been done to analyze the convergence behavior of the APA class. Reference [11] presented a definition of the APA based on the direction vector, but the error signal is still driven by the input vector. In this paper, on the other hand, the error signal is driven by the direction vector. The direction vector behavior was investigated by means of carefully designed comparative experiments [12]. A quantitative analysis of the APA was presented in [13], [14], which analyzes the mean weight error and the mean-square error (MSE) based on an independent and identically distributed input signal, which was proposed earlier [15] to analyze the convergence behavior of the NLMS algorithm. Subsequently, a unified treatment of the MSE, tracking, and transient performance was provided [16], based on energy conservation arguments and without restricting the regressors to specific models. A statistical analytical model for predicting the stochastic behavior of the APA class has been provided [17], [18], [19] for autoregressive (AR) inputs. A statistical analysis model was shown to analyze the AP-DE algorithm for AR input signals [6], [7]. However, generally the AR input signal characteristics are not known. The above results motivate us to continue work on analyzing the APA class of algorithms for various types of input signals.

In this discussion follows the works in [13], [14] and [17], but is not limited to AR process. The new model is applicable to input processes that are AR as well as autoregressive-moving average (ARMA). We analyze the quantitative statistical properties of the direction vector and propose some useful assumptions. We then study the convergence behavior of the weight error and a closed-form expression for the MSE of the APA class of algorithms, appropriate for ARMA input signal models, is obtained. The steady-state weight behavior is also determined. Finally, we show that our analytical results predict simulation results quite well.

Notations used in this paper are fairly standard. Scalars are denoted by plain lowercase or uppercase letters. Vector quantities are denoted by boldface lowercase letters and matrix quantities by boldface capital letters. Throughout the paper, the following notations are also adopted:

  • (•)T Transpose of a vector or matrix;

  • (•)H Hermitian transpose of a vector or matrix;

  • (•)* Complex conjugate for scalars;

  • E[•] Expectation;

  • tr(•) Trace of a matrix;

  • |•| Absolute value of a quantity;

  • real(•) Real part of the complex number.

Section snippets

The APA class of algorithms

Fig. 1 show an adaptive filter used in the system identification mode. The wide sense stationary input process {xn}, which is zero-mean, and the corresponding measured output {dn}, possibly contaminated with the measurement noise {ɛn}, is measurable. The measurement noise {ɛn} is zero mean white noise and is denoted by complex value. The input process is converted into input vectors {xn}, via a tapped delay line (TDL), and are defined as xn=[xnxn1xnN+1]T

The objective is to estimate an N

Statistical properties of the direction vector

The convergence analysis of the APA class is done based on the following assumptions on the underlying system:

A1. The direction vectors {φn} are zero mean with covariance matrix Rφ=E[φnφnH]=VΛVHwhere Λ=diag[λ1λ2λN] and V=[υ1υ2υN]. The latter eigenvectors are orthonormal, i.e. VHV=I. That is, V is a unitary matrix.

A2. Define the transformed direction vector as un=VHφn

From (6) and (7), the covariance matrix equals Ru=E[ununH]=Λwhere the elements in un=[un,1un,2un,N]T are independent of each

Weight error behavior

Under the above four assumptions, as will be seen, the convergence analysis becomes tractable. To analyze the convergence behavior of (3) with the step-size μ set equal to one, first, substitute (16a) into (3c) wn+1=wn+φnxnHφnHφnw˜n+φnɛn*φnHφn

Combining (16b) and (18), the adaptation equation in error form is obtained as w˜n+1=w˜nφnxnHφnHφnw˜nφnɛn*φnHφn

Pre-multiplication of (19) by{φnH},{xnH} and {Xn1H} respectively [11], using (3d) as xnH=φnH+α^nHXn1H and (5), results in φnHw˜n+1=φnHw˜nxnHw

Mean-square error behavior

Based on (37), the covariance of the transformed weight error vector is given by cov(ν˜n+1)=cov(ν˜n)cov(ν˜n)E[ununHunHun]E[ununHunHun]cov(ν˜n)+E[ununHunHunν˜nν˜nHununHunHun]E[(IununHunHun)ν˜nunHϑnunHun]E[ϑn*ununHunν˜nH(IununHunHun)]+E[unϑn*unHunϑnunHunHun]

The second and third terms in (56) are determined based on (40), (8) and (45), which yields cov(ν˜n)E[ununHunHun]=cov(ν˜n)NΛ(F2)i=1Nλiand E[ununHunHun]cov(ν˜n)=NΛ(F2)i=1Nλicov(ν˜n)

Now, consider the fourth term E[ununHunHunν˜nν˜nHununH

Steady-state behavior

Because the element {ν˜n,p},(1pN) in the transformed weight error {ν˜n} is the scale in the pth dimension, it is independent of the other dimensions. For the off-diagonal elements in (73) [cov(ν˜n)]pq=E(ν˜n,pν˜n,q*)=E(ν˜n,p)E(ν˜n,q*)=ρ˜n,pρ˜n,q*where pq, 1 ≤ pN, 1 ≤ qN. Assuming convergence, based on the expression in (50a), as n → ∞, the transformed weight error ρ˜n,p0. So the expression in (85) goes to zero. Consequently, the off-diagonal elements in (73) also go to zero. Thus,

Comparison with simulation results

In this section, we compare the MSE learning curves from simulation with the derived modeled result as given in (73) and (77). The modeled steady-state error expression is shown in (94). The initial estimate for the weights is w0=0. The simulation results shown are obtained by ensemble averaging over 100 independent trials for each experiment. And the driven signal {zn} is denoted by complex.

Case 1a. Consider an AR (1) input signal given by xn=0.95xn1+zn with {zn} zero-mean white Gaussian

Conclusion

The direction vector {φn} is the error vector in estimating the present input vector from the m most recent past input vectors. The direction vector is orthogonal to the m most recent past input vectors. This is the reason that the APA class of algorithms exhibits a faster convergence rate than NLMS, especially for highly colored input vectors. The direction vector {φn} exhibits nice properties for predicting the actual APA weight behavior. In this paper, four assumptions are used that give

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grant no. 61201321 and 61471300, in part by Natural Science Basic Research Plan in Shaanxi Province of China under Grant no. 2014JQ8356 and in part by the Fundamental Research Funds for the Central Universities under Grant no. 3102014JCQ01063.

References (21)

  • S. Haykin

    Adaptive Filter Theory

    (1991)
  • K. Ozeki et al.

    An adaptive filtering algorithm using an orthogonal projection to an affine subspace and its properties

    Electron. Commun. Japan

    (1984)
  • S.G. Kratzer et al.

    The partial-rank algorithm for adaptive beamforming

  • S.G. Sankaran et al.

    Normalized LMS algorithm with orthogonal correction factors

  • D.R. Morgan et al.

    On a class of computationally efficient, rapidly converging, generalized NLMS algorithms

    IEEE Signal Proc. Lett.

    (1996)
  • Y.F. Zhi et al.

    On the convergence behavior of affine projection algorithm with direction error

    Asian J. Control

    (2014)
  • Y.F. Zhi et al.

    A new affine projection algorithm and its statistical behavior

    Chin. J. Electron.

    (2013)
  • S.G. Sankaran et al.

    Fast generalized affine projection algorithm

    Int. J. Adapt. Control Signal Process.

    (2000)
  • S.L. Gay et al.

    The fast affine projection algorithm

  • F. Albu et al.

    New affine projection algorithms based on Gauss-Seidel method

There are more references available in the full text version of this article.

Cited by (0)

View full text