Elsevier

Signal Processing

Volume 172, July 2020, 107534
Signal Processing

Adaptive filtering with quantized minimum error entropy criterion

https://doi.org/10.1016/j.sigpro.2020.107534Get rights and content

Highlights

  • A variant of the quantized minimum error entropy (QMEE) algorithm is proposed.

  • The mean square convergence performance analysis for QMEE algorithm is given.

  • The theoretical values of the steady-state EMSE for QMEE algorithm are validated by simulations.

  • QMEE algorithm is applied in electroencephalo (EEG) de-noising task.

Abstract

Adaptive filtering algorithms have been widely used in many areas, among which the minimum error entropy (MEE) algorithm is a superior choice, due to its excellent performance in the non-Gaussian noise situations. However, the computational complexity of the MEE algorithm is expensive, which leads to the computational bottlenecks, especially for large-scale datasets. In order to address the problem, we propose an adaptive filtering algorithm based on the quantized minimum error entropy (QMEE) criterion with an online quantization method, named QMEE algorithm. Moreover, we analyze the transient behavior characteristic and derive an approximate analytical expression for the steady-state excess mean square error (EMSE) based on the Taylor expansion. The extensive simulation results in linear modeling and electroencephalogram (EEG) denoising task demonstrate that the proposed method can outperform other robust adaptive filtering algorithms.

Introduction

Over the past decades, the fundamental of adaptive filtering has been extensively developed [1], [2], [3] and used in wide areas, such as channel equalization and noise cancellation. Many useful adaptive filter algorithms have been proposed. For example, the well-known least mean square (LMS) is based on the popular minimum mean square error (MMSE) criterion [4], [5], [6], which can perform well under Gaussian noise assumption. In many practical applications, however, the signal may be contaminated by non-Gaussian noises. Thus, the performance of LMS may degrade seriously [7], [8]. Hence, some robust adaptive filtering algorithms have been proposed to deal with this problem, such as sign algorithm (SA) and maximum correntropy criterion (MCC) algorithm. Both methods perform well under impulsive noise. However, they may not lead to a superior performance with more complex distributions noises, such as multi-peak distribution [9], [10], [11], [12], [13], [14]. Therefore, it is necessary to develop a more robust algorithm.

To address the above problem, minimum error entropy (MEE), an information theoretic criterion that involves higher-order statistics, has attracted much attention [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27]. The MEE has been applied in various learning tasks and performs much better than the conventional criteria, especially under complex noise distributions. Basically, the MEE based adaptive filtering algorithm aims to minimize the error entropy so that the adaptive model becomes as close as to the unknown system. In practice, the most widely used MEE cost is the information potential (IP) [16], which can be estimated from data and computed by a double summation over all samples directly. Although, the form of IP is relatively simple, its computation cost is still very expensive, due to the mathematical form of double summation. This may make it quite unsuitable for practical applications.

To address this problem, Chen et al. proposed a quantized MEE (QMEE) criterion [8], [28]. Its basic idea is to simplify the inner summation by quantifying the error samples, which can significantly reduce the computational complexity. In this paper, we propose a novel adaptive filtering algorithm based on the QMEE, named QMEE algorithm. Here, we maintain the same acronym for the proposed algorithm, although it is slightly different. In order to use the quantization method efficiently in adaptive learning, we propose an online method to quantify the error samples from the sliding window into a codebook. As the window slides, the codebook will adjust dynamically. Moreover, compared with MEE algorithm, the proposed QMEE algorithm involves only one additional free parameter (the quantization threshold). In addition, we derive a dynamic recursion equation to characterize the transient behavior of weight error power. Furthermore, we use the Taylor expansion approach to derive an approximate analytical expression for the steady-state excess mean square error (EMSE) [11]. The performance of QMEE algorithm and the theoretical expression are confirmed by several simulations.

The remainder of this paper is organized as follows: In Section 2, we briefly review the MEE algorithm and the QMEE criterion. In Section 3, we develop the QMEE algorithm and derive an approximate analytical expression. In Section 4, the illustrative simulations and a real world application in denoising of electroencephalogram (EEG) signals are provided. Finally, the conclusion is given in Section 5.

Section snippets

MEE algorithm

Consider a linear adaptive system with the channel weight W*=[w1,w2,, wF]T, where F is the length of the channel memory. The desired signal dn at time n is defined as dn=W*TXn+vn, where vn denotes an interference noise, and Xn=[xn,xn1,,xnF+1]T is the input signal. The instantaneous error can be calculated as en=dnWnTXn, where WnRF is the estimated weight value of the adaptive filtering algorithm [17].

Assume that the error en is a random variable with probability density function (PDF) fe(e

Adaptive filtering algorithm

In this subsection, we propose an adaptive filtering algorithm based on the quantized MEE (QMEE) with an online quantized method, named QMEE algorithm. A key technique is how to design a simple and online quantizer Q[ · ], including how to build and update the codebook as the window slides.

The schematic diagram of the proposed online quantized method is presented in Fig. 1. The blue dots denote the error samples, the yellow dots are code words in the codebook and the rectangular frames are

Verification of theoretical results

In this subsection, we show the theoretical and simulated convergence curves of the QMEE algorithm. In this work, the weight vector of the unknown system is assumed to be W*=[0.1,0.2,0.3,0.4, 0.5, 0.4, 0.3, 0.2, 0.1]T, and the initial weight vector of the adaptive filter is a null vector. The input signal is a zero-mean white Gaussian process with variance 1.0. The window length L is 100, and the filter length F is 9 [37], [38]. We consider a noise model with form vn=(1an)An+anBn, where an is

Conclusion

In this work, we proposed a novel adaptive algorithm based on QMEE criterion, called QMEE algorithm. Moreover, an online quantized method was proposed to reduce the computational complexity. Further, we analyze the transient behavior characteristic and derived an approximate analytical expression for the steady-state excess mean square error of the proposed algorithm based on the Taylor expansion. Finally, several simulation results on synthetic and real world dataset confirmed the excellent

Declaration of Competing Interest

The authors declared that they have no conflicts of interest to this work.

CRediT authorship contribution statement

Zhuang Li: Investigation, Writing - original draft. Lei Xing: Theoretical analysis, Writing - review & editing. Badong Chen: Resources, Writing - review & editing, Supervision.

Acknowledgments

This work was supported by National Natural Science Foundation of China (91648208, 61976175), National Natural Science Foundation-Shenzhen Joint Research Program (U1613219), and The Key Project of Natural Science Basic Research Plan in Shaanxi Province of China (2019JZ-05).

References (42)

  • B. Chen et al.

    Kernel minimum error entropy algorithm

    Neurocomputing

    (2013)
  • S. Han et al.

    A minimum-error entropy criterion with self-adjusting step-size (MEE-SAS)

    Signal Process.

    (2007)
  • S. Haykin, Adaptive filter theory, 5e...
  • Kailath et al.

    Linear estimation

    (2008)
  • B. Widrow

    Adaptive inverse control

    (1996)
  • S.C. Pei et al.

    Least mean p-power error criterion for adaptive FIR filter

    Sel. Areas Commun. IEEE J.

    (1995)
  • E. Walach et al.

    The least mean fourth (LMF) adaptive algorithm and its family

    IEEE Trans. Inf. Theory

    (1984)
  • B. Chen et al.

    Adaptive filtering under minimum information divergence criterion

    Int. J. Control Autom. Syst.

    (2009)
  • B. Chen et al.

    System Parameter Identification: Information Criteria and Algorithms

    (2013)
  • B. Chen et al.

    Quantized kernel least mean square algorithm

    IEEE Trans. Neural Netw. Learn. Syst.

    (2012)
  • L. Shi et al.

    Convex combination of adaptive filters under the maximum correntropy criterion in impulsive interference

    IEEE Signal Process. Lett.

    (2014)
  • V. Mathews et al.

    Improved convergence analysis of stochastic gradient adaptive filters using the sign algorithm

    IEEE Trans. Acoust. Speech Signal Process.

    (1987)
  • B. Chen et al.

    Steady-state mean-square error analysis for adaptive filtering under the maximum correntropy criterion

    IEEE Signal Process. Lett.

    (2014)
  • R. He et al.

    Robust principal component analysis based on maximum correntropy criterion

    IEEE Trans. Image Process.

    (2011)
  • S. Zhao et al.

    Kernel adaptive filtering with maximum correntropy criterion

    Neural Networks (IJCNN), The 2011 International Joint Conference on

    (2011)
  • B. Chen et al.

    Convergence of a fixed-point algorithm under maximum correntropy criterion

    IEEE Signal Process. Lett.

    (2015)
  • D. Erdogmus et al.

    An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems

    IEEE Trans. Signal Process.

    (2002)
  • D. Erdogmus et al.

    Generalized information potential criterion for adaptive system training

    IEEE Trans. Neural Netw.

    (2002)
  • Z. Wu et al.

    Minimum error entropy algorithms with sparsity penalty constraints

    Entropy

    (2015)
  • D. Erdogmus et al.

    Convergence properties and data efficiency of the minimum error entropy criterion in adaline training

    IEEE Trans. Signal Process.

    (2003)
  • L. Guo et al.

    Minimum entropy filtering for multivariate stochastic systems with non-gaussian noises

    IEEE Trans. Autom. Control

    (2006)
  • Cited by (26)

    • Minimum total complex error entropy for adaptive filter

      2024, Expert Systems with Applications
    • Quantized kernel recursive minimum error entropy algorithm

      2023, Engineering Applications of Artificial Intelligence
    • Generalized minimum error entropy for robust learning

      2023, Pattern Recognition
      Citation Excerpt :

      For example, MEE is widely applied in the adaptive filtering (AF) family to improve the robustness against non-Gaussian noise. Typical AF algorithms based on MEE are developed for the recognition of linear [11–13] and non-linear [14,15] systems. In addition, the parameter estimation problem of errors-in-variables (EIV) [16] and constrained systems [17] are processed by the MEE-based AF algorithms.

    View all citing articles on Scopus
    View full text