Adaptive filtering with quantized minimum error entropy criterion
Introduction
Over the past decades, the fundamental of adaptive filtering has been extensively developed [1], [2], [3] and used in wide areas, such as channel equalization and noise cancellation. Many useful adaptive filter algorithms have been proposed. For example, the well-known least mean square (LMS) is based on the popular minimum mean square error (MMSE) criterion [4], [5], [6], which can perform well under Gaussian noise assumption. In many practical applications, however, the signal may be contaminated by non-Gaussian noises. Thus, the performance of LMS may degrade seriously [7], [8]. Hence, some robust adaptive filtering algorithms have been proposed to deal with this problem, such as sign algorithm (SA) and maximum correntropy criterion (MCC) algorithm. Both methods perform well under impulsive noise. However, they may not lead to a superior performance with more complex distributions noises, such as multi-peak distribution [9], [10], [11], [12], [13], [14]. Therefore, it is necessary to develop a more robust algorithm.
To address the above problem, minimum error entropy (MEE), an information theoretic criterion that involves higher-order statistics, has attracted much attention [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27]. The MEE has been applied in various learning tasks and performs much better than the conventional criteria, especially under complex noise distributions. Basically, the MEE based adaptive filtering algorithm aims to minimize the error entropy so that the adaptive model becomes as close as to the unknown system. In practice, the most widely used MEE cost is the information potential (IP) [16], which can be estimated from data and computed by a double summation over all samples directly. Although, the form of IP is relatively simple, its computation cost is still very expensive, due to the mathematical form of double summation. This may make it quite unsuitable for practical applications.
To address this problem, Chen et al. proposed a quantized MEE (QMEE) criterion [8], [28]. Its basic idea is to simplify the inner summation by quantifying the error samples, which can significantly reduce the computational complexity. In this paper, we propose a novel adaptive filtering algorithm based on the QMEE, named QMEE algorithm. Here, we maintain the same acronym for the proposed algorithm, although it is slightly different. In order to use the quantization method efficiently in adaptive learning, we propose an online method to quantify the error samples from the sliding window into a codebook. As the window slides, the codebook will adjust dynamically. Moreover, compared with MEE algorithm, the proposed QMEE algorithm involves only one additional free parameter (the quantization threshold). In addition, we derive a dynamic recursion equation to characterize the transient behavior of weight error power. Furthermore, we use the Taylor expansion approach to derive an approximate analytical expression for the steady-state excess mean square error (EMSE) [11]. The performance of QMEE algorithm and the theoretical expression are confirmed by several simulations.
The remainder of this paper is organized as follows: In Section 2, we briefly review the MEE algorithm and the QMEE criterion. In Section 3, we develop the QMEE algorithm and derive an approximate analytical expression. In Section 4, the illustrative simulations and a real world application in denoising of electroencephalogram (EEG) signals are provided. Finally, the conclusion is given in Section 5.
Section snippets
MEE algorithm
Consider a linear adaptive system with the channel weight wF]T, where F is the length of the channel memory. The desired signal dn at time n is defined as where vn denotes an interference noise, and is the input signal. The instantaneous error can be calculated as where is the estimated weight value of the adaptive filtering algorithm [17].
Assume that the error en is a random variable with probability density function (PDF) fe(e
Adaptive filtering algorithm
In this subsection, we propose an adaptive filtering algorithm based on the quantized MEE (QMEE) with an online quantized method, named QMEE algorithm. A key technique is how to design a simple and online quantizer Q[ · ], including how to build and update the codebook as the window slides.
The schematic diagram of the proposed online quantized method is presented in Fig. 1. The blue dots denote the error samples, the yellow dots are code words in the codebook and the rectangular frames are
Verification of theoretical results
In this subsection, we show the theoretical and simulated convergence curves of the QMEE algorithm. In this work, the weight vector of the unknown system is assumed to be 0.5, 0.4, 0.3, 0.2, 0.1]T, and the initial weight vector of the adaptive filter is a null vector. The input signal is a zero-mean white Gaussian process with variance 1.0. The window length L is 100, and the filter length F is 9 [37], [38]. We consider a noise model with form where an is
Conclusion
In this work, we proposed a novel adaptive algorithm based on QMEE criterion, called QMEE algorithm. Moreover, an online quantized method was proposed to reduce the computational complexity. Further, we analyze the transient behavior characteristic and derived an approximate analytical expression for the steady-state excess mean square error of the proposed algorithm based on the Taylor expansion. Finally, several simulation results on synthetic and real world dataset confirmed the excellent
Declaration of Competing Interest
The authors declared that they have no conflicts of interest to this work.
CRediT authorship contribution statement
Zhuang Li: Investigation, Writing - original draft. Lei Xing: Theoretical analysis, Writing - review & editing. Badong Chen: Resources, Writing - review & editing, Supervision.
Acknowledgments
This work was supported by National Natural Science Foundation of China (91648208, 61976175), National Natural Science Foundation-Shenzhen Joint Research Program (U1613219), and The Key Project of Natural Science Basic Research Plan in Shaanxi Province of China (2019JZ-05).
References (42)
- et al.
Kernel minimum error entropy algorithm
Neurocomputing
(2013) - et al.
A minimum-error entropy criterion with self-adjusting step-size (MEE-SAS)
Signal Process.
(2007) - S. Haykin, Adaptive filter theory, 5e...
- et al.
Linear estimation
(2008) Adaptive inverse control
(1996)- et al.
Least mean p-power error criterion for adaptive FIR filter
Sel. Areas Commun. IEEE J.
(1995) - et al.
The least mean fourth (LMF) adaptive algorithm and its family
IEEE Trans. Inf. Theory
(1984) - et al.
Adaptive filtering under minimum information divergence criterion
Int. J. Control Autom. Syst.
(2009) - et al.
System Parameter Identification: Information Criteria and Algorithms
(2013) - et al.
Quantized kernel least mean square algorithm
IEEE Trans. Neural Netw. Learn. Syst.
(2012)
Convex combination of adaptive filters under the maximum correntropy criterion in impulsive interference
IEEE Signal Process. Lett.
Improved convergence analysis of stochastic gradient adaptive filters using the sign algorithm
IEEE Trans. Acoust. Speech Signal Process.
Steady-state mean-square error analysis for adaptive filtering under the maximum correntropy criterion
IEEE Signal Process. Lett.
Robust principal component analysis based on maximum correntropy criterion
IEEE Trans. Image Process.
Kernel adaptive filtering with maximum correntropy criterion
Neural Networks (IJCNN), The 2011 International Joint Conference on
Convergence of a fixed-point algorithm under maximum correntropy criterion
IEEE Signal Process. Lett.
An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems
IEEE Trans. Signal Process.
Generalized information potential criterion for adaptive system training
IEEE Trans. Neural Netw.
Minimum error entropy algorithms with sparsity penalty constraints
Entropy
Convergence properties and data efficiency of the minimum error entropy criterion in adaline training
IEEE Trans. Signal Process.
Minimum entropy filtering for multivariate stochastic systems with non-gaussian noises
IEEE Trans. Autom. Control
Cited by (26)
Minimum total complex error entropy for adaptive filter
2024, Expert Systems with ApplicationsRobust adaptive filtering based on M-estimation-based minimum error entropy criterion
2024, Information SciencesAdaptive filtering under multi-peak noise
2024, Signal ProcessingQuantized kernel recursive minimum error entropy algorithm
2023, Engineering Applications of Artificial IntelligenceGeneralized minimum error entropy for robust learning
2023, Pattern RecognitionCitation Excerpt :For example, MEE is widely applied in the adaptive filtering (AF) family to improve the robustness against non-Gaussian noise. Typical AF algorithms based on MEE are developed for the recognition of linear [11–13] and non-linear [14,15] systems. In addition, the parameter estimation problem of errors-in-variables (EIV) [16] and constrained systems [17] are processed by the MEE-based AF algorithms.