Elsevier

Signal Processing

Volume 201, December 2022, 108677
Signal Processing

A Gaussian mixture filter with adaptive refinement for nonlinear state estimation

https://doi.org/10.1016/j.sigpro.2022.108677Get rights and content

Highlights

Abstract

The state estimation of highly nonlinear dynamic systems is difficult because the probability distribution of their states can be highly non-Gaussian. An adaptive Gaussian mixture filter is developed in this work to address this challenge, in which the Gaussian mixture models are refined based on the system's local severity of nonlinearity to attain a high-fidelity estimation of the state distribution. A set of nonlinearity assessment criteria are designed to trigger the splitting of Gaussian components at both the prediction and update stages of Bayesian filtering and the error bound of estimated distribution is established. The new filter has been benchmarked against the existing methods on two challenging problems and it consistently provides among-the-best accuracy with a reasonable computational cost, which proves that it can be used as a reliable state estimator for engineering systems with highly nonlinear dynamics and subject to high magnitudes of uncertainties.

Introduction

The estimation of unmeasurable states in a dynamic system subject to stochastic uncertainty is of great importance in a broad range of engineering areas, such as signal processing, positioning and navigation, and control system design [1]. Given the model of a system, the recursive inference of its state information over time from noisy measurements based on Bayesian statistics is referred to as Bayesian filtering [2]. For linear systems with Gaussian white noise, the Kalman filter is proved to be the optimal estimator [3]. For nonlinear systems, the extended Kalman filter (EKF) that applies the Kalman filter to the linearization of system model is a widely used solution [4]. However, the EKF may yield inaccurate or even divergent results for highly nonlinear systems as it only retains the first-order Taylor's expansion [5].

Therefore, a variety of nonlinear Kalman filters have been developed over the past few decades. The unscented Kalman filter (UKF) uses the unscented transform (UT) to estimate the moments of nonlinear state variables at second-order accuracy [5]. Similarly, the quadrature Kalman filter (QKF) employs the Gauss–Hermite quadrature [6], and the cubature Kalman filter (CKF) uses cubature rules [7] to evaluate the moment integrals at higher-order accuracy. The ensemble Kalman filter (EnKF) draws random samples from a Gaussian state distribution and applies a Kalman filter to the ensemble of samples [8]. What these filters have in common is that the distribution of state variables is assumed to be always Gaussian, while they utilize different integral evaluation techniques to estimate the mean and covariance. However, due to the nonlinearity in the system, the probability density function (PDF) of state variables may exhibit non-Gaussianity like skewness and multimodality, even if the initial state uncertainty is Gaussian. Hence, only assessing the first two moments can not fully characterize the state PDF and the state estimation result may be suboptimal [9,10].

For nonlinear and non-Gaussian state estimation problems, the particle filter (PF) is one of the most popular methods [11]. Through sequential importance sampling, the PF can draw random particles from non-Gaussian distributions to represent the state PDF [9], and then the particles are propagated through the system dynamics and reweighted according to measurement likelihood to track the PDF's evolution over time. However, though PF is applicable to almost any system and PDF [11], the number of particles needed to achieve a satisfactory result for a high-dimensional problem can be overwhelming and the computational cost might be unaffordable [12,13].

Another non-Gaussian estimation paradigm alternative to PF is the Gaussian mixture filter (GMF). Compared with a particle in the PF that acts like a Dirac delta function with infinitesimal width, a Gaussian component with non-zero covariance carries probability information in a nonnegligible sub-domain of the state space. To estimate a PDF to the same level of accuracy, a Gaussian mixture should need many fewer components than the particles needed by a PF, and thus may resolve the curse of dimensionality [12]. Then for GMF, it is of central importance to determine the Gaussian mixture parameters to ensure PDF tracking accuracy [14]. Although it can be proved that the GMF approaches the optimal estimation if the covariance of each component is sufficiently small [15], using too many narrow components will dramatically increase the computational cost and a compromise must be made. By reviewing the literature, the existing GMFs can be roughly classified into three groups:

1) A fixed number of parallel Gaussian filters. In classical GMFs, with the state PDF represented as a Gaussian mixture, a bank of Gaussian filters (e.g., EKF [16] and QKF [17]) can be applied parallelly to each component without altering mixture size. In [16], Sorenson et al. used an Lk norm objective function to optimize the Gaussian mixture approximation. However, even if optimized to be amply fine for the initial uncertainty, the components are not guaranteed to remain narrow as they evolve over time, while numerically re-optimizing the entire Gaussian mixture in each time step can be too costly. Hence, Tam et al. proposed a method to analytically update the Gaussian mixture likelihood function based on in-process measurement [18], and Terejanu et al. formulated a quadratic programming scheme to only reoptimize the component weights [19].

2) Re-approximation via resampling. One way to regulate the widths of covariance in a GMF is to resample a set of narrow components to reapproximate the calculated Gaussian mixture. Kotecha et al. built a series of particle GMFs and a resampling step was used to prevent the number of PFs from increasing exponentially when combined with Gaussian sum noises [20]. Raihan et al. improved the particle GMF by combining all the particles from different PFs to refit a new Gaussian mixture from the entire particle pool, instead of running PFs in parallel [21]. Stordal et al. derived a GMF from EnKF, in which the ensemble covariance is scaled down by a bandwidth parameter before being assigned to samples so that the attained Gaussian mixture would be fine enough to approximate non-Gaussian PDFs [22]. Psiaki proposed a resampling scheme to generate Gaussian components with bounded covariance [12], based on which a GMF called blob filter was developed [23].

3) Adaptive splitting of components. The resampling will put a universal bound on the covariance so that all the components are uniformly fine. However, since the complexity of state PDF changes over time and varies across the state space, it should be more efficient to refine the Gaussian mixture adaptively, i.e., more and finer components are used when the PDF is complex, while fewer and coarser components can be used when the PDF is simple. The adaptive splitting method is designed to actively assess the impact of nonlinearity such that any component that is distorted to exhibit nonnegligible non-Gaussianity could be detected and split into narrower sub-components. Faubel et al. designed an unscented GMF that splits UKF components until the sigma points are linearly spaced after the measurement function [24]. Horwood et al. compared the Gaussian moments from the current QKF and a finer one after splitting to decide whether the splitting is needed [25]. Leong et al. presented a cubature GMF that assesses the measurement nonlinearity by the deviation of cubature points from a linear fitting [26]. DeMars et al. designed an uncertainty propagation scheme that assesses nonlinearity by the difference between Gaussian entropies estimated using a linear and a nonlinear method [27]. Tuggle et al. designed a measurement update scheme to split components based on the Kullback–Leibler divergence between Gaussian PDFs from an EKF and a second-order EKF [28]. More existing measures of nonlinearity and non-Gaussianity in state estimation can be found in the survey in [29].

It can be seen from the above review that a Gaussian mixture refinement scheme is essential to sustain the fidelity of a GMF, whereas extra computational resources will then be required. Also, the refinement involves additional hyperparameters, such as the number of components and upper bound of covariance, whose optimal values are problem-dependent and have to be determined via trial-and-error. For GMFs, how well they can adapt to the varying complexity of state PDF with a practical computational cost will be the key metric of their performance. In light of this, the adaptive splitting method should be the most promising solution as it can adapt to the impact of nonlinearity and refine Gaussian mixtures only when necessary. However, though state estimation is a two-step procedure in nature, most of the existing splitting schemes are solely designed for the measurement update [24,26,28] with only a few for the state prediction [27]. Also, many of the quantities used to measure nonlinearity, such as the deviation of sigma points [24] and the discrepancies between two Gaussian PDFs [25,28], are not explicitly related to the non-Gaussianity of state PDFs. Though the non-Gaussianity and its induced state estimation error will eventually vanish as these quantities decay to zero, their rates of convergence and the error bounds of state PDF estimations still lack quantitative analysis in most studies.

This paper aims to develop an adaptive Gaussian mixture filter (AGMF) that can resolve the above issues with a novel nonlinearity assessment and Gaussian mixture refinement scheme. For state prediction, a feedforward neural network is used to approximate the state equation and thus the complex system dynamics are converted to tractable sigmoid functions. Then the nonlinearity in state transition is assessed at the neural network's hidden layer using a Kullback–Leibler criterion from [30] to refine the Gaussian mixture before predicting the prior state PDF. For Bayesian update, the measurement nonlinearity is assessed by the divergence between true and approximated likelihoods such that the prior Gaussian mixture could also be refined before estimating the posterior PDF. The innovations of this study include: (1) being inspired by the adaptive Gaussian mixture method in [30], this study extends the adaptive splitting scheme to Bayesian update and thereby creates a new filter in which the refinement is performed for both prior and posterior PDFs; (2) novel nonlinearity criteria are designed in this study to explicitly measure the nonlinearity-induced non-Gaussianity in state PDFs, based on which the Gaussian component splitting can be triggered more efficiently and thus the same level of (or even better) accuracy can be achieved with fewer components than the existing GMFs; (3) the convergence rates of designed nonlinearity measures and error bound of state PDF estimations can be quantified, which provides guidelines in implementing the filter for real applications. The AGMF is compared with the widely used nonlinear Kalman filters, particle filters, and other latest GMFs in the literature on two numerical examples, which proved that the AGMF could achieve state-of-the-art accuracy with a reasonable computational cost on highly nonlinear state estimation problems subject to high magnitudes of uncertainties.

Section snippets

Recursive Bayesian state estimation

This paper considers the state estimation of the following nonlinear dynamic systems in the discrete-time domain:xk=f(xk1,uk,wk)yk=g(xk)+vkwhere k = 1, 2, … is the discrete-time index, xk is the n-dimensional state vector at time k, uk is the nu-dimensional vector of control input, yk is the ny-dimensional vector of measurable output, wk is the nw-dimensional process noise and vk is the nv-dimensional measurement noise. For the simplicity of derivation, both wk and vk are assumed to be

The adaptive gaussian mixture filter

Given that the nonlinear distortion that impacts filter fidelity could occur in both stages of filtering, the nonlinearity assessment and adaptive splitting scheme for state prediction is first presented in Section 3.1, and then this scheme is extended to Bayesian update in Section 3.2. A quantitative analysis of the filter performance is provided in Section 3.3.

Application examples

In this section, the AGMF is compared with the widely used nonlinear Kalman filters, particle filters and latest GMFs on two numerical examples to evaluate the filter's performance.

Conclusion

This paper studies probabilistic state estimation in nonlinear dynamic systems. A novel adaptive Gaussian mixture filter is developed by using Gaussian mixture models to estimate the non-Gaussian distribution of state variables. To attain a better approximation of state PDFs, the Gaussian mixtures are refined adaptively by splitting components based on the local severity of nonlinearity. In the state prediction stage, it is proposed to approximate the state equation with a neural network so

CRediT authorship contribution statement

Bin Zhang: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft. Yung C. Shin: Conceptualization, Methodology, Project administration, Writing – review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (39)

  • K. Ito et al.

    Gaussian filters for nonlinear filtering problems

    IEEE Trans. Automat. Contr.

    (2000)
  • I. Arasaratnam et al.

    Cubature kalman filters

    IEEE Trans. Automat. Contr.

    (2009)
  • G. Evensen

    The Ensemble Kalman Filter: theoretical formulation and practical implementation

    Ocean Dyn.

    (2003)
  • N.J. Gordon et al.

    Novel approach to nonlinear/non-Gaussian Bayesian state estimation

    IEE Proc. F Radar Signal Process.

    (1993)
  • H. Song et al.

    Distributed maximum correntropy filtering for stochastic nonlinear systems under deception attacks

    IEEE Trans. Cybern.

    (2020)
  • M.S. Arulampalam et al.

    A tutorial on particle filters for online nonlinear/non-Gaussian bayesian tracking

    IEEE Trans. Signal Process.

    (2002)
  • M.L. Psiaki et al.

    Gaussian sum reapproximation for use in a nonlinear filter

    J. Guid. Control Dyn.

    (2015)
  • A. Beskos et al.

    A stable particle filter for a class of high-dimensional state-space models

    Adv. Appl. Probab.

    (2017)
  • W. Shao et al.

    Semisupervised Bayesian Gaussian mixture models for non-Gaussian soft sensor

    IEEE Trans. Cybern.

    (2021)
  • Cited by (0)

    View full text