Effectiveness of Bayesian filters: An information fusion perspective

Highlights

• A fundamental issue concerned the effectiveness of the Bayesian filter is raised.

• The observation-only (O2) inference is presented for dynamic state estimation.

• The “probability of filter benefit” is defined and quantitatively analyzed.

• Convincing simulations demonstrate that many filters can be easily ineffective.

Abstract

The general solution for dynamic state estimation is to model the system as a hidden Markov process and then employ a recursive estimator of the prediction–correction format (of which the best known is the Bayesian filter) to statistically fuse the time-series observations via models. The performance of the estimator greatly depends on the quality of the statistical mode assumed. In contrast, this paper presents a modeling-free solution, referred to as the observation-only (O₂) inference, which infers the state directly from the observations. A Monte Carlo sampling approach is correspondingly proposed for unbiased nonlinear O₂ inference. With faster computational speed, the performance of the O₂ inference has identified a benchmark to assess the effectiveness of conventional recursive estimators where an estimator is defined as effective only when it outperforms on average the O₂ inference (if applicable). It has been quantitatively demonstrated, from the perspective of information fusion, that a prior “biased” information (which inevitably accompanies inaccurate modelling) can be counterproductive for a filter, resulting in an ineffective estimator. Classic state space models have shown that a variety of Kalman filters and particle filters can easily be ineffective (inferior to the O₂ inference) in certain situations, although this has been omitted somewhat in the literature.

Introduction

Dynamic state estimation has been a long-standing and vibrant area of research concerned with the sequential process of estimating a/multiple state(s) evolving over time based on noisy observations. It is the core of many fundamental problems including positioning, tracking, econometric forecasting, adaptive control, etc.

A “naïve” estimation solution is to infer the state directly from the noisy observations received in discrete time instants, hereafter referred to as the observation-only (O₂) inference, which will be addressed in this paper. This is a computationally fast estimation method, providing accuracy that is completely dependent on the observation noise regardless of the state process (for which there is, therefore, no need to model it).

In contrast to the straightforward O₂ inference, which provides only the point state-estimate, the prevailing solution that has been most investigated is to model the system as a hidden Markov process and employ a recursive estimator to statistically fuse the observations with models in real time. In this case, a two-step estimation paradigm must be adopted, including model identification based on data and filter design based on the identified model [41]. The optimal recursive state estimator in the Bayesian sense requires the complete posterior density of the state to be determined as a function of time. The posterior probability density function (PDF) can be analytically computed only for linear systems with additive Gaussian noises for which the known Kalman filter [24], [25] gives the optimal estimate (and some other special cases [9]). In the general case of nonlinear system or/and non-Gaussian noises, it is impossible to compute the exact form of the posterior PDF; instead, one has to resort to some form of approximation which can be parametric (e.g. Gaussian filters or Gaussian sum filters), non-parametric (e.g. Monte Carlo methods) or a mixture of both. An astonishing surge of various recursive filters/smoothers has been witnessed since [24], [25].

These recursive estimators, which have the Bayesian paradigm as the theoretically most elaborated base [26], perform well as long as the models used are accurate, having few disturbances, and that the approximation (required in nonlinear systems) is insignificant. Ideally, an optimality (e.g. Cramér–Rao lower bounds, CRLB [14], [51], [57]) can be reached if the physical world and the assumed model coincide perfectly. However, in most practical problems, accurate knowledge of the state process model (and noises) is often missing. The model of a real process may differ from the assumed model or the best available model for that process, leaving a difference we refer to as modeling error.

It has been well acknowledged in literature since at least [13], [21], [22] that modeling errors (and significant disturbances) can easily cause significant performance deteriorations or even failures of filters. Therefore, dealing with modeling errors has been a fundamental problem. This, however, is not a problem for the O₂ inference as it is free of state process modeling. For recursive estimation, a large variety of strategies have been proposed to enhance the filtering performance including model assessment [11], adaptive filtering (e.g. [19], [34]), robust filtering (e.g. effective characteristics [7], particularly including detection and treatment of uncertain noise [45], outlier [38], abrupt motion [36], asynchronous observations [47], [40] and colored noises [53]), “direct” filtering [41], variable rate filtering [15] and finite impulse response filtering [29], just to name a few. Similar issues occur in Bayesian smoothers and predictors [1], [6], [18], [48] as well as other recursive estimators e.g. optimization-based estimator [27], [42], [46]. The situation will be much more complicated in the multi-target case of cluttered environments, see e.g. [3], [30], [31], [55]. We do not intend to detail these in this paper. However, we would like to point out that:

• While considerable efforts have been devoted to developing sophisticated recursive filters, the general effectiveness of these filters has remained elusive. Simply stated, it is rare to be asked whether the use of a filter will pay off when modeling errors (including outlier noise) occur or when too much approximation is triggered. This is primarily because a clear definition of the effectiveness for general filters is still missing. Such a definition would require a clear, efficient and engineer-friendly benchmark that is qualified to assess all filters in a consistent manner. The same holds for the work on smoothers and other recursive estimators.

• It has been demonstrated that the Bayesian inference can behave very badly if the model under consideration is erroneous e.g. [18]. More specifically simple deterministic methods outperform the Bayesian filter in a type of finite-state estimation [44] even when the model is properly set up. In any case, the quantitative analysis of the failure of filters is missing. This paper will thoroughly demonstrate that the O₂ inference can outperform recursive filters in certain situations, thus indicating that filters do not always pay off.

In this paper, two primary contributions have been made with regard to these fundamental issues.

• The O₂ inference is established as a benchmark, a bottom line, to assess the effectiveness of recursive estimators, including the Bayesian filter, where an estimator is defined as effective only when it can at minimum outperform the O₂ inference on average in accuracy. For a nonlinear observation function, a bias is noticed in the O₂ inference and, consequently, a Monte Carlo sampling-based debiasing approach is proposed.

• The effectiveness of the Bayesian filter of the prediction–correction format is quantitatively investigated from the information fusion perspective, and examples are evaluated on classic filtering models via simulation. Both theoretical studies and simulation results show that the O₂ inference can easily outperform the filters in certain situations, more so than expected. This deserves particular attention for the application of any filter.

The remainder of the paper is organized as follows. The basic idea of the Bayesian filter and the O₂ inference is given in Section 2. Section 3 investigates the effectiveness of the recursive filter from the general perspective of information fusion while Section 4 presents simulation results based on three representative problem models to demonstrate the theoretical findings. We conclude in Section 5.