Ensemble-type numerical uncertainty information from single model integrations

doi:10.1016/j.jcp.2015.02.043

Journal of Computational Physics

Volume 292, 1 July 2015, Pages 30-42

https://doi.org/10.1016/j.jcp.2015.02.043 Get rights and content

Abstract

We suggest an algorithm that quantifies the discretization error of time-dependent physical quantities of interest (goals) for numerical models of geophysical fluid dynamics. The goal discretization error is estimated using a sum of weighted local discretization errors. The key feature of our algorithm is that these local discretization errors are interpreted as realizations of a random process. The random process is determined by the model and the flow state. From a class of local error random processes we select a suitable specific random process by integrating the model over a short time interval at different resolutions. The weights of the influences of the local discretization errors on the goal are modeled as goal sensitivities, which are calculated via automatic differentiation. The integration of the weighted realizations of local error random processes yields a posterior ensemble of goal approximations from a single run of the numerical model. From the posterior ensemble we derive the uncertainty information of the goal discretization error. This algorithm bypasses the requirement of detailed knowledge about the models discretization to generate numerical error estimates. The algorithm is evaluated for the spherical shallow-water equations. For two standard test cases we successfully estimate the error of regional potential energy, track its evolution, and compare it to standard ensemble techniques. The posterior ensemble shares linear-error-growth properties with ensembles of multiple model integrations when comparably perturbed. The posterior ensemble numerical error estimates are of comparable size as those of a stochastic physics ensemble.

Introduction

Numerical models of geophysical fluid dynamics (GFD) have inherent and inevitable uncertainties. Numerical uncertainties are usually analyzed during the development phase of the model; the standard approach requires a detailed analysis of the numerical properties of the scheme used. In this paper, we propose an algorithm that allows us to estimate a posteriori the numerical error directly from the model solutions.

The behavior of GFD systems is characterized by physical quantities of interest such as total energy content or transport quantities. These physical quantities can be described as functionals of the model solution and are called “goals”. The algorithm that we propose estimates the error in the calculation of goals due to numerical uncertainties. The starting point of our considerations is the dual-weighted residual method of computational fluid dynamics (e.g., [1], [14], [5], [15], [6], [2]). In the dual-weighted residual method the estimation of the goal error is split into the estimation of local model errors and the sensitivity of the goal to local changes. The sensitivities are calculated as the adjoint solution of a goal-dependent dual problem. In summary, one obtains a (sensitivity-)weighted sum of local errors. In standard forms of the dual-weighted residual methods, the local model errors are estimated using knowledge of the discretization formulation and thus require an in-depth understanding of the underlying discrete model. The latter fact can become prohibitive for complex nonlinear models. So far the quantification of goal approximation errors for GFD applications has been done via deterministic local error estimates [5], [16]. Here the local model errors are treated as a uniquely deterministic consequence of the modeled flow, and the goal error is therefore a deterministic consequence of these local errors.

We deviate from the framework of the dual-weighted residual method by treating the local error as a stochastic process. The key element of our approach is the goal approximation ensemble. The goal approximation ensemble is calculated in two steps. First, a suitable and specific element of a class of stochastic processes such as the Gaussian processes is selected. The selection uses information from short model integrations at a standard resolution and at a higher resolution to determine the parameters of the stochastic process. Second, from realizations of the selected stochastic process and the adjoint solution a goal approximation ensemble is calculated a posteriori. From this goal approximation ensemble we estimate bounds for the goal error. The calculation of the sensitivity information uses Algorithmic Differentiation [9], [13].

The novel aspect of our algorithm is the goal approximation ensemble. This ensemble aims to capture the linear component of information component of a nonlinear model ensemble but generates this information from two short model runs at standard and high resolution, one (forward) integration of the model and one adjoint integration. In terms of computational cost this reduces essentially to a single integration of the model and its adjoint on the lower resolution. We provide numerical evidence that the stochastic characterization of local errors enables us through the goal approximation error to estimate the error in the goal approximation. This work extends our earlier work [16] on deterministic error estimation to a stochastic framework.

This paper is organized as follows: in Section 2 we formulate the general problem: what do we mean when we speak about error bounds for numerical goals? Section 3 presents our proposed algorithm to solve this problem. This section represents the core of our approach and shows where we diverge from classical numerical error estimation techniques. In Section 4 we introduce a model to evaluate our algorithm, and in Section 5 results are shown for this model and two test cases. In Section 6 we discuss the results and especially compare the posterior goal error ensembles with forward ensembles. We conclude in Section 7 with an outlook of potential applications of our algorithm.

Section snippets

General problem statement

We keep the problem statement abstract to permit a general formulation of our error estimation algorithm in Section 3. A more specific formulation will be given in Section 4. We introduce a general, continuous model N that maps a state vector q from its initial value $q_{0}$ to its value $q (t)$ at a later time. The model N might be a linear mapping or a nonlinear one, such as the Shallow-Water Equations in Section 4. The model N acts on the state vector as follows $N (q (x, t)) = 0, q (x, t_{0}) = q^{0} (x), q (x, t) = q_{b} (t$

Local model errors as random process and algorithm proposal

Our algorithm cannot be deduced from a specific numerical discretization technique because we want it to be applicable to any general black-box discretization technique a potential user of a GFD model might encounter. In this section we motivate our algorithm following the extraction of resolved dynamics from a process of higher complexity [8]. This paragraph is not a rigorous mathematical derivation but a motivating argument for why it makes sense to interpret unresolved processes as

The Shallow-Water Equations as a prototype for a nonlinear model

The Shallow-Water Equations (SWE) on a rotating sphere are a specific example of the general model N of Section 2. We write the inviscid SWE on the sphere Ω in vector-invariant form as $\frac{\partial v}{\partial t} = (ξ + f) k \times v - \nabla (g h + \frac{1}{2} {| v |}^{2})$ $\frac{\partial h}{\partial t} + \nabla \cdot (h v) = 0 .$ Here v is the horizontal velocity, ξ the vorticity, f the Coriolis parameter, $g = 9.81 m / s^{2}$ the gravitational acceleration, and h the height of the fluid surface. The initial conditions are $v (t_{0}) = v_{0}$ and $h (t_{0}) = h_{0}$ . We consider (13) on a time interval $T : = [t_{0}, t_{n}]$ . The state vector

Results from the single-run ensemble

We now apply the algorithm of Section 3 to TC1 and TC2, running the model at low resolution and higher resolution to calculate the local error rates of change; see Table 2.

Strengths and weaknesses of posterior ensembles

Our results have shown that the newly introduced posterior ensemble (PE) and stochastic-physics ensembles (SPE) are conceptually and empirically closely connected. PE can be seen as a linearized, simplified version of SPE that can be derived from only one model solution, and that could be used in further studies as a potentially cheaper alternative to SPE. Our local error initialization concept is connected to the concept of optimal perturbations. These perturbations maximize error growth from

What is special about posterior ensembles, and why does it matter?

Our new algorithm is very general, not only with respect to models and discretization techniques but also with respect to the type of problems it could be applied to. The exact specification and execution of the algorithm suggests further research areas: the random processes could and should be (topographically) regional, the adjoints could be simplified, and the local error processes could have memory. It has to be tested how well the method works with more connected state variables. It is

Acknowledgements

We thank the International Max Planck Research School on Earth System Modelling for funding the research that led to this publication. We also thank the Max Planck Society for the Advancement of Sciences for supporting the authors. We thank our colleagues at RWTH Aachen for support with the adjoint compiler, and we especially thank the three reviewers for valuable feedback in shaping this article.

References (20)

J.T. Oden et al.
Estimation of modeling error in computational mechanics
J. Comput. Phys.
(2002)
F. Rauser et al.
Predicting goal error evolution from near-initial-information: a learning algorithm
J. Comput. Phys.
(2011)
D.L. Williamson et al.
A standard test set for numerical approximations to the shallow water equations in spherical geometry
J. Comput. Phys.
(1992)
R. Becker et al.
A feed-back approach to error control in finite element methods: basic analysis and examples
East-West J. Numer. Math.
(1996)
R. Becker et al.
An optimal control approach to a posteriori error estimation in finite element methods
Acta Numer.
(2002)
Alexandre J. Chorin et al.
Stochastic Tools in Mathematics and Science
(2013)
M. Ehrendorfer et al.
Optimal prediction of forecast error covariances through singular vectors
J. Atmos. Sci.
(1997)
M.B. Giles et al.
Progress in adjoint error correction for integral functionals
Comput. Vis. Sci.
(2004)
Michael B. Giles et al.
Adjoint methods for PDEs: a posteriori error analysis and postprocessing by duality
Acta Numer.
(2002)
M. Giorgetta et al.
Conservative space and time regularizations for the ICON model

There are more references available in the full text version of this article.

Cited by (7)

On a posteriori error estimation using distances between numerical solutions and angles between truncation errors
2021, Mathematics and Computers in Simulation
Citation Excerpt :
Thus, the deterministic objects in high dimensions may manifest some features of the probabilistic nature, such as the normal distribution or some properties, which appear with the “probability one”. This may explain results of papers [8,37] regarding the unexpected statistical properties of the truncation and approximation errors. The more information on the relation of the geometry and probability may be found in [13,54].
The geometric properties of the ensemble of numerical solutions obtained by the algorithms of different inner structure are addressed from the prospects for a posteriori error estimation. The numerical results are presented for the two-dimensional inviscid supersonic flows, containing shock waves. The truncation errors are computed using a postprocessor, the approximation errors are calculated by the subtraction of the numerical and the analytic solutions. The angles between the approximation errors are found to be far from zero that enables a posteriori estimation of the error norm. The correlation of the angles between the approximation errors and the corresponding angles between the computable truncation errors is observed in numerical tests and discussed from the viewpoint of the measure concentration effect and the algorithmic randomness. The analysis of the truncation errors’ geometry and the distances between solutions enables the estimation of the approximation error norm on the ensemble of numerical solutions obtained by the independent algorithms.
On uncertainty quantification via the ensemble of independent numerical solutions
2020, Journal of Computational Science
The instance of the epistemic uncertainty quantification concerning the estimation of the approximation error norm is analyzed using the ensemble of numerical solutions obtained via independent numerical algorithms. The analysis is based on the geometry considerations: the triangle inequality and the diameter of the ensemble related with the measure concentration phenomenon in spaces of great dimension. In result, nonintrusive postprocessing may be performed that provides the approximation error norm estimation on the ensemble of the solutions. The ensemble of numerical results obtained by five OpenFOAM solvers (based on independent algorithms) is analyzed from this viewpoint. The numerical tests are made for the inviscid compressible flow around a cone at zero angle of attack. The norm of the approximation error and the error of the valuable functional (drag coefficient) are successfully estimated via ensemble based approach that is confirmed by the comparison with the etalon precise solution. The considered approach provides the error estimation with the acceptable value of the efficiency index.
Stochastic goal-oriented error estimation with memory
2017, Journal of Computational Physics
Citation Excerpt :
First, we extend the algorithm to stochastic processes with memory, a property deemed to be highly important in modeling the effect of the unresolved scales (see e.g. [10–14]). The white-noise process used in [8] does not provide such a memory effect as future states are independent of previous states. Our algorithm now models the local truncation error by considering temporal fluctuations in local truncation errors at all previous timesteps, and thus naturally inherits a memory effect.
We propose a stochastic dual-weighted error estimator for the viscous shallow-water equation with boundaries. For this purpose, previous work on memory-less stochastic dual-weighted error estimation is extended by incorporating memory effects. The memory is introduced by describing the local truncation error as a sum of time-correlated random variables. The random variables itself represent the temporal fluctuations in local truncation errors and are estimated from high-resolution information at near-initial times.
The resulting error estimator is evaluated experimentally in two classical ocean-type experiments, the Munk gyre and the flow around an island. In these experiments, the stochastic process is adapted locally to the respective dynamical flow regime. Our stochastic dual-weighted error estimator is shown to provide meaningful error bounds for a range of physically relevant goals. We prove, as well as show numerically, that our approach can be interpreted as a linearized stochastic-physics ensemble.
Generalized Computational Experiment and Verification Problems
2021, Programming and Computer Software
On the relation of truncation and approximation errors for the set of solutions obtained by different numerical methods
2020, arXiv
On applying of generalized computational experiment to numerical methods verification
2020, CEUR Workshop Proceedings

View all citing articles on Scopus

View full text

Ensemble-type numerical uncertainty information from single model integrations

Abstract

Introduction

Section snippets

General problem statement

Local model errors as random process and algorithm proposal

The Shallow-Water Equations as a prototype for a nonlinear model

Results from the single-run ensemble

Strengths and weaknesses of posterior ensembles

What is special about posterior ensembles, and why does it matter?

Acknowledgements

J. Comput. Phys.

J. Comput. Phys.

J. Comput. Phys.

A feed-back approach to error control in finite element methods: basic analysis and examples

East-West J. Numer. Math.

An optimal control approach to a posteriori error estimation in finite element methods

Acta Numer.

Stochastic Tools in Mathematics and Science

Optimal prediction of forecast error covariances through singular vectors

J. Atmos. Sci.

Progress in adjoint error correction for integral functionals

Comput. Vis. Sci.

Adjoint methods for PDEs: a posteriori error analysis and postprocessing by duality

Acta Numer.

Conservative space and time regularizations for the ICON model