Supermodeling, a convergent data assimilation meta-procedure used in simulation of tumor progression

Abstract

Supermodeling is a modern, model-ensembling paradigm that integrates several self-synchronized imperfect sub-models by controlling a few meta-parameters to generate more accurate predictions of complex systems' dynamics. Continual synchronization between sub-models is an attractive alternative allowing for accurate trajectory predictions compared to a single model or a classical ensemble of independent models whose decision fusion is based on the majority voting or averaging the outcomes. However, it comes out from numerous observations that the supermodeling procedure's convergence depends on a few principal factors such as (1) the number of sub-models, (2) their proper selection, and (3) the choice of the convergent optimization procedure, which assimilates the supermodel meta-parameters to data. Herein, we focus on modeling the evolution of the system described by a set of PDEs. We prove that supermodeling is conditionally convergent to a fixed-point attractor regarding only the supermodel meta-parameters. In our proof, we assume constant parametrization of the sub-models. We investigate the formal conditions of the convergence of the supermodeling scheme theoretically. We employ the Banach fixed point theorem for the supermodeling correction operator, updating the synchronization constants' values iteratively. From the theoretical estimate, we make the following conclusions. The nudging of the supermodel to the ground truth (real data assimilated) should be well balanced because both too small and too large attraction to data cause the supermodel desynchronization. The time-step size can control the convergence of the training procedure, by balancing the Lipshitz continuity constant of the PDE operator. All the sub-models have to be close to the ground-truth along the training trajectory but still sufficiently diverse to explore the phase space better. As an example, we discuss the three-dimensional supermodel of tumor evolution to demonstrate the supermodel's perfect fit to artificial data generated based on real medical images.

Introduction

It is well known that the multi-model ensembling approach can be a closer metaphor of an observed phenomenon than a single-model forecast [23]. In the multi-model ensemble (MME) approach where the models are not synchronized the MME mean prediction is often more skillful as model errors tend to average out [23], whereas the spread between the model predictions is naturally interpreted as a measure of the uncertainty about the mean [22]. Although MME tends to improve predictions in terms of statistics (i.e., mean and variance), a major drawback is that averaging uncorrelated trajectories from different models leads to variance reduction and smoothing. It is not intended and is not helpful, thus MME is not designed to produce an improved trajectory that can be seen as a specific forecast [16]. The alternative approach for taking advantage of many trajectories followed by distinctive models and discovering many “basin of attractions” without (premature) loss of the trajectories diversity is combining models dynamically. One of the first naive approaches of this kind has been proposed in [11], while more mature ones can be followed in [25], [27]. They introduce connection terms into the model equations that $nudge$ the state of one model to each other's states in the ensemble.

The computer model's assimilation to a real phenomenon through a set of observations is a complex inverse problem; thus, its time complexity increases exponentially with the number of parameters. This makes data assimilation procedures useless when applied for multiscale models such as weather/climate forecast or complex biological processes like tumor evolution [25], [26], [13], [14]. Supermodel, the ensemble of dynamically synchronized sub-models, was applied in weather/climate forecast [17], [19], [20], [6], [7], for simulations of geological processes [3], atmospheric phenomena [4], and for the tumor evolution simulations [9], [8].

Supermodel consisting of the sub-models represented by the same baseline model parametrized with different parameter sets has been analyzed in [18], [12]. The authors show that coupling between them can be defined by a radically smaller number of meta-parameters than the number of parameters used in the baseline model. So, instead of matching tens parameters of a single model to observed data, just a few meta-parameters of the supermodel can be sufficient, while the original ones in the sub-models remain constant. These constant parameters can be initialized by an expert or can be generated by diverse solutions of fast pre-training.

Continual synchronization between sub-models is an attractive alternative allowing for accurate trajectory predictions compared to a single model or a classical ensemble of independent models whose decision fusion is based on the majority voting or averaging the outcomes. The results showing the advantage of the supermodels in climate/weather forecast are presented, e.g., in [19], [17], [5], [25], [16], [10], [20].

In the supermodel, for appropriate connections, the sub-models fall into a synchronized motion and can be treated as a single model. Because in general the synchronization will not be perfect due to the different parameter values, the supermodel solution is defined as the average of the different model states. Note that the states will be close for strong connections so that smoothing and loss of variance due to the averaging will be limited. As shown in [26], a connected supermodel allows for more flexibility if the ensemble is not perfectly synchronized.

In this paper, we investigate the conditions of the supermodeling scheme's convergence theoretically and approve them experimentally employing the supermodel of 3D tumor dynamics. Our principal contributions are: (1) providing mathematical proof of conditional convergence of the supermodel to an approximate trajectory assimilated to data and (2) developing a complex supermodel of tumor growth in 3D with perfect data assimilation ability.

The structure of the paper is the following. Section 2 introduces the supermodeling procedure. Section 3 uses the Banach fixed-point theorem for analysis of the supermodeling procedure, and draws some practical conclusions on the construction of the supermodel. Next, Section 4 presents the example of the supermodel construction concerning the tumor progression simulations with isogeometric analysis solvers. Finally, Section 5 discusses the numerical results on the tumor supermodel and their relation to the Banach fixed point theorem. We conclude the paper in Section 6.