Can model structure families be inferred from model output?

https://doi.org/10.1016/j.envsoft.2020.104817Get rights and content
Under a Creative Commons license
open access

Highlights

  • We test whether model structure families can be inferred from model output using a novel combination of approaches based on family trees (dendrograms).

  • Results show that model family trees based on model structure are generally different from model family trees based on signatures of model output.

  • The difference between both family structures depends on the signature evaluated and the climate where the models are applied.

  • The results of this study imply that we need to look for other ways to sample model space.

Abstract

In model studies, a careful consideration of uncertainty is needed. Parameter and data uncertainty can be explored by sampling, but model structure uncertainty is more challenging to capture since the underlying hypotheses of many models are not directly clear. This study explores whether model structure can be inferred from model output. We created a dendrogram (family tree) based on model structure using a modular modelling framework. Subsequently, we created dendrograms based on model output. We determined the correlation between both dendrograms to analysed if model structure families could be inferred from model output. Results from this experiment over 671 climate instances showed that the performance of the inference depends on the type of output evaluated, and the climate. However, the performance of the inference is overall low, implying that model structure cannot be inferred from model output. These results demonstrate the need to further investigate opportunities to sample model space.

Keywords

Model structure uncertainty
Modular modelling framework
Model genealogy
Signatures
Dendrogram
Sampling model space

Data availability

The CAMELS data set is publicly available. It is provided by NCAR (doi: 10.5065/D6MW2F4D). The R-package for FUSE is also publicly available (doi: 10.5281/zenodo.14005, Vitolo et al. (2015)). The R-scripts for our results and analysis are available on request.

Cited by (0)