A global sensitivity analysis approach for identifying critical sources of uncertainty in non-identifiable, spatially distributed environmental models: A holistic analysis applied to SWAT for input datasets and model parameters
Introduction
Deterministic environmental models offer useful methods for exploring problems, providing predictions, and supporting decisions that involve complex environmental phenomena evolving in space and time. It has become increasingly recognized, however, that these models are almost always non-identifiable (Guillaume et al., 2019), in the sense that the quantity and quality of data are insufficient to parameterize the models uniquely, and that the associated modelling must address a wide range of uncertainties, especially those related to predicting the impact of possible management actions. Accordingly, Uusitalo et al. (2015) review various methods that can be applied to evaluate uncertainty of deterministic model outputs. They cover expert assessment, model emulation, sensitivity analysis (SA) and use of multiple models, arguing that the best method for uncertainty evaluation is determined by the definition of a model, and the amount of available information. SA is an established approach to model assessment by quantitatively evaluating the change in model output(s) with respect to changes in model factors (usually parameters, forcing and other input data). A deterministic environmental model can be easily coupled with SA based on Monte Carlo type simulation (Farmer and Vogel, 2016). SA offers a quantitative evaluation for uncertainty in a model, rather than a qualitative evaluation which, for example, expert assessment does (Uusitalo et al., 2015), although qualitative evaluation could provide additional evidence to further support decision making and might be more efficient when a modeler is well-informed. In this paper we employ a global method of SA (Saltelli et al., 2008) as opposed to a local method. A global method allows one to compute the contribution to output sensitivity over a plausible range of factor values.
Global SA is undertaken here as a useful first step in addressing and understanding the criticality of uncertainty sources (see Norton, 2015 for an overview of methods, and Crosetto and Tarantola, 2001; Gan et al., 2014; Tong, 2015) by assessing those factors, and their plausible range of values, that dominate changes in model outputs. Understanding the criticalities can then be valuable for undertaking an uncertainty analysis in the traditional probabilistic sense, for example, where one assumes prior distributions of parameters and finds posterior distributions based on some measure of likelihood.
As indicated in the literature review of Section 2, previous approaches to SA tend to address only a modest set of the sources of uncertainty for spatially distributed environmental models. The aim of this article is to illustrate how a more holistic SA approach to spatially distributed environmental models can be used to identify their critical sources of uncertainty, which would subsequently allow focus on them for more specific uncertainty analysis and even its reduction. The Soil and Water Assessment Tool (SWAT) (Neitsch et al., 2011) is used as an archetypal example as it is the most frequently used water quantity-quality model (Fu et al., 2019; Ray, 2018). It is also a typical example of the type of uncertainties that need to be considered in a spatially distributed environmental model. The global SA approach undertaken is more complete than previous SA studies on spatially distributed environmental models in the sense that: it attempts to address model structure uncertainty in combination with the usual model parameter and data uncertainties; and examines impact on average streamflow as well as water quality outputs. Thus, this article addresses the uncertainty of model structure input parameters related to the submodels of a SWAT application [i.e., watershed delineation and hydrological response unit (HRU) characteristics]. It also explores the measurement uncertainty of the digital elevation model (DEM) (i.e., its vertical accuracy), and the uncertainty of boundaries of classes in land-use-land-cover (LULC) and soil datasets (Crosetto and Tarantola, 2001; Goodchild and Guoqing, 1992) because they have a profound effect on watershed delineation (Oksanen and Sarjakoski, 2005; Wu et al., 2008) and HRU creation, respectively. Another aspect investigated is the impact of measurement errors in meteorological information on model outputs. Finally, the SA is also applied to the model parameters investigated as has been the primary focus in the past (e.g., Setegn et al., 2010; Wu and Liu, 2012; Yang et al., 2018; Zhao et al., 2018), some of which relate to model structure assumptions. Clearly, while it is not possible to investigate all model structure assumptions as these can be innumerable in an environmental modelling exercise, it is acknowledged that expert assessment is a key, complementary and qualitative method that can be used to justify other model structure assumptions in relation to uncertainty (O'Hagan, 2012; Uusitalo et al., 2015). Our SA approach could therefore be followed up with a formal uncertainty analysis to help in setting the range of prior distributions and fixing unimportant factors, which is especially important when sampling needs to be limited because of high computational demands of the environmental model.
The remainder of this article is organized as follows. Section 2 contains a literature review and key qualitative findings of previous SA studies on the SWAT model. Section 3 briefly explains SWAT and the extended Fourier Amplitude Sensitivity Test (FAST) method which is the global SA method undertaken here. Then, in Section 4, the detailed holistic SA process is reported with our SWAT application to the Minjiang River watershed in Sichuan, China. This analysis follows a general process of SA (Cheng et al., 2014; Gan et al., 2014): identifying uncertainty sources associated with submodels in SWAT, and propagating the uncertainty from the identified source. Results of analyzing the uncertainty sources using SA appear in Section 5. Specifically, this article identifies uncertainty sources related to model structure input parameters and datasets, as well as general model parameters. Then, uncertainty propagation methods are utilized with respect to the corresponding uncertainty sources. For spatial input datasets, the measurement uncertainty of a DEM is propagated using a sequential Gaussian simulation to represent spatially autocorrelated uncertainty (Goovaerts, 1997; Pebesma, 2004), and the boundary uncertainties of LULC and soil datasets are simulated by adopting the epsilon band approach (Crosetto and Tarantola, 2001; Shi, 1998). The SA evaluates the relative importance of the uncertainty sources in the average streamflow (FLOW) and loads of nitrate (NO3). This article concludes in Section 6 with a discussion on future work that could profitably be conducted in relation to our analysis.
Section snippets
Previous SA studies on SWAT
There has been too little attention given to sensitivity analysis of spatially distributed models that focuses on a wide range of uncertainties, especially the influence of model structure assumptions. In illustration, this section explores limitations of previous SA studies for the water quantity-quality model known as SWAT, partly because it provides a typical example of the limitations of previous sensitivity studies with respect to a spatially distributed environmental model. SWAT requires
SWAT
SWAT is a watershed model that was developed by the Agricultural Research Service of the U.S. Department of Agriculture (USDA) (Neitsch et al., 2011). SWAT has been extensively applied in various sectors, including water management, hydrology, climate change, land use impact, and pollution, to predict the environmental impacts (e.g., land use and climate change) on water quantity and quality. To implement SA for SWAT applications (Bastin et al., 2013), this paper uses a tool for automated SWAT
Study area and datasets
This study illustrates a more holistic SA approach to spatially distributed environmental models with a SWAT example in a subset of the Minjiang River watershed. The Minjiang River watershed is located in the upper part of the Yangtze River basin, and the subset of the watershed in this analysis covers approximately 12,893 km2 (Fig. 1). The Minjiang River is the largest tributary of the Upper Yangtze River and supplies water to downstream regions for agriculture (e.g., Chengdu and its
Results of analyzing the uncertainty sources through SA
This analysis evaluates the impacts of the fifteen factors on variations in the two SWAT output estimates considered; i.e., FLOW and NO3 at the outlet of the Minjiang River watershed (refer to Fig. 1). Firstly, the impacts of the fifteen factors on FLOW are evaluated in terms of the main and total effect indices using the extended FAST (Fig. 4). As we discussed in Section 4.3, two different ranges of MinStream [i.e., the wide range (W-SA) between 5,000 and 15,000 and the narrow range (N-SA)
Conclusion
A wide range of sources of uncertainty and their relative importance were examined in the context of an integrated, spatially distributed environmental model based on a SWAT application in the Minjiang River watershed in Sichuan, China. This paper identified the strength of uncertainty sources with respect to watershed delineation, HRU creation, and SWAT execution, thereby addressing the uncertainty of model structure through uncertainties of the input parameters and datasets in these
Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
We thank the OpenGMS team from Nanjing Normal University for their assistance with an automated SWAT preparation tool. This research was supported by the NSF for Excellent Young Scholars of China, Grant 41622108, the National Basic Research Program of China, Grant 2015CB954103, and the Priority Academic Program Development of Jiangsu Higher Education Institutions, Grant 164320H116.
References (82)
- et al.
Modelling hydrology and water quality in the pre-alpine/alpine Thur watershed using SWAT
J. Hydrol
(2007) - et al.
Managing uncertainty in integrated environmental modelling: the UncertWeb framework
Environ. Model. Software
(2013) - et al.
An evaluation of the impact of model structure on hydrological modelling uncertainty for streamflow simulation
J. Hydrol
(2004) - et al.
Effect of the accuracy of spatial rainfall information on the modeling of water , sediment , and NO3-N loads at the watershed level
J. Hydrol
(2005) - et al.
Impacts of input parameter spatial aggregation on an agricultural nonpoint source pollution model
J. Hydrol
(2000) - et al.
A review of catchment-scale water quality and erosion models and a synthesis of future prospects
Environ. Model. Software
(2019) - et al.
A comprehensive evaluation of various sensitivity analysis methods: a case study with a hydrological model
Environ. Model. Software
(2014) - et al.
Introductory overview of identifiability analysis: a guide to evaluating whether you have the right type of data for your modeling purpose
Environ. Model. Software
(2019) - et al.
Error in a USGS 30-meter digital elevation model and its impact on terrain modeling
J. Hydrol
(2000) - et al.
Analysis of parameter uncertainty in semi-distributed hydrological models using bootstrap method : a case study of SWAT model applied to Yingluoxia watershed in northwest China
J. Hydrol
(2010)
An introduction to sensitivity assessment of simulation models
Environ. Model. Software
Probabilistic uncertainty specification: overview, elaboration techniques and their application to a mechanistic model of carbon flux
Environ. Model. Software
Multi-method global sensitivity analysis of flood inundation models
Adv. Water Resour.
Multivariable geostatistics in S: the gstat package
Comput. Geosci.
A multi-method Generalized Global Sensitivity Matrix approach to accounting for the dynamical nature of earth and environmental systems models
Environ. Model. Software
Parameter uncertainty analysis of the non-point source pollution in the Daning River watershed of the three Gorges Reservoir region , China
Sci. Total Environ.
A comprehensive study of the effect of GIS data on hydrology and non-point source pollution modeling
Agric. Water Manag.
Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates
Math. Comput. Simulat.
Using precipitation data ensemble for uncertainty analysis in SWAT streamflow simulation
J. Hydrol
A Bayesian total uncertainty analysis framework for assessment of management practices using watershed models
Environ. Model. Software
Geostatistical simulation and error propagation in geomorphometry
Dev. Soil Sci.
Flood inundation modelling: a review of methods, recent advances and uncertainty analysis
Environ. Model. Software
An overview of methods to evaluate uncertainty of deterministic models in decision support
Environ. Model. Software
Global sensitivity analysis for urban water quality modelling: terminology, convergence and comparison of different methods
J. Hydrol
Automating calibration, sensitivity and uncertainty analysis of complex models using the R package Flexible Modeling Environment (FME): SWAT as an example
Environ. Model. Software
Uncertainty analysis of a semi-distributed hydrologic model based on a Gaussian Process emulator
Environ. Model. Software
Hydrological modelling of the Chaohe basin in China: statistical model formulation and Bayesian inference
J. Hydrol
Design and development of a service-oriented wrapper system for sharing and reusing distributed geoanalysis models on the web
Environ. Model. Software
SWAT-CUP: SWAT calibration and uncertainty programs - a user manual. Eawag: Swiss federal institute of aquatic science and technology
Spatial Data Transfer Standard (SDTS) - Part 1. Logical Specifications
Sensitivity analysis of SWAT model to the spatial rainfall distribution and watershed subdivision in streamflow simulations in the Mediterranean context : a case study in the Joumine watershed . Tunisia
Influence of rainfall observation network on model calibration and application
Hydrol. Earth Syst. Sci.
Effect of DEM data resolution on SWAT output
Hydrol. Process.
Handbook of Regional Science
Effect of spatial distribution of rainfall on temporal and spatial uncertainty of SWAT output
Trans. ASABE (Am. Soc. Agric. Biol. Eng.)
The error component in spatial data
Local random errors in tipping-bucket rain gauge measurements
J. Atmos. Ocean. Technol.
Uncertainty and sensitivity analysis: tools for GIS-based model implementation
Int. J. Geogr. Inf. Sci.
Impacts of Forest Changes on Hydrology : a Case Study of Large Watersheds in the Upper Reaches of Minjiang River Watershed in China 4279–4290
Resample or not ?! Effects of resolution of DEMs in watershed modeling
Hydrol. Process.
The digitized soil map of the World and derived soil properties (version 3.5)
FAO L. Water Digit. Media Ser
Cited by (42)
Development of a sensitivity analysis framework for aquatic biogeochemical models using machine learning
2023, Ecological InformaticsAn approach to select optimum inputs for hydrological modeling to improve simulation accuracy in data-scarce regions
2023, Journal of Hydrology: Regional StudiesFarmland nutrient pollution and its evolutionary relationship with plantation economic development in China
2023, Journal of Environmental ManagementReplicating measured site-scale soil organic carbon dynamics in the U.S. Corn Belt using the SWAT-C model
2022, Environmental Modelling and Software