Identifying parametric controls and dependencies in integrated assessment models using global sensitivity analysis

Highlights

• We present Sobol' sensitivity analyses of three climate policy scenarios.

• Parameter interactions are significant, evolve over time, and vary by policy.

• Key interactions and sensitivities misclassified in expert elicited assessments.

• Climate damages not sufficiently sensitive to climate/carbon cycle parameters.

• Costs of aggressive mitigation controlled by participation and renewables.

Abstract

Integrated assessment models for climate change (IAMs) couple representations of economic and natural systems to identify and evaluate strategies for managing the effects of global climate change. In this study we subject three policy scenarios from the globally-aggregated Dynamic Integrated model of Climate and the Economy IAM to a comprehensive global sensitivity analysis using Sobol' variance decomposition. We focus on cost metrics representing diversions of economic resources from global world production. Our study illustrates how the sensitivity ranking of model parameters differs for alternative cost metrics, over time, and for different emission control strategies. This study contributes a comprehensive illustration of the negative consequences associated with using a priori expert elicitations to reduce the set of parameters analyzed in IAM uncertainty analysis. The results also provide a strong argument for conducting comprehensive model diagnostics for IAMs that explicitly account for the parameter interactions between the coupled natural and economic system components.

Introduction

Climate change is one of the most challenging issues confronting the scientific and policy communities. The National Research Council (NRC, 2009) has called for advances in climate change decision support that facilitate a “deliberation with analysis” approach to the problem. A key aspect of “deliberation with analysis” is the need for frameworks that aid in identifying the key uncertainties influencing the trade-off between near-term carbon dioxide (CO₂) mitigation costs and long-term risks posed by climate change. A large body of literature has emerged seeking to better characterize this trade-off using integrated assessment models (IAMs) (Parson and Fisher-Vanden, 1997, Kelly and Kolstad, 1999). IAMs seek to inform our understanding of the coupled natural and economic systems that shape mitigation and adaptation decisions. More formally, Kelly and Kolstad (1999) define an IAM as “… any model which combines scientific and socio-economic aspects of climate change primarily for the purpose of assessing policy options for climate change control”. For evaluating climate mitigation strategies, IAMs must incorporate important aspects of the climate system and the global economy, and yet be sufficiently transparent to be useful for decision support (Kelly and Kolstad, 1999, Stanton et al., 2009). For IAMs to be useful they need to advance our understanding of the linkages between economic activities, greenhouse gas emissions, the carbon cycle, climate and damages (Parson and Fisher-Vanden, 1997, Courtois, 2004, Stanton et al., 2009, Weyant, 2009). Broadly there are two classes of IAMs (Stanton et al., 2009): (1) inter-temporal optimization models, and (2) simulation models. Inter-temporal optimization models seek to identify a best future course based on global/regional welfare or cost optimization. Optimality is typically defined in this class of IAMs subject to an assumption of perfect foresight and the IAM modeler's expected state-of-the-world (SOW). Simulation (or evaluation) models, instead, play out specific policy scenarios over time without explicitly defining or seeking optimality. Both of these classes of IAMs are nonlinear and require large numbers of externally-specified (exogenous) parameters to abstract the economic and natural systems being modeled.

IAMs are now garnering significant roles in shaping climate change impact projections and in the formulation of alternative mitigation policies (IPCC, 1996, Stern, 2007, EPA, 2010, EPA, 2013, UNEP, 2010, UNEP, 2011, NRC, 2011, Rogelj et al., 2011, Rogelj et al., 2013a, Rogelj et al., 2013b). Many agencies (EPA, 2009, EU, 2009) recommend that all models used for policy development and analysis, including IAMs, be rigorously evaluated. The challenges of evaluating IAMs, as has been reviewed over two decades (Risbey et al., 1996, Stanton et al., 2009, Schwanitz, 2013), include the potentially high degrees of model complexity, the degree of integration and resolution of model components, and incomplete knowledge of underlying processes and data. Efforts to model the inherently unknown future behavior of complex, inter-related systems have led to a focus on the uncertainties associated with framing possible futures. This is often done in the context of community model inter-comparison exercises (e.g., Clarke et al., 2009). Our study builds on additional guidance from broader environmental modeling communities for improving diagnostic assessments of complex environmental modeling systems (e.g., Jakeman et al., 2006, Gupta et al., 2008, Gudmundsson et al., 2012; Kelly (Letcher) et al., Kelly et al., 2013, Baroni and Tarantola, 2014).

Recently, Schwanitz (2013) outlines an evaluation framework specifically for the IAM community. Included as one of the tools in this evaluation framework, global sensitivity analysis has the potential to attribute the uncertainty in an IAM's projections to its parameters, both individually and collectively (Saltelli et al., 2008). To date, sensitivity analyses of IAMs focused on specific functions or modules within a given model (Keller et al., 2004, Gillingham et al., 2008, Ackerman et al., 2010) or on exploiting expert elicitations to reduce the set of parameters to be analyzed with a local sensitivity analysis (Peck and Teisberg, 1993, Prinn et al., 1999, Toth et al., 2003). Recent studies that have applied global statistical sampling to IAMs still confine sensitivity testing to a small subset of parameters within a limited Monte Carlo sampling (Pizer, 1999, Scott et al., 1999, Goodess et al., 2003, Campolongo et al., 2007, Nordhaus, 1994, Nordhaus, 2008, Kypreos, 2008, Johansson, 2011). Overall these analyses overlook the potential for multiple parameters in an IAM to interactively influence the outcomes and, consequently, may lead to incorrect inferences as to which parameters or factors most strongly influence key uncertainties (Saltelli and D'Hombres, 2010).

We focus our sensitivity analysis on the globally-aggregated IAM, the Dynamic Integrated model of Climate and the Economy (DICE) (Nordhaus, 1994, Nordhaus and Boyer, 2000, Nordhaus, 2008), and extend the uncertainty and sensitivity analysis reported in Nordhaus (2008). Our purpose is to demonstrate that for IAMs, i.e., non-linear models with many exogenous parameters, the uncertainties of model outputs can arise from complex parameter interactions. DICE presents a simple, yet comprehensive, representation of the world where alternative economy-climate scenarios can be tested without having to explicitly model the complexities of the global system. There are multiple potential foci when designing a global sensitivity analysis of an inter-temporal optimization IAM. The choice of the appropriate experimental approach depends on the overall policy question to be answered. For example, one question that might be explored is, how do scenario pathways for a given stabilization goal change across alternative SOWs? This problem is reflective of the majority of IAM studies where the primary focus is on comparing the resulting optimized policy scenario outcomes. Alternatively, we pursue in this study the question, how vulnerable are specific optimized DICE policy scenarios to uncertainties in the exogenous assumptions? By isolating the policy scenarios from the optimization process, we are exploring which exogenous parameters (e.g., population growth, technology efficiency, climate sensitivity) control deviations from the policy costs attained under the assumption of perfect information. We do not recalibrate the model to external data sources for each sampled SOW, do not re-optimize the model for each sampled SOW, and do not claim to assign likelihoods to exogenous parameter combinations. Rather we measure how exogenous parameters, individually and interactively, affect selected policy-relevant model outputs. For a deterministic, perfect foresight model such as DICE, it is arguably quite useful to know the vulnerabilities of a policy solution and to identify the key model parameters that control its performance over time. Our results could also inform subsequent calibration efforts or uncertainty analyses by giving an improved a posteriori understanding of complex, interactive parametric effects.

Here we use the cost benefit form (see Section 2.2 below) of the DICE model as described in Nordhaus (2008). In this form of the model a policy scenario outcome is characterized by the control variables, emission control rates and investment, which optimize the objective function, the sum of the discounted utility of consumption over time, given the constraints applied, such as available fossil fuel resources and limits to atmospheric temperature increases. Emission pathways are endogenous in this form of the model. A different (cost effectiveness) form of this model is employed for the use of pre-specified emission control pathways (Meinshausen et al., 2011a, Rogelj et al., 2012). See Appendix Fig. A.9 for an example of a DICE policy scenario and resulting emissions pathway.

For this study we construct a simulation version of DICE, called CDICE, which reproduces DICE model outcomes for a supplied policy scenario, given the reference values of all exogenous parameters. With this simulation model, we can explore the vulnerability of a fixed policy scenario to the uncertainty in the DICE model's exogenous parameters. We choose three distinctly different DICE policy scenarios to see how parametric sensitivities change for scenarios with different treatments of the trade-offs between climate damages and abatement costs. In this study, we apply the Sobol' method, a global variance-based sensitivity analysis method (Sobol', 2001, Saltelli et al., 2008), to CDICE simulations of each policy scenario. Using the Sobol' method, we choose model outputs (in this case, climate damages and abatement costs) for the analysis. We create ensembles of these model outputs by iteratively running the CDICE simulation model while simultaneously varying a selection of model parameters over specified ranges using Sobol' quasi-random sampling. The Sobol' method is used to decompose the variance of the damage and abatement cost outputs into portions contributed individually or interactively by the sampled parameters.

This exercise demonstrates the importance of understanding the non-separable, interactive parameter dependencies that control uncertain IAM projections. We also contrast our findings with the more typical local sensitivity analysis as performed in Nordhaus (2008). Our results illustrate the consequences of using a priori expert elicitations to reduce the set of parameters analyzed, especially within the context of a one-at-a-time (OAT) sensitivity analysis. The results of this global sensitivity analysis provide a strong argument for comprehensive model diagnostics for IAMs to explicitly account for the parametric interactions between their coupled natural and economic components. Moreover, this study illustrates how the sensitivity ranking of model parameters differs for alternative cost metrics, over time, and for alternative emission control strategies.

In Section 2 we describe the DICE IAM and the CDICE simulation model as well as the policy scenarios used in this study. Section 3 presents the methods used and descriptions of the computation experiments. Results and implications are discussed in Section 4, followed by conclusions in Section 5.