Metrics to identify meaningful downscaling skill in WRF simulations of intense rainfall events
Introduction
Forecast verification is a core activity for numerical weather prediction, providing information on model skill when simulating weather into the future. As weather eventuates, forecasts can be compared against the recorded weather for a particular time and location. In a climate change context, the importance of accuracy in terms of timing and geographical precision is relaxed, as simulations are representations of plausible climates; simulating weather far beyond any predictive signal in the internal weather system. Skill in regional climate change projection is typically assessed by comparing simulated climatologies with observed ones for multi-decadal periods (Fowler et al., 2005). For methods such as dynamical downscaling (the use of a dynamical model to add regional detail to global climate model projections), long simulation times are associated with large computing costs. Hence, conducting multi-decadal model runs for the purpose of assessing skill of a particular model setup (e.g. the use of different parameter schemes) can be prohibitively expensive from a computing resource point a view.
Nevertheless, assessments of skill in methods used for deriving regional projections are desirable, as they can inform the level of confidence attributed to the simulated future climate. But what verification methods are most appropriate in a climate change context? In a NWP context, the ability to capture timing and extent of an event is central to a skilful simulation; hence metrics evaluated on co-locations are meaningful. This is not necessarily true for models used in a climate change context, where characteristics such as spatial dependence or full distributional representation might be more relevant. Further, the type or manifestation of model skill required by a researcher can differ depending on the intended application and should be reflected in the choice of model verification metrics. Here, different metrics and analyses are used to examine the performance of the more complex microphysics schemes available for the Weather Research and Forecasting (WRF) model (Skamarock and Klemp, 2008). The underlying motivation being an intent to identify the model configuration best suited for research on water resource planning under climate change for southeast Australia.
The meso-scale numerical model WRF hosted at the United State's National Centre for Atmospheric Research (NCAR) supports a wide range of modelling applications within the weather and climate community (Caldwell et al., 2009, Chotamonsak et al., 2011, Coniglio et al., 2013, Del Genio et al., 2012, Done et al., 2004; Heikkila et al., 2011; Kain et al., 2006, Leung et al., 2006, Ma et al., 2012). To accommodate its many uses it has a flexible structure that allows users to select physics and dynamics settings that optimise the model for their particular needs. Selecting parameter schemes and other settings is not necessarily straightforward when multiple theoretically comparable schemes are available to the user. Though numerous assessments of schemes and settings exist in the literature (Evans et al., 2012, Jankov et al., 2005, Liu et al., 2012), some testing is sometimes required to assess performance in a particular geographical area for which there is limited advice to be drawn from the literature.
The complex microphysics schemes become particularly relevant when the researcher seeks to simulate rainfall at a fine resolution (∼4 km), when the resolution is such that convection (at the grid scale resolution) is explicitly simulated by the model rather than parameterised (Kain et al., 2006). In a climate change context, recent research indicate that very fine (∼1.5 km) convective-allowing (or permitting) simulations could be particularly important, as these models have the ability to provide a more realistic simulation of hazardous high intensity rainfall events (Kendon et al., 2012) and thus theoretically an improved understanding of plausible impacts to extreme rainfall events under changing greenhouse concentrations (Kendon et al., 2014, Westra et al., 2014).
However, very fine resolution regional climate models are computationally very expensive in terms of processing (increased iterations) and storage (volume of output), as climate simulations require significantly longer simulations than NWP models (that simulate for temporal domains bounded by days rather than decades) to allow detection of a change in the climate signal. Thus, with finite computing resources, researchers conducting downscaling with meso-scale dynamical models have to make important decisions around the extent and the resolution of the spatial and temporal model domain of their experiment. This is particularly relevant if output is intended to inform on policy, since the researchers should balance the potential added value of realism (by increasing the resolution of an experiment) against adequately representing the typically large uncertainty in projected rainfall stemming from lack of knowledge about future emissions, and inadequacies in simulating the global climate response to these emissions (see discussion in Ekström et al. (2015) in their appraisal of downscaling methods). In short, increasing the model resolution may hamper ability to sample the climate signal contained in an ensemble of global climate models (i.e. conducting downscaling only on a small sample of global models).
If limitations in computing resources exist, it is sensible to first test whether very fine convection-allowing configurations can add value relative to the intended application. Whilst recent experiments indicate that this is indeed the case for extreme rainfall impact assessments, it is not immediately clear that the finer resolution experiments add value for impact research in a water resource application; where the scale relevant to the topic is greater both in time (seasonal to annual rainfall) and scale (typically rainfall across multiple catchments) compared to the scale relevant to represent individual rainfall events generating extreme rainfall.
To assess whether fine resolution experiments are application appropriate for water resource impact research in southeast Australia, there is an interest in using WRF to conduct a multi-year simulation to assess relative differences in parameterised versus explicitly resolved convection. Given the multiple configuration options available, an assessment of physics parameter scheme options is desirable to ensure that the most appropriate configuration is used to conduct a multi-year simulation for current climate (a long simulation period being required to derive robust estimates about average climate conditions, so called climatologies). Of course, crucial to the assessment is the definition of ‘appropriate’. What skills or characteristics are desired of the model and what metrics can be used to quantify these skills to enable a performance ranking of differently configured models.
This paper demonstrates learnings from a case study in southeast Australia, where application relevant combinations of WRF physics scheme combinations are assessed on their ability to capture gross spatial, temporal and distributional characteristics desired from rainfall fields intended for impact work in the water resource domain.
Section snippets
WRF setup
The simulations presented here are generated using WRF version 3.6.1 with the Advanced Research WRF (ARW) dynamical core. A one-way telescopic nest with 3 spatial domains using a Lambert conformal projection is used. The outer nest (D01) cover the Australian continent into the Southern Ocean with a 50 km resolution, the intermediate domain (D02) focus on southeast Australia and coastal waters with a 10 km resolution, and the innermost domain (D03) include the southernmost part of the Great
Daily rainfall characteristics
For comparison with observed daily rainfall fields WRF simulations for D03 were regridded to the regular 0.05° latitude and longitude coordinates of AWAP. If aggregating rainfall across the entire domain, rainfall totals and the sequencing of events are overall well captured for the case studies (Fig. 3). For the winter case study (C1), daily rainfall totals are somewhat lower than observed on day 12 for the majority of ensemble members with the exception of best fit ensemble N1; the best fit
Discussion and conclusions
Ensemble members assessed here differ only in two aspects: having a different combination of PBL and MP physics schemes. All tested schemes are standard options of WRF and as such are all expected to do well. The purpose of this assessment is to identify a configuration with characteristics that are desirable for water resource impact assessments using easy to implement metrics and analyses. Qualities that are deemed relevant are: timing of events (an indication of how the model simulates the
Acknowledgements
This research was undertaken with the assistance of resources from the National Computational Infrastructure (NCI), which is supported by the Australian Government. The work is conducted as part of the Victorian Climate Initiative (VicCI) funded by the Victorian Government's Department of Environment, Land, Water and Planning (DELWP), the Australian Bureau of Meteorology and the Australian Commonwealth Science, Industry and Research Organisation (CSIRO).
References (51)
- et al.
Models of daily rainfall cross-correlation for the United Kingdom
Environ. Model. Softw.
(2013) - et al.
Temporal and spatial variability of rainfall at the urban hydrological scale
J. Hydrol.
(2012) - et al.
New estimates of future changes in extreme rainfall across the UK using regional climate model integrations. 1. Assessment of control climate
J. Hydrol.Hydrol.
(2005) - et al.
Variography of rainfall accumulation in presence of advection
J. Hydrol.
(2012) - et al.
A time-split nonhydrostatic atmospheric model for weather research and forecasting applications
J. Comput. Phys.
(2008) - et al.
Evaluation of a WRF dynamical downscaling simulation over California
Clim. Change
(2009) New developments of the intensity-scale technique within the spatial verification methods intercomparison project
Weather Forecast.
(2010)- et al.
Geostatistics: Modeling Spatial Uncertainty
(2012) - et al.
Projected climate change over Southeast Asia simulated using a WRF regional climate model
Atmos. Sci. Lett.
(2011) - et al.
Verification of convection-allowing WRF model forecasts of the planetary boundary layer using sounding observations
Weather Forecast.
(2013)