Metrics to identify meaningful downscaling skill in WRF simulations of intense rainfall events

https://doi.org/10.1016/j.envsoft.2016.01.012Get rights and content

Highlights

  • Six configurations of a regional climate model are assessed for water resource impact work.

  • Model ensemble comprises three microphysics and two planetary boundary layer parameter schemes.

  • Meaningful metrics are derived from an understanding of model application needs.

  • Metrics targeting spatial characteristics in daily rainfall fields are applied to determine skill.

  • Variography analysis quantifies desirable characteristics in daily rainfall patterns.

Abstract

Dynamical downscaling attempts to provide regional detail to climate change projections that subsequently can be used as input to climate change impact models. However, unlike forecasts by numerical weather prediction models, downscaled projections cannot be tested for skill because the future of interest is decades away. Nevertheless, models can be tested in terms of how well they simulate current weather or climate, thus giving an indication of skill in representing the process of interest. Here, six configurations using different combinations of three microphysics and two planetary boundary layer schemes are assessed on their skill to simulate desired characteristics in daily rainfall fields from three two week simulations in southeast Australia; ‘desired’ meaning desirable in relation to the intended application. Of different metrics and analysis assessed, a metric based on variography analysis, summarising characteristics about spatial variability and dissimilarity, is shown to provide the most informative guidance relative to the desirable characteristics.

Introduction

Forecast verification is a core activity for numerical weather prediction, providing information on model skill when simulating weather into the future. As weather eventuates, forecasts can be compared against the recorded weather for a particular time and location. In a climate change context, the importance of accuracy in terms of timing and geographical precision is relaxed, as simulations are representations of plausible climates; simulating weather far beyond any predictive signal in the internal weather system. Skill in regional climate change projection is typically assessed by comparing simulated climatologies with observed ones for multi-decadal periods (Fowler et al., 2005). For methods such as dynamical downscaling (the use of a dynamical model to add regional detail to global climate model projections), long simulation times are associated with large computing costs. Hence, conducting multi-decadal model runs for the purpose of assessing skill of a particular model setup (e.g. the use of different parameter schemes) can be prohibitively expensive from a computing resource point a view.

Nevertheless, assessments of skill in methods used for deriving regional projections are desirable, as they can inform the level of confidence attributed to the simulated future climate. But what verification methods are most appropriate in a climate change context? In a NWP context, the ability to capture timing and extent of an event is central to a skilful simulation; hence metrics evaluated on co-locations are meaningful. This is not necessarily true for models used in a climate change context, where characteristics such as spatial dependence or full distributional representation might be more relevant. Further, the type or manifestation of model skill required by a researcher can differ depending on the intended application and should be reflected in the choice of model verification metrics. Here, different metrics and analyses are used to examine the performance of the more complex microphysics schemes available for the Weather Research and Forecasting (WRF) model (Skamarock and Klemp, 2008). The underlying motivation being an intent to identify the model configuration best suited for research on water resource planning under climate change for southeast Australia.

The meso-scale numerical model WRF hosted at the United State's National Centre for Atmospheric Research (NCAR) supports a wide range of modelling applications within the weather and climate community (Caldwell et al., 2009, Chotamonsak et al., 2011, Coniglio et al., 2013, Del Genio et al., 2012, Done et al., 2004; Heikkila et al., 2011; Kain et al., 2006, Leung et al., 2006, Ma et al., 2012). To accommodate its many uses it has a flexible structure that allows users to select physics and dynamics settings that optimise the model for their particular needs. Selecting parameter schemes and other settings is not necessarily straightforward when multiple theoretically comparable schemes are available to the user. Though numerous assessments of schemes and settings exist in the literature (Evans et al., 2012, Jankov et al., 2005, Liu et al., 2012), some testing is sometimes required to assess performance in a particular geographical area for which there is limited advice to be drawn from the literature.

The complex microphysics schemes become particularly relevant when the researcher seeks to simulate rainfall at a fine resolution (∼4 km), when the resolution is such that convection (at the grid scale resolution) is explicitly simulated by the model rather than parameterised (Kain et al., 2006). In a climate change context, recent research indicate that very fine (∼1.5 km) convective-allowing (or permitting) simulations could be particularly important, as these models have the ability to provide a more realistic simulation of hazardous high intensity rainfall events (Kendon et al., 2012) and thus theoretically an improved understanding of plausible impacts to extreme rainfall events under changing greenhouse concentrations (Kendon et al., 2014, Westra et al., 2014).

However, very fine resolution regional climate models are computationally very expensive in terms of processing (increased iterations) and storage (volume of output), as climate simulations require significantly longer simulations than NWP models (that simulate for temporal domains bounded by days rather than decades) to allow detection of a change in the climate signal. Thus, with finite computing resources, researchers conducting downscaling with meso-scale dynamical models have to make important decisions around the extent and the resolution of the spatial and temporal model domain of their experiment. This is particularly relevant if output is intended to inform on policy, since the researchers should balance the potential added value of realism (by increasing the resolution of an experiment) against adequately representing the typically large uncertainty in projected rainfall stemming from lack of knowledge about future emissions, and inadequacies in simulating the global climate response to these emissions (see discussion in Ekström et al. (2015) in their appraisal of downscaling methods). In short, increasing the model resolution may hamper ability to sample the climate signal contained in an ensemble of global climate models (i.e. conducting downscaling only on a small sample of global models).

If limitations in computing resources exist, it is sensible to first test whether very fine convection-allowing configurations can add value relative to the intended application. Whilst recent experiments indicate that this is indeed the case for extreme rainfall impact assessments, it is not immediately clear that the finer resolution experiments add value for impact research in a water resource application; where the scale relevant to the topic is greater both in time (seasonal to annual rainfall) and scale (typically rainfall across multiple catchments) compared to the scale relevant to represent individual rainfall events generating extreme rainfall.

To assess whether fine resolution experiments are application appropriate for water resource impact research in southeast Australia, there is an interest in using WRF to conduct a multi-year simulation to assess relative differences in parameterised versus explicitly resolved convection. Given the multiple configuration options available, an assessment of physics parameter scheme options is desirable to ensure that the most appropriate configuration is used to conduct a multi-year simulation for current climate (a long simulation period being required to derive robust estimates about average climate conditions, so called climatologies). Of course, crucial to the assessment is the definition of ‘appropriate’. What skills or characteristics are desired of the model and what metrics can be used to quantify these skills to enable a performance ranking of differently configured models.

This paper demonstrates learnings from a case study in southeast Australia, where application relevant combinations of WRF physics scheme combinations are assessed on their ability to capture gross spatial, temporal and distributional characteristics desired from rainfall fields intended for impact work in the water resource domain.

Section snippets

WRF setup

The simulations presented here are generated using WRF version 3.6.1 with the Advanced Research WRF (ARW) dynamical core. A one-way telescopic nest with 3 spatial domains using a Lambert conformal projection is used. The outer nest (D01) cover the Australian continent into the Southern Ocean with a 50 km resolution, the intermediate domain (D02) focus on southeast Australia and coastal waters with a 10 km resolution, and the innermost domain (D03) include the southernmost part of the Great

Daily rainfall characteristics

For comparison with observed daily rainfall fields WRF simulations for D03 were regridded to the regular 0.05° latitude and longitude coordinates of AWAP. If aggregating rainfall across the entire domain, rainfall totals and the sequencing of events are overall well captured for the case studies (Fig. 3). For the winter case study (C1), daily rainfall totals are somewhat lower than observed on day 12 for the majority of ensemble members with the exception of best fit ensemble N1; the best fit

Discussion and conclusions

Ensemble members assessed here differ only in two aspects: having a different combination of PBL and MP physics schemes. All tested schemes are standard options of WRF and as such are all expected to do well. The purpose of this assessment is to identify a configuration with characteristics that are desirable for water resource impact assessments using easy to implement metrics and analyses. Qualities that are deemed relevant are: timing of events (an indication of how the model simulates the

Acknowledgements

This research was undertaken with the assistance of resources from the National Computational Infrastructure (NCI), which is supported by the Australian Government. The work is conducted as part of the Victorian Climate Initiative (VicCI) funded by the Victorian Government's Department of Environment, Land, Water and Planning (DELWP), the Australian Bureau of Meteorology and the Australian Commonwealth Science, Industry and Research Organisation (CSIRO).

References (51)

  • D.P. Dee et al.

    The ERA-Interim reanalysis: configuration and performance of the data assimilation system

    Q. J. Royal Meteorol. Soc.

    (2011)
  • A.D. Del Genio et al.

    Characteristics of mesoscale organization in WRF simulations of convection during TWP-ICE

    J. Clim.

    (2012)
  • C.V. Deutsch et al.

    GSLIB: Geostatistical Software Library and User's Guide

    (1998)
  • P.J. Diggle et al.

    Model-based Geostatistics

    (2007)
  • J. Done et al.

    The next generation of NWP: explicit forecasts of convection using the weather research and forecasting (WRF) model

    Atmos. Sci. Lett.

    (2004)
  • E.E. Ebert

    Fuzzy verification of high-resolution gridded forecasts: a review and proposed framework

    Meteorol. Appl.

    (2008)
  • E.E. Ebert et al.

    Toward better understanding of the contiguous rain area (CRA) method for spatial forecast verification

    Weather Forecast.

    (2009)
  • M. Ekstrom

    Test of WRF Physics Schemes for Project 6 of the Victorian Climate Initiative: An Outline of Selected Physical Parameterisation Schemes and Other Runtime Options

    (2014)
  • M. Ekström et al.

    An appraisal of downscaling methods used in climate change research

    WIREs Clim Change

    (2015)
  • J. Evans et al.

    Evaluating the performance of a WRF physics ensemble over South-East Australia

    Clim. Dyn.

    (2012)
  • H.J. Fowler et al.

    Multi-model ensemble estimates of climate change impacts on UK seasonal precipitation extremes

    Int. J. Climatol.

    (2009)
  • E. Gilleland et al.

    Intercomparison of spatial forecast verification methods

    Weather Forecast.

    (2009)
  • E. Gilleland et al.

    Verifying forecasts spatially

    Bull. Am. Meteorol. Soc.

    (2010)
  • J. Gilmore et al.

    Extreme precipitation in WRF during the newcastle east coast low of 2007

    Theor. Appl. Climatol.

    (2015)
  • U. Heikkila et al.

    Dynamical downscaling of ERA-40 in complex terrain using the WRF regional climate model

    Clim. Dynam.

    (2011)
  • Cited by (0)

    View full text