Elsevier

Environmental Modelling & Software

Volume 61, November 2014, Pages 287-296
Environmental Modelling & Software

When do aquatic systems models provide useful predictions, what is changing, and what is next?

https://doi.org/10.1016/j.envsoft.2014.01.009Get rights and content

Highlights

  • Recent changes have resulted from changes technology and the modelling community.

  • Active topics include integration, uncertainty, operationalisation and data assimilation.

  • Emerging approaches more deeply combine statistical and mechanistic thinking.

  • Conditions in which models are likely to provide useful predictions are discussed.

  • We need to honestly and transparently report both model successes and failures.

Abstract

This article considers how aquatic systems modelling has changed since 1995 and how it must change in future if we are to continue to advance. A distinction is made between mechanistic and statistical models, and the relative merits of each are considered. The question of “when do aquatic systems models provide accurate and useful predictions?” is addressed, implying some guidelines for model development. It is proposed that, in general, ecological models only provide management-relevant predictions of the behaviour of real systems when there are strong physical (as opposed to chemical or ecological) drivers. Developments over the past 15 years have included changes in technology, changes in the modelling community and changes in the context in which modelling is conducted: the implications of each are briefly discussed. Current trends include increased uptake of best practice guidelines, increasing integration of models, operationalisation, data assimilation, development of improved tools for skill assessment, and application of models to new management questions and in new social contexts. Deeper merging of statistical and mechanistic modelling approaches through such techniques as Bayesian Melding, Bayesian Hierarchical Modelling and surrogate modelling is identified as a key emerging area. Finally, it is suggested that there is a need to systematically identify areas in which our current models are inadequate. We do not yet know for which categories of problems well-implemented aquatic systems models can (or cannot) be expected to accurately predict observational data and system behaviour. This can be addressed through better modelling and publishing practices.

Introduction

In this article, I reflect on changes in aquatic systems modelling over the past 15–20 years, current trends, and the most promising emerging areas of development. I focus particularly on mechanistic biogeochemical and ecological modelling of lakes, estuaries and coastal systems.

A useful starting point may be Jørgensen's (1995) paper, “State of the art in ecological modelling in limnology.” Jørgensen set out to chart the history of aquatic ecological modelling from the 1920s onwards, and to identify the major challenges in the field for the 1990s and beyond. Somewhat optimistically, he identified the questions, “how can we build a reliable model?”, “can we set up a [best practise] procedure for modelling?” and “how do we select the complexity of models?” as problems of the past, and suggested that the key question for the 1990s was “how can we make models better reflect the properties of ecosystems?”

Already in 1995, aquatic systems modelling was a rapidly growing field, and a field which at that time dominated the pages of Ecological Modelling. I will show here that this growth has continued, though other domains of environmental modelling have also grown rapidly. I will turn around the question, “how can we build a reliable model?” to ask, “when can we build a reliable model?,” or more specifically, “in what circumstances do aquatic systems models provide accurate and useful predictions?” Finally, I will ask, “what has changed since 1995?”, “what is happening now,” and “what is coming next?”

To provide some context to the later discussion of emerging trends, I will begin with a brief discussion of modelling approaches. Several schemes have been proposed to categorise environmental models (Guisan and Zimmermann, 2000, Loehle, 1983), but one reasonably intuitive categorisation is into two classes: mechanistic models and statistical models. The distinction between the two is arguably more conceptual than real, but serves as a useful starting point.

Sharpe (1990) proposed three classes of models: statistical models, mechanistic models and theoretical mathematical models. They argued that statistical models support realism and precision, but are less generalisable, mechanistic models favour reality and generalisability, but compromise on precision, and mathematical models combine generality with precision at a cost to realism. Others have contested this claim, demonstrating that both mechanistic and statistical models vary widely in generalisability and precision (Guisan and Zimmermann, 2000, Korzukhin et al., 1996). Here, we will consider only two classes, as the distinction between mechanistic models and mathematical models (in this sense) lies largely in the complexity and realism of the context within which they are applied.

Mechanistic models are also referred to as process-based, deterministic, physics-based, physiological or causal models (Guisan and Zimmermann, 2000). These models are systems of equations that encapsulate the modeller's understanding of the physical, chemical and biological processes that drive a system. Examples include hydrodynamic models, biogeochemical water quality models and trophic transfer models.

When moving beyond the most basic processes, mechanistic models can become very complex, with heavy data requirements for parameterisation and testing (Perrin et al., 2001). Often, they may prove too complex to be properly verified. In such cases, the appropriate application of models is to explore possible internal interactions or outcomes of change rather than to predict specific outcomes or events (Oreskes et al., 1994). A recent review of aquatic systems models applied to simulation of phosphorus processes (Robson, 2013, Robson, 2014) found that published models are generally getting more complex over time, both in the number of biogeochemical and ecological processes included, and in their physical framework.

While engineers and physicists often favour mechanistic models, statisticians and ecologists often prefer statistical models (also known as empirical or data-based models). Rather than designing systems of equations to reflect a conceptual understanding of the system, pure statistical modellers look for patterns and relationships in the observational data that can be used to make predictions, regardless of the causative processes involved. Examples of statistical models include regression models of all varieties, neural networks, and most Bayesian models.

Though these two schools reflect different approaches to modelling, in practice, the design of statistical models (including the selection of data and relationships to consider) is influenced by a mechanistic conceptualisation of the system, while mechanistic models in fact rely on statistical submodels (that is, statistical models of component processes) and require application of statistical methods to ensure rigorous calibration, sensitivity analysis and performance characterisation.

Models of physical systems, including hydraulic and hydrodynamic models, can be built almost entirely from recognised physical theories (such as conservation of momentum) and approximations of these theories (such as hydrostatic implementations of the Navier–Stokes equations Chorin, 1968), but even hydrodynamic models incorporate empirical treatments of bottom friction and turbulence (Xu and Wright, 1995) and empirically observed coefficients for dispersion of momentum, heat and dissolved substances. Often, these parameters are calibrated to counteract the errors introduced by numerical dispersion and poorly resolved bathymetry.

Applied deterministic models of biogeochemical and ecological processes, however, are composed mainly of simple statistical models bolted together into a complex, mechanistic framework. For example, with the exception of those few models that simulate variable intracellular carbon storages and model light interception as a function of inherent optical properties (Baird et al., 2013), aquatic systems models that simulate the effect of light on primary production include P–I (photosynthesis–irradiance) curves (Jassby and Platt, 1976) or minor variations on standard P–I curves, to relate growth rates of phytoplankton or other plants to ambient light conditions. These curves are typically obtained by (statistically) fitting hyperbolic curves through laboratory-derived measurements of phytoplankton responses to varied irradiance. This relationship is then combined with other, similarly empirically-based submodels into a mechanistic biogeochemical model.

Once the complete model has been constructed, it is usual practise for parameter values to be further tuned by calibration of the model as a whole (Ward et al., 2010). The implications and statistical properties of the underlying empirical process representations are rarely considered further. This matters, because information used in building the model is lost in calibration and sensitivity analysis phases, resulting in neglect of structural uncertainty and exaggeration of parameter uncertainty. New approaches have recently emerged that make better use of this information (Cheng et al., 2010, Chiu and Westveld, 2013, Chiu and Gould, 2010), and these will be discussed in Section 4.

Primarily statistical models might be a better choice than primarily mechanistic models when:

  • There are sufficient data to support a robust statistical analysis; and

  • The system is not moving beyond the range of the historical observations (in this sense, the role of statistical modelling approaches might be said to be chiefly interpolative); and

  • The role of any positive or negative feedbacks in the system is small.

In such cases, statistical models are likely to be simpler (i.e. involve fewer equations and parameters), cheaper (i.e. require less modeller time to implement and lesser computational resources) and able to provide more accurate predictions (for any given standard measure of model performance) than mechanistic models, even when the system is well understood on a conceptual and mechanistic level (e.g. Altunkaynak and Wang, 2011, Hakanson, 1997).

Statistical models, in comparison with mechanistic models, also more readily yield estimates of uncertainty such as confidence intervals. For a given set of inputs, a traditional mechanistic model will yield only a single, deterministic output set. Estimates of input and parameter uncertainty can be obtained from this type of model only by conducting many model runs with varying inputs and parameters. The computational demands of aquatic systems models that rely on computational fluid dynamics and/or have a large number (sometimes hundreds) of parameters place real limits on the approaches that can be used. Morris et al. (2014) discuss these issues and present a successful global sensitivity analysis of a complex marine ecosystem model.

Altunkaynak and Wang (2011) conducted one of the few direct comparisons of mechanistic and statistical models in an earth science application, comparing a hydrodynamic model to predict variations in suspended sediment concentrations in a coastal system with a Geno-Kalman Filtering and fuzzy logic models of the same system. The most accurate predictions (as measured by mean squared error and coefficient of efficiency comparisons with observations) were produced by the Geno-Kalman Filtering model, followed by the fuzzy logic model, both of which were conceptually simpler than the hydrodynamic model. The authors note, however, that the simpler models were less flexible than the hydrodynamic model in terms of the range of predictions they could generate: having been tailored to produce local predictions at a specific point, they could not generate spatial predictions, nor simulate the dynamic variability of suspended sediment concentrations. In other words, these models could not readily be applied to answer new questions about the system, as the mechanistic model could.

Statistical models are also a necessary starting point when the system is not yet understood well enough to allow a mechanistic model to be built (e.g. Cloern and Jassby, 2010), as identifying the existence and nature of empirically observable relationships is the first step towards understanding causation. Statistical models may also be appropriate when it is not feasible to obtain the necessary input data to drive a mechanistic model. For example, it may often be possible to estimate the salinity of coastal floodwaters from remotely sensed colour, without needing to measure bathymetry or river flow or to simulate the complex mechanistic interactions between flow, mixing, salinity and coloured dissolved organic matter.

Primarily mechanistic models, on the other hand, might be preferred when:

  • The system is well understood and can be described with a relatively simple set of process equations, with few unconstrained parameters; and

  • It is necessary to extrapolate beyond the range of historical observations (e.g. to predict how flow will change if a weir is built or how a system might respond to long-term climate change) – i.e. the model is to be used for prognosis (e.g. Korzukhin et al., 1996); or

  • It is desirable to use the model to quantify underlying processes (for example, the relative contributions of dreissenids and macrophytes to a phosphorus budget (Kim et al., 2013) or cyanobacteria to a nitrogen budget (Robson et al., 2013)) and not just a final, observable result (water-column phosphorus or nitrogen concentrations); or

  • The goal is to falsify a hypothesised conceptual understanding of how the system functions, which can be represented mechanistically; or

  • Strong feedback loops exist: such feedbacks cannot usually be accounted for in statistical models.

Section snippets

When do aquatic systems models provide accurate and useful predictions?

There is a wealth of recent literature regarding how to characterise the performance of environmental models (e.g. Bennett et al., 2013). As Alexandrov et al. (2011) point out, what is considered a good model depends on the objectives of the study, so that different types of assessment are likely to be appropriate for models that are designed to explore scientific theories than for models designed to aid management decisions.

There may be social, educational or exploratory applications for which

Technological progress

Many of the advances in aquatic systems modelling since 1995 years have been simply a matter of taking advantage of technological improvements – faster computers with more memory and multiple processors, faster, cheaper storage, better graphics, improved GIS and web services – to do the things we wanted to do twenty years ago. The technology has allowed models of higher resolution to be applied to larger geographic areas. It has allowed greater use of models with higher dimensionality (such as

What is happening right now?

Current trends leading to improvements in environmental modelling include:

  • Increased uptake of best practice modelling guidelines. The 2006 Environmental Modelling & Software position paper, “Ten steps in development and evaluation of models,” (Jakeman et al., 2006) for instance, has been cited 258 times in Web of Science (as at 10 December 2013), including only 15 citations in 2007 but 46 in 2013. The more recent position paper, “Characterising the performance of environmental models” (Bennett

What still needs to happen?

A major problem in the current state of the art is that we still do not know – beyond the level of gut instinct – for which categories of problems in aquatic systems science our current models can be expected to accurately predict observational patterns, and for which categories of problems they are likely to fail to provide useful predictions. We cannot yet reliably predict the predictive capacity of our models.

This situation could be improved with a more thorough understanding of what

Acknowledgements

Thanks to the CSIRO Water for a Healthy Country National Research Flagship for supporting the thinking time behind this commentary. An earlier version of this paper was presented at the “Nutrient dynamics in riverine estuaries: understanding, modelling and managing inputs” symposium held at the University of Technology, Sydney, in May 2012, with support from the NSW Government Environmental Trust.

References (109)

  • A. Castelletti et al.

    Visualization-based multi-objective improvement of environmental decision-making using linearization of response surfaces

    Environ. Model. Softw.

    (2010)
  • S.W. Chung et al.

    Modelling the propagation of turbid density inflows into a stratified lake: Daecheong Reservoir, Korea

    Environ. Model. Softw.

    (2009)
  • N.M.J. Crout et al.

    Is my model too complex? Evaluating model formulation using model reduction

    Environ. Model. Softw.

    (2009)
  • Y.L. deReynier et al.

    Bringing stakeholders, scientists, and managers together through an integrated ecosystem assessment process

    Mar. Policy

    (2010)
  • M.A.M. Friedrichs et al.

    Ecosystem model complexity versus physical forcing: quantification of their relative impact with assimilated Arabian Sea data

    Deep-Sea Res. Part Ii-Topical Stud. Oceanogr.

    (2006)
  • E.A. Fulton et al.

    Mortality and predation in ecosystem models: is it important how these are expressed?

    Ecol. Model.

    (2003)
  • D.A. Greenberg et al.

    Resolution issues in numerical models of oceanic and coastal circulation

    Cont. Shelf Res.

    (2007)
  • A. Guisan et al.

    Predictive habitat distribution models in ecology

    Ecol. Model.

    (2000)
  • L. Hakanson

    Modelling of radiocesium in lakes – on predictive power and lessons for the future

    Freshw. Estuar. Radioecol.

    (1997)
  • L. Hakanson

    On the principles and factors determining the predictive success of ecosystem models, with a focus on lake eutrophication models

    Ecol. Model.

    (1999)
  • M. Heymann

    Understanding and misunderstanding computer simulation: the case of atmospheric and climate science-An introduction

    Stud. Hist. philosophy Mod. Phys.

    (2010)
  • A.J. Jakeman et al.

    Ten iterative steps in development and evaluation of environmental models

    Environ. Model. Softw.

    (2006)
  • X.M. Ji et al.

    Process study of circulation in the Pearl River Estuary and adjacent coastal waters in the wet season using a triply-nested circulation model

    Ocean. Model.

    (2011)
  • J.K. Jolliff et al.

    Summary diagrams for coupled hydrodynamic-ecosystem model skill assessment

    J. Mar. Syst.

    (2009)
  • S.E. Jorgensen

    State-of-the-Art of ecological modeling in limnology

    Ecol. Model.

    (1995)
  • D.K. Kim et al.

    Improving the representation of internal nutrient recycling with phosphorus mass balance models: a case study in the Bay of Quinte, Ontario, Canada

    Ecol. Model.

    (2013)
  • M. Kulawiak et al.

    Interactive visualization of marine pollution monitoring and forecasting data via a Web-based GIS

    Comput. Geosci.

    (2010)
  • G.F. Laniak et al.

    Integrated environmental modeling: a vision and roadmap for the future

    Environ. Model. Softw.

    (2013)
  • C. Letellier et al.

    Analysis of the dynamics of a realistic ecological model

    Chaos Solit. Fractals

    (2002)
  • N. Margvelashvili et al.

    Satellite data assimilation and estimation of a 3D coastal sediment transport model using error-subspace emulators

    Environ. Model. Softw.

    (2013)
  • N. Margvelashvili et al.

    Sequential data assimilation in fine-resolution models using error-subspace emulators: theory and preliminary evaluation

    J. Mar. Syst.

    (2012)
  • M. Mateus

    A process-oriented model of pelagic biogeochemistry for marine systems. Part I: model description

    J. Mar. Syst.

    (2012)
  • K.B. Matthews et al.

    Raising the bar? – the challenges of evaluating the outcomes of environmental modelling and software

    Environ. Model. Softw.

    (2011)
  • D.J. Morris et al.

    Global sensitivity analysis of an end-to-end marine ecosystem model of the North Sea: factors affecting the biomass of fish and benthos

    Ecol. Model.

    (2014)
  • A. O'Hagan

    Probabilistic uncertainty specification: overview, elaboration techniques and their application to a mechanistic model of carbon flux

    Environ. Model. Softw.

    (2012)
  • M. Omlin et al.

    Biogeochemical model of Lake Zurich: sensitivity, identifiability and uncertainty analysis

    Ecol. Model.

    (2001)
  • K. Park et al.

    Three-dimensional hydrodynamic-eutrophication model (HEM-3D): application to Kwang-Yang Bay, Korea

    Mar. Environ. Res.

    (2005)
  • R. Paudel et al.

    Does increased model complexity improve description of phosphorus dynamics in a large treatment wetland?

    Ecol. Eng.

    (2012)
  • J. Pearl

    Fusion, propagation, and structuring in Belief networks

    Artif. Intell.

    (1986)
  • C. Perrin et al.

    Does a large number of parameters enhance model performance? Comparative assessment of common catchment model structures on 429 catchments

    J. Hydrol.

    (2001)
  • N.G. Plant et al.

    The effect of bathymetric filtering on nearshore process model results

    Coast. Eng.

    (2009)
  • P.J. Radtke et al.

    A Bayesian strategy for combining predictions from empirical and process-based models

    Ecol. Model.

    (2006)
  • B.J. Robson

    State of the art in modelling of phosphorus cycles in aquatic systems: review, criticisms and commentary

    Environ. Model. Softw.

    (2014)
  • B.J. Robson et al.

    Ten steps applied to development and evaluation of process-based biogeochemical models of estuaries

    Environ. Model. Softw.

    (2008)
  • R.P. Silberstein

    Hydrological models are so good, do we still need data?

    Environ. Model. Softw.

    (2006)
  • B. Sterk et al.

    How, when, and for what reasons does land use modelling contribute to societal problem solving?

    Environ. Model. Softw.

    (2011)
  • C.A. Stow et al.

    Skill assessment for coupled biological/physical models of marine systems

    J. Mar. Syst.

    (2009)
  • A. Voinov et al.

    Modelling with stakeholders

    Environ. Model. Softw.

    (2010)
  • I. Alameddine et al.

    Optimizing an estuarine water quality monitoring program through an entropy-based hierarchical spatiotemporal Bayesian framework

    Water Resour. Res.

    (2013)
  • J.I. Allen et al.

    Linking physiology to ecology: towards a new generation of plankton models

    J. Plankton Res.

    (2011)
  • Cited by (0)

    Thematic Issue on Novel Approaches to Challenges in Aquatic Ecosystem Modelling.

    View full text