Elsevier

Environmental Modelling & Software

Volume 119, September 2019, Pages 182-196
Environmental Modelling & Software

A stream computing approach for live environmental models using a spatial data infrastructure with a waterlogging model case study

https://doi.org/10.1016/j.envsoft.2019.06.009Get rights and content

Highlights

  • A framework for coupling Sensor Web and models in a stream computing environment.

  • O-Stream information model facilitates the observation stream processing in SPARK.

  • Case study of the live waterlogging model to support the model cloud.

Abstract

Traditional environmental model simulations often use archived data as inputs. Recent advancement of Sensor Web technologies in Spatial Data Infrastructures (SDIs) allows real-time observations to be fed into models to generate “live” models. A key challenge is how to efficiently process observation streams in models, which is particularly important in time-critical cases like disaster management. This paper presents an observation stream computing model for live modelling, which couples Sensor Web and models in stream computing environment to provide timely decision-support information. Observation Streams are proposed as information models to deal with observation stream processing. The approach shows how MapReduce and Apache Spark stream processing can be leveraged to support coupling of observation streams and models. The approach is applied in a disaster management case, where in-situ observation streams are processed to compute the waterlogging information in near real time. The results illustrate applicability and effectiveness of the approach.

Section snippets

Software availability

Program name: EMGeoStreaming

Developer: Boyi Shangguan

Contact address: [email protected]

Year first required: 2018

Software required: Apache Spark 2.1.0 or later, Apache Kafka 2.11

Programming language: Java 1.8, Scala 2.11

Program size: 16.6 MB (compressed source code)

Availability: https://github.com/whu-ypfamily/EMGeoStreaming

Cost: Free of charge

The motivating example of waterlogging disaster

Environmental models play key roles in geographical process simulation, whose goal is to dynamically show processes of geographical phenomena with GIS technologies to provide decision-support information. The waterlogging disaster management is a typical case of geographical process simulation, which is taken as the motivating example in this paper.

As one of the most destructive natural hazards, the waterlogging disaster has been studied intensively and there is a number of simulation models

Background

This section introduces two basic technologies that are related to our approach: Sensor Web and Stream computing.

An observation stream computing model

The observation stream from Sensor Web can be fed into the stream computing framework by using a set of extended RDD types, which constitute a distributed in-memory model for observation stream computing. This section describes how observation stream can be modelled (Section 4.1), and how the model can provide in-house support for stream computing (Section 4.2). A theoretical analysis of the performance is provided in Section 4.3 to help understand how to achieve low latency in observation

Walk-through for waterlogging information derivation

The use case of waterlogging disaster management described in Section 2 is used to illustrate the approach. The walk-through example includes the transformation of environmental models as a set of operators in the event model (Section 5.1) and enactment of O-Stream computing (Section 5.2).

Implementation

In this section, we introduce the implementation of a prototype system (Section 6.1) for waterlogging disaster management based on the model and framework proposed in Section 4. Experimental analysis and discussion are given in Section 6.2.

Conclusions and future work

This paper presents an approach for feeding real-time observation data of the Sensor Web into environmental models with stream computing technologies to generate “live” models. The approach makes it possible to derive timely decision-support information in time-critical environmental events. A case on waterlogging disaster management is used to illustrate the approach.

In the paper, an interoperable, extensible, and scalable framework is proposed to couple Sensor Web with environment model in a

Acknowledgements

We appreciate the reviewers and editors for their constructive comments that helped improve the quality of the paper. The work was supported by Major State Research Development Program of China (No. 2017YFB0503704), National Natural Science Foundation of China (No. 41722109, 61825103, 91738302), Hubei Provincial Natural Science Foundation of China (No. 2018CFA053), Nature Science Foundation Innovation Group Project of Hubei Province, China (No. 2016CFA003), and Wuhan Yellow Crane Talents

References (52)

  • E. Todini

    Rainfall-runoff modeling — past, present and future

    J. Hydrol.

    (1988)
  • P. Yue et al.

    A geoprocessing workflow system for environmental monitoring and integrated modelling

    Environ. Model. Softw

    (2015)
  • A. Zerger et al.

    Impediments to using GIS for real-time disaster decision support

    Comput. Environ. Urban Syst.

    (2003)
  • S. Zhang et al.

    An urban storm-inundation simulation method based on GIS

    J. Hydrol.

    (2014)
  • M.H. Ali et al.

    Online Visualization of Geospatial Stream Data Using the WorldWide Telescope

    (2011)
  • Y. Ao et al.

    26 PFLOPS Stencil Computations for Atmospheric Modeling on Sunway TaihuLight. 2017 IEEE International Parallel And Distributed Processing Symposium

    (2017)
  • Apache
  • Apache
  • A. Auger et al.

    Sensor observation streams within cloud-based IoT platforms: challenges and directions

  • B. Babcock et al.

    Models and issues in data stream systems

  • M. Batty

    Big Data and the City

    (2016)
  • K. Beven

    Robert E. Horton's perceptual model of infiltration processes

    Hydrol. Process.

    (2004)
  • A. Biem et al.

    IBM Infosphere Streams for Scalable, Real-Time, Intelligent Transportation Services

    (2010)
  • M. Botts et al.

    OGC® Sensor Web Enablement: Overview and High Level Architecture

    (2007)
  • A. Bröring et al.

    New generation sensor web enablement

    Sensors

    (2011)
  • A. Bröring et al.

    OGC® sensor observation service interface standard

    (2012)
  • Cited by (8)

    View all citing articles on Scopus
    View full text