Skip to main content
SpringerLink
Log in

Scalable enrichment of mobility data with weather information

  • Published:
GeoInformatica Aims and scope Submit manuscript

Abstract

More and more real-life applications for mobility analytics require the joint exploitation of positional information of moving objects together with weather data that correspond to the movement. In particular, this is evident in fleet management applications for improved routing and reduced fuel consumption, in the maritime domain for more accurate trajectory prediction, as well as in air-traffic management for predicting regulations and reducing delays. Motivated by such applications, in this paper, we present a system for the enrichment of mobility data with weather information. Our main application scenario concerns streaming positional information (such as GPS traces of vehicles) that is collected and is enriched in an online fashion with stored weather data. We present the system architecture of a centralized version that runs on a single machine and exploits caching to improve its efficiency. Also, we extend our approach to a parallel implementation on top of Apache Kafka, which can scale to hundreds of thousands of processed records when provided with more computing nodes. Furthermore, we present extensions of our system for: (a) enrichment of more complex geometries than point data, and (b) providing linked RDF data as output. Our experimental evaluation on a medium-sized cluster shows the scalability of our approach in terms of number of processed records per second.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. Most notably, EU H2020 projects: Track&Know (https://trackandknowproject.eu/), BigDataStack (https://bigdatastack.eu/) and datAcron (http://datacron-project.eu/).

  2. http://weather.mailasail.com/Franks-Weather/Grib-Files-Getting-And-Using

  3. https://www.ncdc.noaa.gov/data-access/model-data/model-datasets

  4. https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/numerical-weather-prediction

  5. https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs

  6. https://www.unidata.ucar.edu/software/thredds/current/netcdf-java/documentation.htm

  7. https://okeanos-knossos.grnet.gr

  8. https://github.com/SciSpark/SciSpark

  9. The conversion of GRIB files to Network Common Data Form format is due to the underlying third-party library, as it does not currently support reading GRIB files from HDFS

References

  1. Batty E (2017) Data analytics enables advanced AIS applications. In: Mobility analytics for spatio-temporal and social data - first international workshop, MATES 2017, munich, germany, september 1, 2017, revised selected papers, pp 22–35

  2. Claramunt C, Ray C, Camossi E, Jousselme A, Hadzagic M, Andrienko GL, Andrienko NV, Theodoridis Y, Vouros GA, Salmon L (2017) Maritime data integration and analysis: recent progress and research challenges. In: Proceedings of the 20th international conference on extending database technology, EDBT 2017, Venice, Italy, March 21-24, 2017., pp 192–197

  3. Dodge S, Bohrer G, Weinzierl R, Davidson S, Kays R, Douglas D, Cruz S, Han J, Brandes D, Wikelski M (2013) The environmental-data automated track annotation (env-data) system: Linking animal tracks with environmental data. Movement Ecology 1. https://doi.org/10.1186/2051-3933-1-3

  4. Dong XL, Srivastava D (2015) Big Data Integration. Synthesis Lectures on Data Management Morgan & Claypool Publishers

  5. Doulkeridis C, Pelekis N, Theodoridis Y, Vouros GA (2017) Big data management and analytics for mobility forecasting in datacron. In: Proceedings of the workshops of the EDBT/ICDT 2017 joint conference (EDBT/ICDT 2017), Venice, Italy, March 21-24, 2017

  6. Gooch R, Chandrasekar V (2017) Integration of real-time weather radar data and internet of things with cloud-hosted real-time data services for the geosciences (CHORDS). In: 2017 IEEE International geoscience and remote sensing symposium, IGARSS 2017, fort worth, TX, USA, July 23-28, 2017, pp 4519–4521

  7. Kolokolov YV, Monovskaya AV, Volkov V, Frolov A (2017) Intelligent integration of open-access weather-climate data on local urban areas. In: 9Th IEEE international conference on intelligent data acquisition and advanced computing systems: technology and applications, IDAACS 2017, bucharest, romania, september 21-23, 2017, pp 465–470

  8. Koutroumanis N, Santipantakis GM, Glenis A, Doulkeridis C, Vouros GA (2019) Integration of mobility data with weather information. In: Proceedings of the workshops of the EDBT/ICDT 2019 joint conference, EDBT/ICDT 2019, Lisbon, Portugal, March 26, 2019

  9. Lundblad P, Eurenius O, Heldring T (2009) Interactive visualization of weather and ship data. In: 13Th international conference on information visualisation, IV 2009, 15-17 july 2009, barcelona, spain, pp 379–386

  10. Pelekis N, Theodoridis Y (2014) Mobility data management and exploration. Springer. https://doi.org/10.1007/978-1-4939-0392-4

  11. Santipantakis GM, Glenis A, Kalaitzian N, Vlachou A, Doulkeridis C, Vouros GA (2018) FAIMUSS: Flexible data transformation to RDF from multiple streaming sources. In: Proceedings of the 21th International Conference on Extending Database Technology, EDBT 2018, Vienna, Austria, March 26-29, 2018., pp 662–665

  12. Santipantakis GM, Glenis A, Patroumpas K, Vlachou A, Doulkeridis C, Vouros GA, Pelekis N, Theodoridis Y (2020) SPARTAN: Semantic integration of big spatio-temporal data from streaming and archival sources. Future Gener Comput Syst 110:540–555

    Article  Google Scholar 

  13. Santipantakis GM, Kotis KI, Vouros GA, Doulkeridis C (2018) Rdf-gen: Generating RDF from streaming and archival data. In: Proceedings of the 8th international conference on Web intelligence, mining and semantics, WIMS 2018, Novi Sad, Serbia, June 25-27, 2018, pp 28:1–28:10

  14. dos Santos Mello R, Bogorny V, Alvares LO, Santana LHZ, Ferrero CA, Frozza AA, Schreiner GA, Renso C (2019) MASTER: A multiple aspect view on trajectories. Trans GIS 23(4):805–822

    Google Scholar 

  15. da Silva Brum Bastos V, Long JA, Demsar U (2018) Weather effects on human mobility: a study using multi-channel sequence analysis. Comput Environ Urban Syst 71:131–152

    Article  Google Scholar 

  16. Soares A, Dividino R, Abreu F, Brousseau M, Isenor AW, Webb S, Matwin S (2019) Crisis: Integrating ais and ocean data streams using semantic web standards for event detection. In: 2019 International conference on military communications and information systems (ICMCIS), pp 1–7

  17. Vouros GA, Vlachou A, Santipantakis GM, Doulkeridis C, Pelekis N, Georgiou HV, Theodoridis Y, Patroumpas K, Alevizos E, Artikis A, Claramunt C, Ray C, Scarlatti D, Fuchs G, Andrienko GL, Andrienko NV, Mock M, Camossi E, Jousselme A, Garcia JMC (2018) Big data analytics for time critical mobility forecasting: Recent progress and research challenges. In: Proceedings of the 21th international conference on extending database technology, EDBT 2018, Vienna, Austria, March 26-29, 2018., pp 612–623

Download references

Acknowledgements

This work is supported by projects datAcron, Track&Know, BigDataStack, and MASTER (Marie Sklowdoska-Curie), which have received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 687591, No 780754, No 779747 and No 777695 respectively.

Author information

Authors and Affiliations

  1. Department of Digital Systems, School of Information and Communication Technologies, University of Piraeus, Pireas, Greece

    Nikolaos Koutroumanis, Georgios M. Santipantakis, Apostolos Glenis, Christos Doulkeridis & George A. Vouros

Authors
  1. Nikolaos Koutroumanis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Georgios M. Santipantakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Apostolos Glenis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christos Doulkeridis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. George A. Vouros
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nikolaos Koutroumanis.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Koutroumanis, N., Santipantakis, G.M., Glenis, A. et al. Scalable enrichment of mobility data with weather information. Geoinformatica 25, 291–309 (2021). https://doi.org/10.1007/s10707-020-00423-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10707-020-00423-w

Keywords

Search

Navigation