Abstract
More and more real-life applications for mobility analytics require the joint exploitation of positional information of moving objects together with weather data that correspond to the movement. In particular, this is evident in fleet management applications for improved routing and reduced fuel consumption, in the maritime domain for more accurate trajectory prediction, as well as in air-traffic management for predicting regulations and reducing delays. Motivated by such applications, in this paper, we present a system for the enrichment of mobility data with weather information. Our main application scenario concerns streaming positional information (such as GPS traces of vehicles) that is collected and is enriched in an online fashion with stored weather data. We present the system architecture of a centralized version that runs on a single machine and exploits caching to improve its efficiency. Also, we extend our approach to a parallel implementation on top of Apache Kafka, which can scale to hundreds of thousands of processed records when provided with more computing nodes. Furthermore, we present extensions of our system for: (a) enrichment of more complex geometries than point data, and (b) providing linked RDF data as output. Our experimental evaluation on a medium-sized cluster shows the scalability of our approach in terms of number of processed records per second.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10707-020-00423-w/MediaObjects/10707_2020_423_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10707-020-00423-w/MediaObjects/10707_2020_423_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10707-020-00423-w/MediaObjects/10707_2020_423_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10707-020-00423-w/MediaObjects/10707_2020_423_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10707-020-00423-w/MediaObjects/10707_2020_423_Fig5_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10707-020-00423-w/MediaObjects/10707_2020_423_Fig6_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10707-020-00423-w/MediaObjects/10707_2020_423_Fig7_HTML.png)
Similar content being viewed by others
Notes
Most notably, EU H2020 projects: Track&Know (https://trackandknowproject.eu/), BigDataStack (https://bigdatastack.eu/) and datAcron (http://datacron-project.eu/).
The conversion of GRIB files to Network Common Data Form format is due to the underlying third-party library, as it does not currently support reading GRIB files from HDFS
References
Batty E (2017) Data analytics enables advanced AIS applications. In: Mobility analytics for spatio-temporal and social data - first international workshop, MATES 2017, munich, germany, september 1, 2017, revised selected papers, pp 22–35
Claramunt C, Ray C, Camossi E, Jousselme A, Hadzagic M, Andrienko GL, Andrienko NV, Theodoridis Y, Vouros GA, Salmon L (2017) Maritime data integration and analysis: recent progress and research challenges. In: Proceedings of the 20th international conference on extending database technology, EDBT 2017, Venice, Italy, March 21-24, 2017., pp 192–197
Dodge S, Bohrer G, Weinzierl R, Davidson S, Kays R, Douglas D, Cruz S, Han J, Brandes D, Wikelski M (2013) The environmental-data automated track annotation (env-data) system: Linking animal tracks with environmental data. Movement Ecology 1. https://doi.org/10.1186/2051-3933-1-3
Dong XL, Srivastava D (2015) Big Data Integration. Synthesis Lectures on Data Management Morgan & Claypool Publishers
Doulkeridis C, Pelekis N, Theodoridis Y, Vouros GA (2017) Big data management and analytics for mobility forecasting in datacron. In: Proceedings of the workshops of the EDBT/ICDT 2017 joint conference (EDBT/ICDT 2017), Venice, Italy, March 21-24, 2017
Gooch R, Chandrasekar V (2017) Integration of real-time weather radar data and internet of things with cloud-hosted real-time data services for the geosciences (CHORDS). In: 2017 IEEE International geoscience and remote sensing symposium, IGARSS 2017, fort worth, TX, USA, July 23-28, 2017, pp 4519–4521
Kolokolov YV, Monovskaya AV, Volkov V, Frolov A (2017) Intelligent integration of open-access weather-climate data on local urban areas. In: 9Th IEEE international conference on intelligent data acquisition and advanced computing systems: technology and applications, IDAACS 2017, bucharest, romania, september 21-23, 2017, pp 465–470
Koutroumanis N, Santipantakis GM, Glenis A, Doulkeridis C, Vouros GA (2019) Integration of mobility data with weather information. In: Proceedings of the workshops of the EDBT/ICDT 2019 joint conference, EDBT/ICDT 2019, Lisbon, Portugal, March 26, 2019
Lundblad P, Eurenius O, Heldring T (2009) Interactive visualization of weather and ship data. In: 13Th international conference on information visualisation, IV 2009, 15-17 july 2009, barcelona, spain, pp 379–386
Pelekis N, Theodoridis Y (2014) Mobility data management and exploration. Springer. https://doi.org/10.1007/978-1-4939-0392-4
Santipantakis GM, Glenis A, Kalaitzian N, Vlachou A, Doulkeridis C, Vouros GA (2018) FAIMUSS: Flexible data transformation to RDF from multiple streaming sources. In: Proceedings of the 21th International Conference on Extending Database Technology, EDBT 2018, Vienna, Austria, March 26-29, 2018., pp 662–665
Santipantakis GM, Glenis A, Patroumpas K, Vlachou A, Doulkeridis C, Vouros GA, Pelekis N, Theodoridis Y (2020) SPARTAN: Semantic integration of big spatio-temporal data from streaming and archival sources. Future Gener Comput Syst 110:540–555
Santipantakis GM, Kotis KI, Vouros GA, Doulkeridis C (2018) Rdf-gen: Generating RDF from streaming and archival data. In: Proceedings of the 8th international conference on Web intelligence, mining and semantics, WIMS 2018, Novi Sad, Serbia, June 25-27, 2018, pp 28:1–28:10
dos Santos Mello R, Bogorny V, Alvares LO, Santana LHZ, Ferrero CA, Frozza AA, Schreiner GA, Renso C (2019) MASTER: A multiple aspect view on trajectories. Trans GIS 23(4):805–822
da Silva Brum Bastos V, Long JA, Demsar U (2018) Weather effects on human mobility: a study using multi-channel sequence analysis. Comput Environ Urban Syst 71:131–152
Soares A, Dividino R, Abreu F, Brousseau M, Isenor AW, Webb S, Matwin S (2019) Crisis: Integrating ais and ocean data streams using semantic web standards for event detection. In: 2019 International conference on military communications and information systems (ICMCIS), pp 1–7
Vouros GA, Vlachou A, Santipantakis GM, Doulkeridis C, Pelekis N, Georgiou HV, Theodoridis Y, Patroumpas K, Alevizos E, Artikis A, Claramunt C, Ray C, Scarlatti D, Fuchs G, Andrienko GL, Andrienko NV, Mock M, Camossi E, Jousselme A, Garcia JMC (2018) Big data analytics for time critical mobility forecasting: Recent progress and research challenges. In: Proceedings of the 21th international conference on extending database technology, EDBT 2018, Vienna, Austria, March 26-29, 2018., pp 612–623
Acknowledgements
This work is supported by projects datAcron, Track&Know, BigDataStack, and MASTER (Marie Sklowdoska-Curie), which have received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 687591, No 780754, No 779747 and No 777695 respectively.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Koutroumanis, N., Santipantakis, G.M., Glenis, A. et al. Scalable enrichment of mobility data with weather information. Geoinformatica 25, 291–309 (2021). https://doi.org/10.1007/s10707-020-00423-w
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10707-020-00423-w