Sboing4Real: A real-time crowdsensing-based traffic management system
Introduction
Geographical Information Systems (GISs) and, more broadly, the development of applications based on or including geo-spatial data is a mature and hot area with several tools, both commercial and open-source, e.g., ArcGIS, PostGIS, GeoSpark [27], HadoopGIS [1], SpatialSpark [26]. In general, these tools and frameworks differ in the queries they support [19] and the quality of maps they utilize. For the latter, popular alternatives include Google Maps and OpenStreetMap,1 which can be considered as data-as-a-service. At the same time, urban trips form a big source of data; e.g., Uber Movement2 shares data from more than 10 billion trips. Developing a system that can process real-time traffic data in order to report and forecast current traffic conditions combines all the elements mentioned above, e.g., modern GIS applications built on top of detailed world maps leveraging real-time big data sources, scalable stores and parallel processing platforms.
The aim of this work is to present the architectural details regarding a novel back-end system developed on behalf of Sboing.3 Sboing is an SME that implements innovative mobile technologies for the collection, processing and exploitation of location and mobility-based data. In particular, it implements an internationally patented, collaborative, crowdsourcing methodology for the collection, processing and distribution of traffic, road and sensory data, aiming to provide improved multi-GNSS (Global Navigation Satellite System) navigation and routing, globally. It offers an app, called UltiNavi, that can be installed on in-vehicle consoles and smartphones. Through this app, Internet-connected users share their location information in real-time and contribute to the collection of real-time traffic data. More specifically, Sboing collaborates with academia in order to extend their system and provide a crowdsensing solution [9] with a view to (i) continuously receiving feedback regarding traffic conditions from end users and process it on the fly; and (ii) providing real-time and accurate travel time estimates to users without relying on any other type of sensors to receive data apart from the data reported by the users. In order to achieve these goals, the back-end system needs to be extended to fulfill the following requirements:
- R1:
Provide real-time information about traffic conditions. This boils down to be capable of (i) providing speed and travel time conditions per road segment for the last few minutes and (ii) reporting incidents upon the receipt (and validation) of such feedback.
- R2:
Provide estimates for future traffic conditions. This is important in order to provide accurate estimates regarding predicted travel times, which typically refer to the next couple of hours and are computed using both current and historical data.
- R3:
Manage historical information to train the prediction models needed by R2. This entails the need to store past information at several levels of granularity.
- R4:
Scalability. Traffic reporting and forecasting can be inherently parallelised in a geo-distributed manner, i.e., each region to be served by a separate cluster of servers. A country as large as France is mapped using no more than 100M road segments. Therefore, the challenge is not that much in the volume of data to be processed at runtime but in the velocity of new update streams and the need to store historical data; i.e., the data volume becomes an issue when there is a need to store historical data for sophisticated prediction model building and further analysis, as in our case.
- R5:
Fault-tolerance. Any modules to be included in the back-end need to be capable of tolerating failures, so that the system availability is high and thus the solution can be used in practice.
There are several other tools that provide this type of information; e.g., Google Maps, Waze4 and TomTom.5 However none of these tools that are being developed by big companies have published information about their back-end processing engine. By contrast, we explain architectural details, while we also employ publicly available open-source tools, so that third parties can rebuild our solution with reasonable effort.
This work makes the following contributions in relation to the requirements and the setting already described:
- 1.
It presents an end-to-end crowdsensing-based solution for supporting real-time traffic reporting and forecasting as far as the communication subsystem and the back-end system engine is concerned. The architecture consists of several modules and integrates different tools and data stores. The different tools need to account for both streaming and batch processing, while storage needs to support diverse technical requirements, such as meeting real-time constraints and supporting analytical queries over historical data while keeping the volume of the underlying data cubes limited.
- 2.
It discusses several alternatives regarding design choices in a manner that lessons can be transferred to other similar settings. For example, our work explains why using a main-memory database along with pushing the results of online analysis to persistent storage is important for real-time requirements and compares state-of-the-art scalable OLAP solutions.
- 3.
It includes indicative performance results that provide strong insights into the performance of each individual module in the architecture so that design choices can be evaluated, the efficiency in which requirements are met can be assessed, and bottlenecks can be identified.
- 4.
It presents a pilot case study, where the system has been successfully deployed on 800 taxis in the region of the city of Thessaloniki, Greece, for a period of three months.6
From a system's point of view, the novelty of our work lies in (i) presenting a non-intuitive non-monolithic architecture encompassing three different data store types and two different stream processing platforms with complementary roles in order to meet the defined requirements; (ii) to the best of our knowledge, it is the first work that compares the two specific main alternatives regarding back-end analytics databases examined, namely Apache Kylin and Apache Druid; and (iii) the results presented are meaningful for software architects in different domains with similar requirements. Our architecture, which is termed as Sboing4Real, goes beyond the reference lambda architecture [16] and the main distinctive feature is that the stream and the batch processing components form an asynchronous pipeline and benefit from different data store types.
The remainder of this work is structured as follows. The next section gives the background on real-time traffic analysis. Section 3 presents the overall architecture. In the next section, we present the messaging component connecting the monitoring devices with the back-end system. Section 5 discusses the pre-processing of source data. In Section 6, we describe the underlying data warehouse and the queries that run over it. Indicative experiments showing the scalability of the system and its capability to support the aforementioned requirements are in Section 7. The pilot case study is presented in Section 8. We conclude in Section 9.
Section snippets
Background on real-time traffic reporting and forecasting
Real-time traffic reporting is an important part of navigation systems, which, nowadays, typically also provide a travel time prediction functionality; e.g., Google maps navigation annotates each proposed route with metadata about the length and the expected travel time. Effective forecasting of traffic can lead to accurate travel time prediction. Due to its practical applications, short-term traffic forecasting is a hot research field with many research works being published. Vlahogianni et
System architecture
This section provides the high-level architecture and consists of two parts. The first part discusses the monitoring mechanisms and the communication between data sources and the back-end system. This part is further elaborated in Section 4. The second part provides the overview of the back-end pipeline that is explained in detail in Sections 5 and 6.
The messaging component and security considerations
For mobile networks and IoT systems, a common characteristic that affects design decisions is the inherent trade-off between performance and security, when a system scales to higher-load usage scenarios. In these cases, we expect the system to retain the same quality of service in terms of performance, while having to face increased computation/communication load due to the cryptographic protocols and authorization requirements that assure adequate security. Moreover, we also expect the system
The stream processing component
In this section, we present in detail the first part of the pipeline that includes the stream processing module, materialized on top of (i) Flink, through which the raw data pass to be further processed, (ii) Kafka, which is the stream controller module that transfers the processed data to the storage, and (ii) Redis, which is a temporary main-memory database. We also present a fourth module that gathers weather data to complement the data reported by user devices.
Data storage and querying
Here, we describe the OLAP solutions and the type of queries over such solutions and Redis to support R1, R2, and R3 in a scalable manner.
Performance evaluation
In this section, we provide concrete experimental results regarding the Sboing4Real architecture's efficiency and capability to handle large data volumes. We first focus on the stream processing component and we measure the resource consumption of Flink by simulating an intense input stream as well as the time it takes to write the intermediate data referring to the last 5-minute window to Redis. We continue by experimenting with both data warehouses using different settings and comparing their
Pilot case study
To test the solution, we deployed our clients on 800 taxis in the period of March-May 2020 in the area of Thessaloniki, Greece's second largest city. Unfortunately, this period largely overlapped with the lockdown due to COVID-19, and most of the taxis resumed their operation near beginning of May when the COVID-19 lockdown measures were first relaxed in Greece, as shown in Fig. 8. In this figure, the messages are sent to two MQTT brokers and the solid black line shows the total devices
Lessons learned, additional issues and conclusions
This work presented the back-end system in the context of the Sboing navigator to support real-time traffic reporting and forecasting. We presented a modular architecture and explained in detail all the main design choices so that our solution can be easily re-engineered by third parties. The main lessons learned can be summarized as follows: (1) To support our requirements, we need a scalable messaging component and two big-data processing platform instantiations, one for streaming data and
CRediT authorship contribution statement
Theodoros Toliopoulos: Methodology, Software, Validation, Writing – original draft. Nikodimos Nikolaidis: Methodology, Software. Anna-Valentini Michailidou: Methodology, Software. Andreas Seitaridis: Methodology, Software. Theodoros Nestoridis: Methodology, Software. Chrysa Oikonomou: Methodology, Software. Anastasios Temperekidis: Methodology, Software. Fotios Gioulekas: Methodology. Anastasios Gounaris: Conceptualization, Methodology, Supervision, Writing – original draft, Writing – review &
Declaration of Competing Interest
We declare no CoI apart from colleagues with a) Aristotle University of Thessaloniki and b) International HEllenic University.
Acknowledgements
This research has been co-financed by the European Union and Greek national funds through the Operational Program Competitiveness, Entrepreneurship and Innovation, under the call RESEARCH - CREATE - INNOVATE (project code:T1EDK-01944).
Theodoros Toliopoulos is a PhD candidate in Aristotle University of Thessaloniki working in the field of big data management and more specifically with big data streams. He graduated from the Dept. of Informatics of Thessaloniki, Greece in 2013 (BSc) and 2018 (MSc). He has participated as a research assistant in projects such as “Time series analysis over user data” and “Knowledge and Predictive Modelling”. His main research interests are in the areas of outlier detection and continuous stream
References (27)
- et al.
A mobile crowd sensing ecosystem enabled by cupus: cloud-based publish/subscribe middleware for the Internet of things
Future Gener. Comput. Syst.
(2016) - et al.
A scalable and manageable iot architecture based on transparent computing
J. Parallel Distrib. Comput.
(2018) - et al.
Bi-objective traffic optimization in geo-distributed data flows
Big Data Res.
(2019) - et al.
Continuous outlier mining of streaming data in flink
Inf. Syst.
(2020) - et al.
Short-term traffic forecasting: where we are and where we're going
Transp. Res., Part C, Emerg. Technol.
(2014) - et al.
Hadoop-gis: a high performance spatial data warehousing system over mapreduce
PVLDB
(2013) - et al.
Overview of device access control in the iot and its challenges
IEEE Commun. Mag.
(2019) - et al.
Travel speed forecasting by means of continuous conditional random fields
Transp. Res. Rec.
(2011) - et al.
A survey on geographically distributed big-data processing using mapreduce
IEEE Trans. Big Data
(2019) - et al.
The impact of rainfall and temperature on peak and off-peak urban traffic
A survey on security, privacy, and trust in mobile crowdsourcing
IEEE Int. Things J.
Network-scale traffic modeling and forecasting with graphical lasso
Mobile crowd sensing and computing: when participatory sensing meets participatory social media
IEEE Commun. Mag.
Cited by (0)
Theodoros Toliopoulos is a PhD candidate in Aristotle University of Thessaloniki working in the field of big data management and more specifically with big data streams. He graduated from the Dept. of Informatics of Thessaloniki, Greece in 2013 (BSc) and 2018 (MSc). He has participated as a research assistant in projects such as “Time series analysis over user data” and “Knowledge and Predictive Modelling”. His main research interests are in the areas of outlier detection and continuous stream analysis.
Nikodimos Nikolaidis is a graduate of computer science department of Aristotle university. He currently works in R&D projects in predictive maintenance domain. Other areas of interest include adaptive query processing, distributed computing, system/software engineering.
Anna-Valentini Michailidou is a PhD student in the Department of Informatics at Aristotle University of Thessaloniki, Greece. Her research interests include distributed data analytics, dataflow and workflow optimization, data-quality and Edge Computing. She received her BSc degree in Informatics from the Aristotle University of Thessaloniki in 2018. More information can be found at http://annavalen.webpages.auth.gr/.
Andreas Seitaridis is an indepented Research Software Engineer. He received his MSc on Knowlegde, Data and Software Technologies from the Computer Science Department at the Aristotle University of Thessaloniki (Greece) in 2018 and graduated from the same department in 2015. He has worked as a teaching assistant as well on a research project at the Computer Science Dep. of Aristotle University of Thessaloniki. His research interests include linear optimization techniques on problems concerning electric vehicles charging and he has two publications on this subject.
Theodoros Nestoridis has started his PhD at the Dept. of Informatics of the Aristotle University of Thessaloniki, Greece, in the field of software engineering. Theodoros Nestoridis has successfully completed his undergraduate and postgraduate studies in the field of informatics at the Dept. of Informatics of the Aristotle University of Thessaloniki, Greece. His interests are in the fields of software and network systems engineering.
Chrysa Oikonomou is a Cyber Security Consultant at Deloitte, Thessalonki, Greece. C. Oikonomou received her Msc from the Aristotle University of Thessaloniki (Greece) in 2020. She graduated from the Dept. of Electrical and Computer Engineering of University of Thessaly in Volos, Greece in 2018. Her main points of interests are in the areas of Information and Network Security, Intrusion Detection Systems and Machine Learning.
Anastasios Temperekidis has started his PhD at the Dept. of Informatics of Aristotle University of Thessaloniki, Greece, in the field of software engineering. Mr. Temperekidis has successfully completed his undergraduate studies at the Dept. of Informatics of the Aristotle University of Thessaloniki, Greece. His interests are in the fields of formal verification, programming languages and autonomous
Fotios Gioulekas graduated from the Department of Electrical and Computer Engineering, University of Patras, Greece in 2000, and received his Ph.D. Degree in Electrical and Computer Engineering in 2007 from the same University. He also obtained his Post-Doctoral Research Certificate from the Department of Informatics at the Aristotle University of Thessaloniki, Greece in 2020. He is an ICT Engineer at the University General Hospital of Larissa at the 5 Regional Health Authority of Thessaly and Sterea, Greece. He has published more than 30 papers in international research journals, conferences and books and is the principal inventor in 3 international patents. He is a member of the Technical Chamber of Greece and the ENISA (European Union Agency for Cybersecurity) eHealth External Security Expert Group.
Anastasios Gounaris is an associate professor at the Dept. of Informatics of the Aristotle University of Thessaloniki, Greece. A. Gounaris received his PhD from the University of Manchester (UK) in 2005. He also graduated from the Dept. of Electrical and Computer Engineering of Thessaloniki, Greece in 1999 (BSc), and from the Dept. of Computation, UMIST, Manchester, UK, in 2002 (MPhil). His main research interests are in the areas of autonomic, adaptive and wide-area data management, massive parallelism, flow and query optimization, big data analytics, data mining and resource scheduling. More details can be found at http://datalab.csd.auth.gr/~gounaris/.
Nick Bassiliades (http://tinyurl.com/bassiliades) received his MSc in Applied Artificial Intelligence from the Computing Science Department of Aberdeen University, in 1992, and his PhD degree in parallel knowledge base systems from the School of Informatics at the Aristotle University of Thessaloniki (AUTh), Greece, in 1998, where he is currently a Professor. His research interests include knowledge-based and rule systems, multiagent systems, ontologies, linked data and the Semantic Web. He has published more than 200 papers at journals, conferences, and books, has coauthored 5 books and co-edited 10 volumes. His published research has received over 4000 citations (h-index 31), while 6 of his papers have received awards. He was the Program Chair of 11 conferences/workshops, he has been member of the Program Committee of more than 140 and on the Organizational Committee of 9 conferences/workshops. He has been involved in 38 R&D projects leading 11 of them. He has been the general secretary of the Board of the Greek Artificial Intelligence Society; he is a director of RuleML, Inc., and also a member of the Greek Computer Society, the IEEE, and the ACM. He is currently serving as the president of the IT Center of AUTh and as the director of the Web, Data and Knowledge Engineering Sector of the School of Informatics.
Panagiotis Katsaros is Associate Professor at the School of Informatics of the Aristotle University of Thessaloniki (AUTh), Greece. He holds a Bachelor in Mathematics from AUTh, a Master of Science on Software Engineering from Aston University in Birmingham and a PhD in Computer Science from AUTh. Prof. Katsaros has published over 100 research papers in international journals and conference proceedings on Software Engineering. His specific interests, including those for software engineering for the Internet of Things, refer to the formal verification of software/systems, the model-based design, the analysis of dependability and security and the simulation-based performance analysis and optimization. Prof. Katsaros is coordinator (or participates in) national and European R&D projects focusing on engineering of software for Internet of Things systems, space systems and more recently autonomous systems. Prof. Katsaros was the General Chair of the European Joint Conferences on Theory and Practice of Software (ETAPS 2018) that was organized in Thessaloniki in 2018. Regular updates on Prof. Katsaros recent research achievements can be accessed online at: http://depend.csd.auth.gr.
Apostolos Georgiadis is a Senior Telecommunications & Software Engineer at SBOING. He received his PhD from the University of Edinburgh (UK) in 2001 and graduated from the Dept. of Computer Engineering & Informatics at the University of Patras (Greece) in 1995. He has worked as a Research Fellow with the Signals and Systems Group, University of Edinburgh (UK) and as a part-time Lecturer at the Dept of Telecommunications & Informatics, International Hellenic University (Greece). His expertise and research interests include non-linear estimation & optimization in telecommunication channels, signal processing, software-defined radio, cryptography, and software-as-a-service.
Fotis K. Liotopoulos (SM IEEE, 2002) is a Computer Engineer with M.Sc. and Ph.D. degrees from the Univ. of Wisconsin-Madison, USA. He is an academic teacher & researcher with 50+ published research papers in the areas of computer & network architectures, parallel computing, performance analysis & optimization. He is the founder & CEO/CTO of SBOING.net, an innovative ICT SME in Greece, developing transport applications and systems, and he has managed 100+ ICT RTD projects. Extended bio: https://sboing.net/people/liotop