CO-STAR: A collaborative prediction service for short-term trends on continuous spatio-temporal data

Highlights

• Online stream and offline batch processing work collaboratively in the service.

• Business interpretable prediction model can hold minute-level executive latency with nearly 10% precision improvement.

• The service has been used in practice and evaluated extensively.

Abstract

Over various sensory data of Internet of Things, not only the current situation but also the future trends of many fields are required instantly to promote the business. As a typical requirement, the short-term prediction on spatio-temporal data stream is imperative, but challenges still remain due to the inherent limitation of long calculative time and insufficient predictive precision. In this paper, a novel prediction service CO-STAR is proposed in the highway domain. On the continuous toll data of the whole highway network, the service employs non-parametric regression model to predict the traffic volume of all the stations periodically. Considering both spatial and temporal business characteristics, a collaborative paradigm of online stream computing and offline batch processing is adopted to balance the efficiency and precision. On the real data of one Chinese provincial highway and the simulated data, our service can hold minute-level executive latency with nearly 10 percent improvement for the predictive precision in extensive experiments.

Introduction

In modern cities, abundant sensors in different types are adopted nowadays, such as surveillance cameras on the trunk roads, smart-card readers on the buses, loop detectors at the toll stations, and transducer in the power plants. The spatio-temporal data is generated from those sensors in a timely manner, which includes the attributes of spaces and times [1]. Such data as typical stream in IoT (Internet of Things) environment owns unique characteristics. Firstly, the spatio-temporal data is always generated fast, continuously and concurrently. For example, in a metropolis like Beijing, more than 10 thousand cameras have been deployed at the road intersections, each of which would produce one record in a second [2]. It yields data stream in high throughput. Secondly, the spatio-temporal data has to be processed in a low latency. For example, to instantly locate the illegal vehicles in the road network, any analytical application of traffic polices on the data from surveillance cameras requires the calculation to respond in no longer than three seconds [3]. Thirdly, the correlation among spatial and temporal attributes often implies complexity and dynamics behind the superficial facts. As a result, the uncertainty must exist in short-term trends prediction. For example, on GPS data of OBU (on-board unit) device, the travel time of road segment follows different distribution in a day, and varies much among different road segments [4].

Over such spatio-temporal data stream, not only the current situation but also the future trends are required instantly [5] for business management. Therefore, the short-term prediction service has become the necessary and essential in many domains. For bus schedule planning, the predictive delivery time between two adjacent stops in coming hours is the crucial; for the traffic guidance, the predictive traffic volume of road segments in the next 5–10 min are imperative; to avoid the energy blackouts, the predictive peaks of power load in urban functional areas on summer days are vital. Accordingly, such prediction for short-term trends within 30 minutes is sensitive to factors of both time and space.

However, challenges still remain for the short-term trends prediction on continuous spatio-temporal data due to inherent limitations. (1) It is impossible to hold the low calculative latency and higher predictive precision simultaneously on typical data stream. There is a contradiction between the efficiency and the accuracy: much computational resource and time are required for the accurate the results, and vice versa. Therefore, it is not trivial to balance the consequential precision and the run-time performance in the data stream conditions. (2) Traditional modeling methods cannot comprehensively cover the distinctive characteristics of spatio-temporal data, which restricts their feasibility and interpretability in practice. For example, time restricted models applied in many scenes [6], [7], [8] only emphasize the temporal feature, and limit the predictive effects. Moreover, the classic statistical models only work well on limited samples at a given location, which are not suitable in current Big Data condition. Some modern models like neural network can preferably fit the trends, but still lack the business interpretability. Accordingly, it is not easy to find reasonable feature models for trend prediction, considering not only its functional capacity but also the business specialty. (3) It is not convenient to do the prediction on spatio-temporal data in hybrid computing paradigm due to the absence of general methodology. For the short-term trends, the real urban sensory data is essentially continuous stream, but most of classic solutions are batch computing paradigm like [9], [10], [11], [12]. The gap lying between the online data and offline processing makes it hard to guarantee the performance. The collaborative service to integrate offline and online paradigms for short-term trends prediction is seldom considered yet on spatio-temporal data.

Against the problems above, a novel prediction service is proposed in this paper, and it is illustrated in the highway domain. The contributions of our work can be summarized as follows. (1) To balance the calculative performance on continuous spatio-temporal data, the service architecture mixes online stream and offline batch computing paradigms, which can hold the processing latency in minute-level. The service can organize the continuous input data in millisecond, and then predict the traffic volume trends in the next time slot of all the stations with less than 80 s. (2) To improve the predictive precision on spatio-temporal data stream, a non-parametric regression model is designed for short-term trends prediction. Considering the interpretable business characteristics of time and space, it can reduce the predictive error about nearly 10 percentages than traditional ways. (3) Having been evaluated in a practical project, the service has the convincing benefits for the business technicians on the real and simulated data in extensive experiments.

The rest of this paper is organized as follows: Section 2 shows the motivation and related works; Section 3 presents the preliminaries including methodology and data pre-processing; Section 4 elaborates the short-term prediction service with its model and algorithm; Section 5 quantitatively demonstrates performance and effects in the experiments; Section 6 summarizes the conclusion.