Traffic situation visualization based on video composition

Highlights

• We visualize traffic statistic data by composing surveillance videos.

• This example-based visualization is flexible to all kinds of driving behaviors.

• We build the relations between traffic statistic data and surveillance videos.

• Vehicle flows and average speed in the composed video fit the demanded traffic statistic.

• A seamless transition and illumination smoothing between videos are presented to minimize visual artifacts.

Abstract

Vehicle detectors (VDs) are usually distributed in a road network to detect macroscopic traffic situations. These detectors provide global information such as vehicle flows, average speed, and road occupancy. Given that the collected statistic data are difficult for citizens to interpret, we visualize the data by providing users with realistic traffic videos. To achieve this aim, our system collects the surveillance videos and VD data that represent the traffic situation of a position. It then builds the connection between these two types of data. Considering the distribution of VDs is much denser than that of surveillance cameras, for those road segments with a VD but without a surveillance camera, one can utilize our system to synthesize videos for visually depicting the traffic situations over there. That is, we estimate vehicle flows from a video and apply the regression model to build the mapping between the flows and VD data. After that, given by a VD dataset, our system retrieves videos that match the VD data and seamlessly composes them to synthesize a traffic video. The evaluations and the experimental results demonstrate the feasibility of our system.

Introduction

Vehicle detectors are usually distributed in-road to acquire spatiotemporal traffic statistic. The detectors record the number and the average speed of vehicles that pass within a time span. These data are important for city planners and traffic controllers [1] because the data depict overview and details of traffic situations over time. However, interpreting traffic statistic demands expertise and is not suitable for general populations. For example, the same driving speed in a countryside and an urban city could have very different experience because of light and heavy traffic densities. Providing users with a number of vehicles that pass is also unintuitive because it depends on the number of lanes on the road. Moreover, a light traffic flow may indicate few vehicles on the road or a serious traffic jam, which easily induces misleading. Accordingly, providing an interface for general users to realize traffic situations is essential.

Since transmitting videos captured from road surveillance cameras consumes expensive load, simulation techniques are presented for traffic visualization. The methods estimate velocity and density fields over the road network, followed by applying an agent-based traffic simulator to create 3D animations. They enjoy the visualization from various viewpoints and even allow users to observe traffic from a driver׳s perspective. However, a simulator cannot always realize the real traffic flows because driving behaviors are often different in countries and regions (Fig. 1), not to mention other conditions such as weather, rush hours, and holidays. To overcome this problem, we present an example-based system that visualizes traffic situations by synthesizing road surveillance videos.

Our goal is to visualize statistic VD data using real world materials. Specifically, we find a road segment where VD and surveillance camera are both available and extract the relations between them. For the place with a VD but without cameras, we synthesize a streaming video by composing the video clips in our database to visualize its traffic situations over time. The main advantage of this framework is generality. While a simulation technique is insufficient to create an animation that satisfies all driving behaviors, our system does not have this problem because they are already provided by road surveillance videos. Note that this framework also consumes light data transmission load because example videos are collected in advance. Only statistic VD data are transmitted when the traffic flows of a road segment are visualized.

The problems of traffic visualization in our framework are video retrieval and seamless composition. Considering the traffic situation of a video is unclear, to obtain the information, we seed particles on the video and track their motions. The number and speed of these particles that go outside the video coordinate or gather at the vanish point are recorded. After that, we compute a regression model to map the VD data and the particle flows so as to retrieve proper videos for composition when a macroscopic traffic statistic of another place is given. To prevent artifacts of video transition, we overlap consecutive videos by a number of frames and compute a surface that passes through pixels with the least distortion. Specifically, pixels on this surface should have small color variations and zero motions to avoid discontinuity artifacts and suddenly appearing or disappearing vehicles. We also apply the Poisson blending to smooth the difference of illumination conditions in videos for achieving high visual quality.

Our method synthesizes a streaming video to visualize the traffic situations of a place over time. This example-based framework can realize the traffic flows consistent with the sparse VD data at different regions. The main advantage of this framework is generality, which is able to visualize different traffic situations derived from driving behaviors, weathers, etc. We show the experimental results in Fig. 6, Fig. 7, and in the accompanying video to demonstrate the feasibility of our technique.