An In-Database Streaming Solution to Multi-camera Fusion

Abstract

Multi-camera based video object tracking is a multi-stream data fusion and analysis problem. With the current technology, video analysis software architecture generally separates the analytics layer from the data management layer, which has become the performance bottleneck because of large scaled data transfer, inefficient data access and duplicate data buffering and management. Motivated by providing a convergent platform, we use user-defined Relation Valued Functions (RVFs) to have visual data computation naturally integrated to SQL queries, and pushed down to the database engine; we model complex applications with general graph based data-flows and control-flows at the process level where “actions” are performed by RVFs and “linked” in SQL queries. We further introduce Stream Query Process with stream data input and continuous execution. Our solutions to multi-camera video surveillance also include a new tracking method that is based on P2P time-synchronization of video streams and P2P target fusion.

These techniques represent a major shift in process management from one-time execution to data stream driven, open-ended execution, and constitute a novel step to the use of a query engine for running processes, towards the “In-DB Streaming” paradigm.

We have prototyped the proposed approaches by extending the open-sourced database engine Postgres, and plan to transfer the implementation to a commercial and proprietary parallel database system. The empirical study in a surveillance setting reveals their advantages in scalability, real-time performance and simplicity.