Mobile Surveillance by 3D-Outlier Analysis

Abstract

We present a novel online method to model independent foreground motion by using solely traditional structure and motion (S+M) algorithms. On the one hand, the visible static scene can be reconstructed and on the other hand, the position and orientation (pose) of the observer (mobile camera) are estimated. Additionally, we use 3D-outlier analysis for foreground motion detection and tracking. First, we cluster the available 3D-information such that, with high probability, each cluster corresponds to a moving object. Next, we establish a purely geometry-based object representation that can be used to reliably estimate each object’s pose. Finally, we extend the purely geometry-based object representation and add local descriptors to solve the loop closing problem for the underlying S+M algorithm. Experimental results on single and multi-object video data demonstrate the viability of this method. Major results include the computation of a stable representation of moving foreground objects, basic recognition possibilities due to descriptors, and motion trajectories that can be used for motion analysis of objects. Our novel multibody structure and motion (MSaM) approach runs online and can be used to control active surveillance systems in terms of dynamic scenes, observer pose, and observer-to-object pose estimation, or to enrich available information in existing appearance- and shape-based object categorization.