Inlier Estimation for Moving Camera Motion Segmentation

Abstract

In moving camera videos, motion segmentation is often performed on the optical flow. However, there exist two challenges: (1) Camera motions lead to three primary flows in optical flow: translation, rotation, and radial flow. They are not all solved in existing frameworks under Cartesian coordinate system; (2) A moving camera introduces 3D motion, the depth discontinuities cause the motion discontinuities that severely confuse the motion segmentation. Meanwhile, the mixture of the camera motion and moving objects’ motions make indistinctness between foreground and background. In this work, our solution is to find a low order polynomial to model the background flow field due to its coherence. To this end, we first amend the Helmholts-Hodge Decomposition by adding coherence constraints, which can handle translation, rotation, and radial flow fields under a unified framework. Secondly, we introduce an Incoherence Map and a progressive Quad-Tree partition to reject moving objects and motion discontinuities. Finally, the low order polynomial is achieved from the rest flow samples on two potentials in HHD. We present results on more than twenty videos from four benchmarks. Extensive experiments demonstrate a better performance in dealing with challenging scenes with complex backgrounds. Our method improves the segmentation accuracy of state-of-the-art by $10\%\sim 30\%$.