Abstract
We propose to tackle the optical flow problem by a combination of two recent advances in the computation of dense correspondences, namely the incorporation of image segmentation and robust global optimization via graph-cuts. In the first step, each segment (extracted by colour segmentation) is assigned to an affine motion model from a set of sparse correspondences. Using a layered model, we then identify those motion models that represent the dominant image motion. This layer extraction task is accomplished by optimizing a simple energy function that operates in the domain of segments via graph-cuts. We then estimate the spatial extent that is covered by each layer and identify occlusions. Since treatment of occlusions is hardly possible when using entire segments as matching primitives, we propose to use the pixel level in addition. We therefore define an energy function that measures the quality of an assignment of segments and pixels to layers. This energy function is then extended to work on multiple input frames and minimized via graph-cuts. In the experimental results, we show that our method produces good-quality results, especially in regions of low texture and close to motion boundaries, which are challenging tasks in optical flow computation.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Horn, B., Schunck, B.: Determining optical flow. Artificial Intelligence 17, 185–203 (1981)
Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo. In: IJCAI, pp. 121–130 (1981)
Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)
Zitnick, C., Kanade, T.: A cooperative algorithm for stereo matching and occlusion detection. TPAMI 22(7), 675–684 (2000)
Black, M., Jepson, A.: Estimating optical flow in segmented images using variable-order parametric models with local deformations. TPAMI 18(10), 972–986 (1996)
Tao, H., Sawhney, H., Kumar, R.: A global matching framework for stereo computation. In: ICCV, pp. 532–539 (2001)
Hong, L., Chen, G.: Segment-based stereo matching using graph cuts. In: CVPR, vol. 1, pp. 74–81 (2004)
Ke, Q., Kanade, T.: A subspace approach to layer extraction. In: CVPR, pp. 255–262 (2001)
Altunbasak, Y., Eren, P., Tekalp, A.: Region-based parametric motion segmentation using color information. GMIP 60(1), 13–23 (1998)
Ayer, S., Sawhney, H.: Layered representation of motion video using robust maximum-likelihood estimation of mixture models and mdl encoding. In: ICCV, pp. 777–784 (1995)
Willis, J., Agarwal, S., Belongie, S.: What went where. In: CVPR, pp. 37–44 (2003)
Xiao, J., Shah, M.: Motion layer extraction in the presence of occlusion using graph cuts. TPAMI 27(10), 1644–1659 (2005)
Christoudias, C., Georgescu, B., Meer, P.: Synergism in low-level vision. In: ICPR, vol. 4, pp. 150–155 (2002)
Shi, J., Tomasi, C.: Good features to track. In: CVPR, pp. 593–600 (1994)
Wang, J., Adelson, E.: Representing moving images with layers. Transactions on Image Processing 3(5), 625–638 (1994)
Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. TPAMI 23(11), 1222–1239 (2001)
Bleyer, M.: Segmentation-based Stereo and Motion with Occlusions. PhD thesis, Vienna University of Technology (2006)
Xiao, J., Shah, M.: Accurate motion layer segmentation and matting. In: CVPR, vol. 2, pp. 698–703 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bleyer, M., Rhemann, C., Gelautz, M. (2006). Segmentation-Based Motion with Occlusions Using Graph-Cut Optimization. In: Franke, K., Müller, KR., Nickolay, B., Schäfer, R. (eds) Pattern Recognition. DAGM 2006. Lecture Notes in Computer Science, vol 4174. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11861898_47
Download citation
DOI: https://doi.org/10.1007/11861898_47
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44412-1
Online ISBN: 978-3-540-44414-5
eBook Packages: Computer ScienceComputer Science (R0)