Abstract
We describe a computationally efficient scheme to perform model selection while simultaneously segmenting a short video stream into an unknown number of detachable objects. Detachable objects are regions of space bounded by surfaces that are surrounded by the medium other than for their region of support, and the region of support changes over time. These include humans walking, vehicles moving, etc. We exploit recent work on occlusion detection to bootstrap an energy minimization approach that is solved with linear programming. The energy integrates both appearance and motion statistics, and can be used to seed layer segmentation approaches that integrate temporal information on long timescales.
Keywords
Research supported by ARO 56765, ONR N000140810414, AFOSR FA95500910427.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Gibson, J.J.: The ecological approach to visual perception. LEA (1984)
Wang, J., Adelson, E.: Representing moving images with layers. IEEE Transactions on Image Processing 3, 625–638 (1994)
Jackson, J.D., Yezzi, A.J., Soatto, S.: Dynamic shape and appearance modeling via moving and deforming layers. In: Rangarajan, A., Vemuri, B.C., Yuille, A.L. (eds.) EMMCVPR 2005. LNCS, vol. 3757, pp. 427–438. Springer, Heidelberg (2005)
Jackson, J., Yezzi, A.J., Soatto, S.: Dynamic shape and appearance modeling via moving and deforming layers. Intl. J. of Comp. Vision 79(1), 71–84 (2008)
Ayvaci, A., Raptis, M., Soatto, S.: Occlusion detection and motion estimation with convex optimization. In: Advances in Neural Information Processing Systems (2010)
Cremers, D., Soatto, S.: Motion competition: a variational approach to piecewise parametric motion segmentation. International Journal of Computer Vision 62, 249–265 (2005)
Huang, Y., Liu, Q., Metaxas, D.: Video object segmentation by hypergraph cut. In: Proc. of the Conference on Computer Vision and Pattern Recognition, pp. 1738–1745 (2009)
Bai, X., Wang, J., Simons, D., Sapiro, G.: Video SnapCut: robust video object cutout using localized classifiers. In: ACM SIGGRAPH (2009)
Unger, M., Mauthner, T., Pock, T., Bischof, H.: Tracking as segmentation of spatial-temporal volumes by anisotropic weighted TV. In: Proc of the Energy Minimization Methods in Computer Vision and Pattern Recognition (2009)
Ince, S., Konrad, J.: Occlusion-aware optical flow estimation. IEEE Transactions on Image Processing 17, 1443–1451 (2008)
Ayvaci, A., Soatto, S.: Detachable object detection. Technical Report CSD100036, UCLA Computer Science Department (November 19, 2010)
Wang, J., Xu, Y., Shum, H., Cohen, M.: Video tooning. In: ACM SIGGRAPH (2004)
Grady, L.: Random walks for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 28, 1768–1783 (2006)
Grunwald, P., Rissanen, J.: The Minimum Description Length Principle. The MIT Press, Cambridge (2007)
Dahleh, M.A., Diaz-Bobillo, I.J.: Control of uncertain systems: a linear programming approach. Prentice-Hall, Englewood Cliffs (1994)
Leclerc, Y.: Constructing simple stable descriptions for image partitioning. International Journal of Computer Vision 3, 73–102 (1989)
Delong, A., Osokin, A., Isack, H., Boykov, Y.: Fast approximate energy minimization with label costs. In: Proc. of the Conference on Computer Vision and Pattern Recognition (2010)
Lim, Y., Jung, K., Kohli, P.: Energy minimization under constraints on label counts. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 535–551. Springer, Heidelberg (2010)
Yuan, J., Boykov, Y.: Tv-based image segmentation with label cost prior. In: Proc. of the Britih Machine Vision Conference (2010)
Schoenemann, T., Cremers, D.: High resolution motion layer decomposition using dual-space graph cuts. In: Proc. of the Conference on Computer Vision and Pattern Recognition (2008)
Sun, D., Sudderth, E., Black, M.: Layered Image Motion with Explicit Occlusions, Temporal Consistency, and Depth Ordering. In: Advances in Neural Information Processing Systems (2010)
Brox, T., Malik, J.: Object segmentation by long term analysis of point trajectories. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6315, pp. 282–295. Springer, Heidelberg (2010)
Pawan Kumar, M., Torr, P., Zisserman, A.: Learning layered motion segmentations of video. International Journal of Computer Vision 76, 301–319 (2008)
Irani, M., Peleg, S.: Motion analysis for image enhancement: Resolution, occlusion, and transparency. Journal of Visual Communication and Image Representation 4, 324–324 (1993)
Jepson, A.D., Fleet, D.J., Black, M.J.: A layered motion representation with occlusion and compact spatial support. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 692–706. Springer, Heidelberg (2002)
Ogale, A., Ferm, C., Aloimonos, Y.: Motion segmentation using occlusions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 988–992 (2005)
Stein, A., Stepleton, T., Hebert, M.: Towards unsupervised whole-object segmentation: Combining automated matting with boundary detection. In: Proc. of the Conference on Computer Vision and Pattern Recognition (2008)
Apostoloff, N., Fitzgibbon, A.: Automatic video segmentation using spatiotemporal T-junctions. In: Proc. of the Britih Machine Vision Conference (2006)
Stein, A., Hebert, M.: Occlusion boundaries from motion: low-level detection and mid-level reasoning. International Journal of Computer Vision 82, 325–357 (2009)
He, X., Yuille, A.: Occlusion boundary detection using pseudo-depth. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 539–552. Springer, Heidelberg (2010)
Apostoloff, N., Fitzgibbon, A.: Learning Spatiotemporal T-Junctions for Occlusion Detection. In: Proc. of the Conference on Computer Vision and Pattern Recognition (2005)
Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 1222–1239 (2002)
Shi, J., Malik, J.: Normalized Cuts and Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 888–905 (2002)
Morel, J., Salembier, P.: Monocular Depth by Nonlinear Diffusion. In: Proc. of the Indian Conference on Computer Vision, Graphics & Image Processing (2008)
Amer, M., Raich, R., Todorovic, S.: Monocular Extraction of 2.1D Sketch. In: Proc. of the International Conference on Image Processing (2010)
Sinop, A.K., Grady, L.: A seeded image segmentation framework unifying graph cuts and random walker which yields a new algorithm. In: Proc. of the International Conference on Computer Vision (2007)
Grant, M., Boyd, S.: Cvx: Matlab software for disciplined convex programming, version 1.21 (2010), http://cvxr.com/cvx
Freedman, D., Zhang, T.: Interactive graph cut based segmentation with shape priors. In: Proc. of the Conference on Computer Vision and Pattern Recognition (2005)
Martin, D., Fowlkes, C., Malik, J.: Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans. on Pattern Analysis and Machine Intelligence 26, 530–549 (2004)
Arbeláez, P., Maire, M., Fowlkes, C., Malik, J.: From contours to regions: An empirical evaluation. In: Proc. of the Conference on Computer Vision and Pattern Recognition (2009)
Zelnik-Manor, L., Perona, P.: Self-tuning spectral clustering. In: Advances in Neural Information Processing Systems (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ayvaci, A., Soatto, S. (2011). Detachable Object Detection with Efficient Model Selection. In: Boykov, Y., Kahl, F., Lempitsky, V., Schmidt, F.R. (eds) Energy Minimization Methods in Computer Vision and Pattern Recognition. EMMCVPR 2011. Lecture Notes in Computer Science, vol 6819. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23094-3_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-23094-3_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23093-6
Online ISBN: 978-3-642-23094-3
eBook Packages: Computer ScienceComputer Science (R0)