Abstract
This work addresses the problem of video matting, that is extracting the opacity-layer of a foreground object from a video sequence. We introduce the notion of alpha-flow which corresponds to the flow in the opacity layer. The idea is derived from the process of rotoscoping, where a user-supplied object mask is smoothly interpolated between keyframes while preserving its correspondence with the underlying image. Our key contribution is an algorithm which infers both the opacity masks and the alpha-flow in an efficient and unified manner. We embed our algorithm in an interactive video matting system where the first and last frame of a sequence are given as keyframes, and additional user strokes may be provided in intermediate frames. We show high quality results on various challenging sequences, and give a detailed comparison to competing techniques.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Rav-Acha, A., Kohli, P., Rother, C., Fitzgibbon, A.: Unwrap mosaics: A new representation for video editing. In: SIGGRAPH, pp. 17:1–17:11 (2008)
Bai, X., Wang, J., Simons, D., Sapiro, G.: Video SnapCut: robust video object cutout using localized classifiers. In: SIGGRAPH, pp. 1–11 (2009)
Wang, J., Cohen, M.F.: Optimized color sampling for robust matting. Computer Vision and Pattern Recognition, 1–8 (2007)
Rother, C., Kolmogorov, V., Blake, A.: “GrabCut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23, 309–314 (2004)
He, K., Sun, J., Tang, X.: Guided Image Filtering. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 1–14. Springer, Heidelberg (2010)
Wang, J., Bhat, P., Colburn, R.A., Agrawala, M., Cohen, M.F.: Interactive video cutout. In: SIGGRAPH, pp. 585–594 (2005)
Li, Y., Sun, J., Shum, H.Y.: Video object cut and paste. In: SIGGRAPH, pp. 595–600 (2005)
Bai, X., Sapiro, G.: Geodesic matting: A framework for fast interactive image and video segmentation and matting. IJVC 82, 113–132 (2009)
Ding, Z., Chen, H., Guan, Y., Chen, W., Peng, Q.: GPU accelerated interactive space-time video matting. In: Computer Graphics International (2010)
Tang, Z., Miao, Z., Wan, Y., Zhang, D.: Video matting via opacity propagation. The Visual Computer, 1–15 (2011)
Lee, S.Y., Yoon, J.C., Lee, I.K.: Temporally coherent video matting. Graphical Models 72, 25–33 (2010)
Chuang, Y.Y., Agarwala, A., Curless, B., Salesin, D.H., Szeliski, R.: Video matting of complex scenes. SIGGRAPH 21, 243–248 (2002)
Apostoloff, N.E., Fitzgibbon, A.W.: Automatic video segmentation using spatiotemporal T-junctions. In: BMVC (2006)
Lezama, J., Alahari, K., Sivic, J., Laptev, I.: Track to the future: Spatio-temporal video segmentation with long-range motion cues. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2011)
Sand, P., Teller, S.J.: Particle video: Long-range motion estimation using point trajectories. International Journal of Computer Vision 80, 72–91 (2008)
Chen, J., Paris, S., Wang, J., Cohen, M., Cohen, M., Durand, F.: The video mesh: A data structure for image-based video editing. Artificial Intelligence (2009)
Agarwala, A., Hertzmann, A., Salesin, D.H., Seitz, S.M.: Keyframe-based tracking for rotoscoping and animation. In: SIGGRAPH, pp. 584–591 (2004)
Tsai, D., Flagg, M., Rehg, J.M.: Motion coherent tracking with multi-label MRF optimization. In: BMVC (2010)
Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: CVPR, pp. 2432–2439 (2010)
Levin, A., Lischinski, D., Weiss, Y.: A closed form solution to natural image matting. In: IEEE CVPR, vol. 1, pp. 61–68 (2006)
Rhemann, C., Rother, C., Kohli, P., Gelautz, M.: A Spatially Varying PSF-based Prior for Alpha Matting. In: IEEE Conference on Computer Vision and Pattern Recognition (2010)
Liu, C.: Beyond pixels: Exploring new representations and applications for motion analysis. Doctoral Thesis. Massachusetts Institute of Technology (2009)
Steinbrucker, F., Pock, T., Cremers, D.: Large displacement optical flow computation without warping. In: ICCV, pp. 1609–1614 (2009)
Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.B.: PatchMatch: A randomized correspondence algorithm for structural image editing. In: SIGGRAPH (2009)
Agrawal, A., Raskar, R., Chellappa, R.: What Is the Range of Surface Reconstructions from a Gradient Field? In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 578–591. Springer, Heidelberg (2006)
Xiao, J., Cheng, H., Sawhney, H., Rao, C., Isnardi, M.: Bilateral Filtering-Based Optical Flow Estimation with Occlusion Detection. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 211–224. Springer, Heidelberg (2006)
Strecha, C., Fransens, R., Van Gool, L.: A Probabilistic Approach to Large Displacement Optical Flow and Occlusion Detection. In: Comaniciu, D., Mester, R., Kanatani, K., Suter, D. (eds.) SMVP 2004. LNCS, vol. 3247, pp. 71–82. Springer, Heidelberg (2004)
Sindeev, M., Konushin, A., Rother, C.: Alpha-flow for video matting. Technical Report (2012)
Grundmann, M., Kwatra, V., Han, M., Essa, I.: Efficient hierarchical graph based video segmentation. In: IEEE CVPR (2010)
Bai, X., Wang, J., Simons, D.: Towards Temporally-Coherent Video Matting. In: Gagalowicz, A., Philips, W. (eds.) MIRAGE 2011. LNCS, vol. 6930, pp. 63–74. Springer, Heidelberg (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sindeev, M., Konushin, A., Rother, C. (2013). Alpha-Flow for Video Matting. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds) Computer Vision – ACCV 2012. ACCV 2012. Lecture Notes in Computer Science, vol 7726. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37431-9_34
Download citation
DOI: https://doi.org/10.1007/978-3-642-37431-9_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-37430-2
Online ISBN: 978-3-642-37431-9
eBook Packages: Computer ScienceComputer Science (R0)