Abstract
In this paper, a novel video matting method based on opacity propagation is proposed. It adopts a two-step framework by first segmenting each frame into bilayer and generating trimaps, and then applying matting algorithm to extract the final alpha mattes. In the bilayer segmentation step, an opacity propagation algorithm is used to predict the foreground object in the following frame. It propagates the opacity values of the former frame to the current, and generates an Opacity Map (OM), which is in most cases much more accurate than the previously used Probability Map (PM). OM is used to extract the foreground object by combining it with the graph cut algorithm. And then an accurate trimap is generated by a trimap refinement method based on a Local Gaussian Mixture Model (LGMM). At last, the opacity propagation is again applied to the unknown regions of the trimap to generate the accurate alpha mattes in a temporally coherent way. The experiment results demonstrate the effectiveness of our method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ahn, J.K., Kim, C.S.: Real-time segmentation of objects from video sequences with non-stationary backgrounds using spatio-temporal coherence. In: ICIP, pp. 1544–1547 (2008)
Bai, X., Sapiro, G.: Geodesic matting: A framework for fast interactive image and video segmentation and matting. Int. J. Comput. Vis. 82(2), 113–132 (2009)
Bai, X., Wang, J., Simons, D., Sapiro, G.: Video snapcut: Robust video object cutout using localized classifiers. In: Proc. of ACM Trans. Graphics (2009)
Chuang, Y.Y., Agarwala, A., Curless, B., Salesin, D.H., Szeliski, R.: Video matting of complex scenes. ACM Trans. Graph. 21, 243–248 (2002)
Corrigan, D., Robinson, S., Kokaram, A.: Video matting using motion extended grabcut. In: IET European Conference on Visual Media Production (2008)
Grady, L., Schiwietz, T., Aharon, S., Westermann, R.: Random walks for interactive alpha-matting. In: Proc. of Fifth IASTED Int’l Conf. Visualization, Imaging, and Image Processing (2005)
Gu, Y., Jin, C., Xue, X.: Quick matting: A matting method based on pixel spread and propagation. In: Proc. IEEE ICIP, Hong Kong, China, Sep., pp. 4661–4664 (2010)
He, K., Sun, J., Tang, X.: Fast matting using large kernel matting laplacian matrices. In: Proc. IEEE CVPR, San Francisco, USA, pp. 2165–2172 (2010)
Jain, A., Agrawal, M., Gupta, A., Khandelwal, V.: A novel approach to video matting using automated scribbling by motion analysis. In: Proc. of IEEE VECIMS, pp. 25–30, July (2008)
Juan, O., Keriven, R.: Trimap segmentation for fast and user-friendly alpha matting. In: IEEE Workshop on Variational, Geometric & Level Set Methods, Beijing, China, Oct., pp. 186–197 (2005)
Lee, P.G., Wu, Y.: L1 matting. In: Proc. IEEE ICIP, Hong Kong, China, Sep., pp. 4665–4668 (2010)
Lee, S.Y., Yoon, J.C., Lee, I.K.: Temporally coherent video matting. Graph. Models 72(3), 25–33 (2010)
Levin, A., Lischinski, D., Weiss, Y.: A closed form solution to natural image matting. IEEE Trans. Pattern. Anal. Mach. Intell. 30 (2008)
Li, Y., Sun, J., Shum, H.Y.: Video object cut and paste. In: Proc. of ACM Trans. Graphics, pp. 595–600 (2005)
Pham, V.Q., Takahashi, K., Naemura, T.: Real-time video matting based on bilayer segmentation. In: ACCV, Sep (2009)
Price, B.L., Morse, B.S., Cohen, S.: Simultaneous foreground background and alpha estimation for image matting. In: Proc. IEEE CVPR, San Francisco, USA, pp. 2157–2164 (2010)
Rhemann, C., Rother, C., Rav-Acha, A., Sharp, T.: High resolution matting via interactive trimap segmentation. In: Proc. IEEE CVPR, Anchorage, USA, pp. 1–8 (2008)
Rhemann, C., Rother, C., Kohli, P., Gelautz, M.: A spatially varying psf-based prior for alpha matting. In: Proc. IEEE CVPR, San Francisco, USA, pp. 2149–2156 (2010)
Rother, C., Kolmogorov, V., Blake, A.: Grabcut-interactive foreground extraction using iterated graph cut. ACM Trans. Graph. 309–314 (2004)
Shum, H., Sun, J., Yamazaki, S., Li, Y., Tang, C.: Pop-up light field: An interactive image-based modeling and rendering system. ACM Trans. Graph. 23, 143–162 (2004)
Sindeyev, M., Konushin, V.: A novel interactive image matting framework. In: Proc. GraphiCon, Moscow, Russia, June, pp. 41–45 (2008)
Sindeyev, M., Konushin, V., Vezhnevets, V.: Improvements of Bayesian matting. In: Proc. of Graphicon, pp. 88–95, June (2007)
Sun, J., Jia, J., Tang, C.K., Shum, H.Y.: Poisson matting. ACM Trans. Graph. 23, 315–321 (2004)
Sun, Y.D., Yuan, B.Z., Miao, Z.J., Wan, C.K.: Better foreground segmentation for static cameras via new energy form and dynamic graph-cut. In: ICPR, pp. 49–52 (2006)
Tang, Z., Miao, Z.J., Wan, Y.L.: Temporaly consistent video matting based on bilayer segmentation. In: ICME (2010)
Wang, J., Bhat, P.: Interactive video cutout. In: Proc. of ACM Trans. Graphics, pp. 585–594 (2005)
Wang, J., Cohen, M.: An iterative optimization approach for unified image segmentation and matting. In: ICCV, pp. 936–943 (2005)
Wang, J., Cohen, M.: Image and video matting: a survey. Found. Trends Comput. Graph. Vis. 3(2), 97–175 (2007)
Wang, J., Cohen, M.: Optimized color sampling for robust matting. In: CVPR, pp. 1–8 (2007)
Wang, J., Agrawala, M., Cohen, M.: Soft scissors: An interactive tool for real time high quality matting. ACM Trans. Graph. 26, 9 (2007)
Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-L1 optical flow. Pattern Recognit. 214–223 (2007)
Author information
Authors and Affiliations
Corresponding author
Electronic Supplementary Material
Below is the link to the electronic supplementary material.
(WMV 3.41 MB)
Rights and permissions
About this article
Cite this article
Tang, Z., Miao, Z., Wan, Y. et al. Video matting via opacity propagation. Vis Comput 28, 47–61 (2012). https://doi.org/10.1007/s00371-011-0598-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-011-0598-3