skip to main content
10.1145/3009977.3009979acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicvgipConference Proceedingsconference-collections
research-article

Automatic video matting through scribble propagation

Published: 18 December 2016 Publication History

Abstract

Video matting is an extension of image matting and is used to extract the foreground matte from an arbitrary background of every frame in a video sequence. An automatic scribbling approach based on the relative motion of the foreground object with respect to the background in a video is introduced for video matting. The proposed scribble propagation and the subsequent isolation of foreground and background is much more intuitive than the conventional trimap propagation approach used for video matting. Alpha maps are propagated according to the optical flow estimated from the consecutive frames to get a preliminary estimate of the foreground and background in the following frame. Accurate scribbles are placed near the boundary of the foreground region for refining the scribbled image with the help of morphological operations. We show that a high quality matte of foreground object can be obtained using a state-of-the-art image matting technique. We show that the results obtained using the proposed method are accurate and comparable with that of other state-of-the-art video matting techniques.

References

[1]
Tracking dataset. http://cmp.felk.cvut.cz/~vojirtom/dataset/. Accessed: 2016-10-12.
[2]
R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk. Slic superpixels. Technical report, 2010.
[3]
N. Apostoloff and A. Fitzgibbon. Bayesian video matting using learnt image priors. In CVPR. IEEE, 2004.
[4]
B. Babenko, M.-H. Yang, and S. Belongie. Visual tracking with online multiple instance learning. In CVPR. IEEE, 2009.
[5]
X. Bai, J. Wang, D. Simons, and G. Sapiro. Video snapcut: robust video object cutout using localized classifiers. In ACM Transactions on Graphics (TOG), volume 28, page 70. ACM, 2009.
[6]
A. Berman, A. Dadourian, and P. Vlahos. Method for removing from an image the background surrounding a selected object, Oct. 17 2000. US Patent 6,134,346.
[7]
A. Berman, P. Vlahos, and A. Dadourian. Comprehensive method for removing from an image the background surrounding a selected subject, Oct. 17 2000. US Patent 6,134,345.
[8]
Q. Chen, D. Li, and C.-K. Tang. Knn matting. IEEE transactions on pattern analysis and machine intelligence, 35(9):2175--2188, 2013.
[9]
Y.-Y. Chuang, A. Agarwala, B. Curless, D. H. Salesin, and R. Szeliski. Video matting of complex scenes. In ACM Transactions on Graphics (TOG), volume 21, pages 243--248. ACM, 2002.
[10]
Y.-Y. Chuang, B. Curless, D. H. Salesin, and R. Szeliski. A bayesian approach to digital matting. In CVPR. IEEE, 2001.
[11]
R. T. Collins, Y. Liu, and M. Leordeanu. Online selection of discriminative tracking features. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 27(10):1631--1643, 2005.
[12]
P. Doğan, T. O. Aydin, N. Stefanoski, and A. Smolic. Key-frame based spatiotemporal scribble propagation. In Proceedings of the Eurographics Workshop on Intelligent Cinematography and Editing, pages 13--20. Eurographics Association, 2015.
[13]
M. Erofeev, Y. Gitman, D. Vatolin, A. Fedorov, and J. Wang. Perceptually motivated benchmark for video matting. In BMVC, 2015.
[14]
J. Fan, X. Shen, and Y. Wu. Scribble tracker: a matting-based approach for robust tracking. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 34(8):1633--1644, 2012.
[15]
J. Kwon and K. M. Lee. Visual tracking decomposition. In CVPR. IEEE, 2010.
[16]
A. Levin, D. Lischinski, and Y. Weiss. Colorization using optimization. In ACM Transactions on Graphics (TOG), volume 23, pages 689--694. ACM, 2004.
[17]
A. Levin, D. Lischinski, and Y. Weiss. A closed-form solution to natural image matting. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 30(2):228--242, 2008.
[18]
Y. Li, J. Sun, and H.-Y. Shum. Video object cut and paste. ACM Transactions on Graphics (TOG), 24(3):595--600, 2005.
[19]
M. McGuire, W. Matusik, H. Pfister, J. F. Hughes, and F. Durand. Defocus video matting. In ACM Transactions on Graphics (ToG), volume 24, pages 567--576. ACM, 2005.
[20]
Y. Mishima. Soft edge chroma-key generation based upon hexoctahedral color space, Oct. 11 1994. US Patent 5,355,174.
[21]
R. J. Qian and M. I. Sezan. Video background replacement without a blue screen. In ICIP. IEEE, 1999.
[22]
X. Ren and J. Malik. Tracking as repeated figure/ground segmentation. In CVPR. IEEE, 2007.
[23]
M. A. Ruzon and C. Tomasi. Alpha estimation in natural images. In CVPR. IEEE, 2000.
[24]
M. Sindeev, A. Konushin, and C. Rother. Alpha-flow for video matting. In ACCV, pages 438--452. Springer, 2012.
[25]
A. R. Smith and J. F. Blinn. Blue screen matting. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pages 259--268. ACM, 1996.
[26]
D. Sun, S. Roth, and M. J. Black. Secrets of optical flow estimation and their principles. In CVPR. IEEE, 2010.
[27]
Z. Tang, Z. Miao, Y. Wan, and D. Zhang. Video matting via opacity propagation. The Visual Computer, 28(1):47--61, 2012.
[28]
J. Xiao and M. Shah. Accurate motion layer segmentation and matting. In CVPR. IEEE, 2005.
[29]
M. Yang, Y. Wu, and G. Hua. Context-aware visual tracking. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 31(7):1195--1209, 2009.
[30]
D. Zou, X. Chen, G. Cao, and X. Wang. Video matting via sparse and low-rank representation. In ICCV, 2015.

Cited By

View all
  • (2019)Boundary Matched Human Area Segmentation for Chroma Keying Using Hybrid Depth-Color Analysis2019 IEEE 4th International Conference on Signal and Image Processing (ICSIP)10.1109/SIPROCESS.2019.8868469(761-767)Online publication date: Jul-2019
  • (2019)Efficient Human Detection Algorithm using Color & Depth information with Accurate Outer Boundary Matching2019 International Conference on Computer, Control, Informatics and its Applications (IC3INA)10.1109/IC3INA48034.2019.8949572(64-69)Online publication date: Oct-2019
  • (2018)Saliency Driven Video Motion MagnificationComputer Vision, Pattern Recognition, Image Processing, and Graphics10.1007/978-981-13-0020-2_9(89-100)Online publication date: 26-Apr-2018
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICVGIP '16: Proceedings of the Tenth Indian Conference on Computer Vision, Graphics and Image Processing
December 2016
743 pages
ISBN:9781450347532
DOI:10.1145/3009977
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

  • Google Inc.
  • QI: Qualcomm Inc.
  • Tata Consultancy Services
  • NVIDIA
  • MathWorks: The MathWorks, Inc.
  • Microsoft Research: Microsoft Research

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 December 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. computational photography
  2. optical flow
  3. video matting

Qualifiers

  • Research-article

Funding Sources

Conference

ICVGIP '16
Sponsor:
  • QI
  • MathWorks
  • Microsoft Research

Acceptance Rates

ICVGIP '16 Paper Acceptance Rate 95 of 286 submissions, 33%;
Overall Acceptance Rate 95 of 286 submissions, 33%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2019)Boundary Matched Human Area Segmentation for Chroma Keying Using Hybrid Depth-Color Analysis2019 IEEE 4th International Conference on Signal and Image Processing (ICSIP)10.1109/SIPROCESS.2019.8868469(761-767)Online publication date: Jul-2019
  • (2019)Efficient Human Detection Algorithm using Color & Depth information with Accurate Outer Boundary Matching2019 International Conference on Computer, Control, Informatics and its Applications (IC3INA)10.1109/IC3INA48034.2019.8949572(64-69)Online publication date: Oct-2019
  • (2018)Saliency Driven Video Motion MagnificationComputer Vision, Pattern Recognition, Image Processing, and Graphics10.1007/978-981-13-0020-2_9(89-100)Online publication date: 26-Apr-2018
  • (2017)Interest Region Based Motion MagnificationImage Analysis and Processing - ICIAP 201710.1007/978-3-319-68560-1_3(27-39)Online publication date: 13-Oct-2017

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media