Skip to main content
SpringerLink
Log in

Alpha matting using robust color sampling and fully connected conditional random fields

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Alpha matting refers to the problem of softly extracting the foreground from a given image. Previous matting approaches often focused on using naïve color sampling methods to estimate foreground and background colors for unknown pixels. Existing sampling-based matting methods often collect samples only near the unknown pixels, which may yield poor results if the true foreground and background samples are not found. In this paper, we present novel approach to extract foreground elements from an image through color and opacity (i.e., alpha) estimations, which consider available samples in a search window of variable size for each unknown pixel. Our proposed sampling method is robust in that similar sampling results can be generated for input trimaps of different unknown regions. Further, after the initial estimation of the alpha matte, a fully connected conditional random field (CRF) is used to correct the predicted matte at the pixel level. Our experiments show that visually plausible alpha mattes can indeed be produced.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. Most existing approaches require additional information via user input, either as trimaps or scribbles. The trimap, as shown in Fig. 1b, identifies known foreground (or background) pixels with opacity value α i = 1 (or α i = 0). The uncertain pixels are marked as unknown.

  2. All numerical values used in this section are determined empirically.

  3. The Chebyshev distance between two points p i and q i is \( \underset{i}{ \max}\left(\left|{p}_i-{q}_i\right|\right) \).

  4. Only one trimap is available for each of the images shown in Figs. 3 and 4.

  5. CRFs are used in different computer vision applications, particularly for low-level vision tasks such as image de-noising, optical flow, binocular stereo, and segmentation.

  6. http://graphics.stanford.edu/projects/densecrf/

  7. The parameters are specified for different test images used in [10].

References

  1. Berman A, Dadourian A, Vlahos P (2000) Method for removing from an image the background surrounding a selected object. US Patent 6:134,346

    Google Scholar 

  2. Chen Q, Li D, Tang C-K (2013) KNN Matting. IEEE Trans Pattern Anal Mach Intell 35(9):2175–2188

    Article  Google Scholar 

  3. Chen X, Zou D, Zhou S, Zhao Q, Tan P (2013) Image matting with local and nonlocal smooth priors. In Proceedings of 2013 I.E. conference on computer vision and pattern recognition, p 1902–1907

  4. Cho D, Tai Y-W, Kweon IS (2016) Natural image matting using deep convolutional neural networks. In Proc. ECCV

  5. Chuang Y, Curless B, Salesin DH, Szeliski R (2001) A Bayesian approach to digital matting. Proc. IEEE Conf. Computer vision and Pattern Recognition, p 264–271

  6. Gastal ESL, Oliveira MM (2010) Shared sampling for real-time alpha matting. Comput Graphics Forum 29(2):575–584

    Article  Google Scholar 

  7. He K, Rhemann C, Rother C, Tang X, Sun J (2011) A global sampling method for alpha matting. Proc. IEEE Conf. Computer vision and Pattern Recognition, p 2049–2056

  8. Johnson J, Rajan D, Cholakkal H (2014) Sparse codes as alpha matte. In Proceedings of British Machine Vision Conference. BMVA Press

  9. Karacan L, Erdem A, Erdem E (2015) Image matting with KL-divergence based sparse sampling. In IEEE 15th International Conference on Computer Vision, p 424–432

  10. Krähenbühl P, Koltun V (2011) Efficient inference in fully connected CRFs with Gaussian edge potentials. In Adv. Neural Info. Process. Syst. (NIPS), p 109–117

  11. Levin A, Lischinski D, Weiss Y (2006) A closed form solution to natural image matting. Proc. IEEE Int’l Conf. Computer vision and pattern recognition, p 61–68

  12. Levin A, Rav-Acha A, Lischinski D (2008) Spectral matting. IEEE Trans Pattern Anal Mach Intell 30(10):1699–1712

    Article  Google Scholar 

  13. Rhemann C, Rother C, Gelautz M (2009) Improving color modeling for alpha matting. In Proc. British machine vision conference, p 1155–1164

  14. Rhemann C, Rother C, Wang J, Gelautz M, Kohli P, Rott P (2009) A perceptually motivated online benchmark for image matting. Proc IEEE Conf Comput Vis Pattern Recognit:1826–1833

  15. Ruzon MA, Tomasi C (2000) Alpha estimation in natural images. Proc. IEEE Conf. Computer vision and Pattern Recognition, p 18–25, 2000

  16. Shahrian E, Rajan D (2012) Weighted color and texture sample selection for image matting. Proc. IEEE Conf. Computer vision and pattern recognition, p 718–725

  17. Shahrian E, Rajan D, Price B, Cohen S (2013) Improving image matting using comprehensive sampling sets. In Proceedings of 2013 I.E. conference on computer vision and pattern recognition, p 636–643

  18. Sun J, Jia J, Tang C-K, Shum H-Y (2004) Poisson Matting. ACM Trans Graph 23(3):315–321

    Article  Google Scholar 

  19. Tarabalka Y, Fauvel M, Chanussot J, Benediktsson JA (2010) SVM-and MRF-based method for accurate classification of hyperspectral images. IEEE Geosci Remote Sens Lett 7(4):736–740

    Article  Google Scholar 

  20. Wang J, Cohen M (2007) Optimized color sampling for robust matting. In IEEE conference on computer vision and pattern recognition (CVPR), p 1–8

  21. Zheng Y, Kambhamettu C (2009) Learning based digital matting. Proc. IEEE Int’l Conf. Computer vision, p 889–896

  22. Zhu Q-S, Shao L, Li X, Wang L (2015) Targeting accurate object extraction from an image: a comprehensive study of natural image matting. IEEE Trans Neural Netw Learn Syst 26(2):185–207

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The Department of Computer Science, National Chiao Tung University, Hsinchu, 30010, Taiwan

    Fang-Ju Lin & Jen-Hui Chuang

Authors
  1. Fang-Ju Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jen-Hui Chuang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fang-Ju Lin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, FJ., Chuang, JH. Alpha matting using robust color sampling and fully connected conditional random fields. Multimed Tools Appl 77, 14327–14342 (2018). https://doi.org/10.1007/s11042-017-5031-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5031-0

Keywords

Search

Navigation