Skip to main content
SpringerLink
Log in

Compressive environment matting

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

The existing high-quality environment matting methods usually require the capturing of a few thousand sample images and spend a few hours in data acquisition. In this paper, a novel environment matting algorithm is proposed to capture and extract the environment matte data effectively and efficiently. First, the recently developed compressive sensing theory is incorporated to reformulate the environment matting problem and simplify the data acquisition process. Next, taking into account the special properties of light refraction and reflection effects of transparent objects, two advanced priors, group clustering and Gaussian priors, as well as other basic constraints are introduced during the matte data recovery process to combat with the limited image samples, suppress the effects of the measurement noise resulted from data acquisition, and faithfully recover the sparse environment matte data. Compared with most of the existing environment matting methods, our algorithm significantly simplifies and accelerates the environment matting extraction process while still achieving high-accurate composition results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Baraniuk, R., Davenport, M., DeVore, R., Wakin, M.: A simple proof of the restricted isometry property for random matrices. Constr. Approx. 28(3), 253–263 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  2. Candès, E.J., Romberg, J.K., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)

    Article  MATH  Google Scholar 

  3. Candès, E.J., Tao, T.: Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Trans. Inf. Theory 52(12), 5406–5425 (2006)

    Article  MATH  Google Scholar 

  4. Chuang, Y.-Y., Zongker, D.E., Hindorff, J., Curless, B., Salesin, D., Szeliski, R.: Environment matting extensions: towards higher accuracy and real-time capture. In: SIGGRAPH ’00, pp. 121–130. ACM Press, New York (2000)

  5. Duan, Q., Cai, J., Zheng, J., Lin, W.: Fast environment matting extraction using compressive sensing. In: 2011 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6 (2011)

  6. Duarte, M., Davenport, M., Takhar, D., Laska, J., Sun, T., Kelly, K., Baraniuk, R.: Single-pixel imaging via compressive sampling. IEEE Signal Process. Mag. 25(2), 83–91 (2008)

    Article  Google Scholar 

  7. Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., Sagar, M.: Acquiring the reflectance field of a human face. In: SIGGRAPH ’00, pp. 145–156. ACM Press, New York (2000)

  8. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  9. Fuchs, M., Blanz, V., Lensch, H.P., Seidel, H.-P.: Adaptive sampling of reflectance fields. ACM Trans. Gr. 26(2), 10 (2007)

    Article  Google Scholar 

  10. Goyal, V., Fletcher, A., Rangan, S.: Compressive sampling and lossy compression. IEEE Signal Process. Mag. 25(2), 48–56 (2008)

    Article  Google Scholar 

  11. Gu, J., Nayar, S., Grinspun, E., Belhumeur, P., Ramamoorthi, R.: Compressive structured light for recovering inhomogeneous participating media. In: Proceedings of the 10th European Conference on Computer Vision, ECCV 2008, pp. 845–858. Springer, Berlin (2008)

  12. Garg, G.G., Talvala, E.-V., Levoy, M., Lensch, H.P.A.: Symmetric photography: Exploiting data-sparseness in reflectance fields. In: Anenine-Möller, T., Heidrich, W., (eds.) Rendering Techniques 2006: Eurographics Symposium on Rendering, pp. 251–262. Eurographics Association, Nicosia (2006)

  13. Haldar, J., Hernando, D., Liang, Z.P.: Compressed-sensing MRI with random encoding. IEEE Trans. Med. Imaging 30(4), 893–903 (2011)

    Article  Google Scholar 

  14. Huang, J., Huang, X., Metaxas, D.N.: Learning with dynamic group sparsity. In: IEEE 12th International Conference on Computer Vision, ICCV 2009, pp. 64–71. Kyoto (2009)

  15. Ihrke, I., Ziegler, G., Tevs, A., Theobalt, C., Magnor, M.A., Seidel, H.-P.: Eikonal rendering: efficient light transport in refractive objects. ACM Trans. Gr. 26(3), 59 (2007)

    Article  Google Scholar 

  16. Lustig, M., Donoho, D., Pauly, J.M.: Sparse MRI: The application of compressed sensing for rapid MR imaging. Magn. Reson. Med 58(6), 1182–1195 (2007)

    Article  Google Scholar 

  17. Levoy, M., Hanrahan, P.: Light field rendering. In: SIGGRAPH, pp. 31–42 (1996)

  18. Matusik, W., Loper, M., Pfister, H.: Progressively-refined reflectance functions from natural illumination. In: Proceedings of the 15th Eurographics workshop on rendering, Eurographics Association, pp. 299–308 (2004)

  19. Masselus, V., Peers, P., Dutré, P., Willems, Y.D.: Relighting with 4D incident light fields. In: SIGGRAPH ’03, pp. 613–620. ACM, New York (2003)

  20. Matusik, W., Pfister, H., Ziegler, R., Ngan, A., McMillan, L.: Acquisition and rendering of transparent and refractive objects. In: Proceedings of the 13th Eurographics workshop on rendering, Eurographics Association, pp. 267–278 (2002)

  21. Peers, P., Dutré, P.: Wavelet environment matting. In: Proceedings of the 14th Eurographics workshop on rendering, Eurographics Association, pp. 157–166 (2003)

  22. Peers, P., Dutré, P.: Inferring reflectance functions from wavelet noise. In: Proceedings of the 16th Eurographics workshop on rendering, Eurographics Association, pp. 173–182 (2005)

  23. Peyre, G.: Best basis compressed sensing. IEEE Trans. Signal Process. 58(5), 2613–2622 (2010)

    Article  MathSciNet  Google Scholar 

  24. Peers, P., Mahajan, D., Lamond, B., Ghosh, A., Matusik, W., Ramamoorthi, R., Debevec, P.E.: Compressive light transport sensing. ACM Trans. Gr. 28(1), 1355–1363 (2009)

  25. Sen, P., Chen, B., Garg, G., Marschner, S.R., Horowitz, M., Levoy, M., Lensch, H.P.A.: Dual photography. ACM Trans. Gr. 24(3), 745–755 (2005)

    Article  Google Scholar 

  26. Sen, P., Darabi, S.: Compressive dual photography. Comput. Gr. Forum 28(2), 609–618 (2009)

    Article  Google Scholar 

  27. Sen, P., Darabi, S.: Compressive estimation for signal integration in rendering. Comput. Gr. Forum 29(4), 1355–1363 (2010)

  28. Sen, P., Darabi, S.: Compressive rendering: a rendering application of compressed sensing. IEEE Trans. Vis. Comput. Gr. 17(4), 487–499 (2011)

    Article  Google Scholar 

  29. Wang, J., Dong, Y., Tong, X., Lin, Z., Guo, B.: Kernel nyström method for light transport. In: SIGGRAPH ’09: ACM SIGGRAPH 2009 papers, , pp. 1–10. ACM, New York (2009)

  30. Wexler, Y., Fitzgibbon, A.W., Zisserman, A.: Image-based environment matting. In: Rendering Techniques, pp. 279–290 (2002)

  31. Wakin, M., Laska, J., Duarte, M., Baron, D., Sarvotham, S., Takhar, D., Kelly, K., Baraniuk, R.: An architecture for compressive imaging. In: 2006 IEEE International Conference on Image Processing, pp. 1273–1276 (2006)

  32. Zongker, D.E., Werner, D.M., Curless, B., Salesin, D.: Environment matting and compositing. In: SIGGRAPH, pp. 205–214 (1999)

  33. Zhu, J., Yang, Y.-H.: Frequency-based environment matting. In: Pacific Conference on Computer Graphics and Applications, pp. 402–410. IEEE Computer Society (2004)

  34. Zhang, S., Zhan, Y., Dewan, M., Huang, J., Metaxas, D.N., Zhou, X.S.: Towards robust and effective shape modeling: Sparse shape composition. Med Image Anal 16(1), 265–277 (2012)

Download references

Author information

Authors and Affiliations

  1. School of Computer Engineering, Nanyang Technological University, Singapore, Singapore

    Qi Duan, Jianfei Cai & Jianmin Zheng

Authors
  1. Qi Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianfei Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianmin Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianfei Cai.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Duan, Q., Cai, J. & Zheng, J. Compressive environment matting. Vis Comput 31, 1587–1600 (2015). https://doi.org/10.1007/s00371-014-1035-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-014-1035-1

Keywords

Search

Navigation