Skip to main content
SpringerLink
Log in

Light mixture intrinsic image decomposition based on a single RGB-D image

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

We propose a novel intrinsic image decomposition method based on a single RGB-D image. We first separate the shading image into an illumination color component, a distant shading component and a local shading component, inducing a novel intrinsic image model that can encode color and spatial variation of scene illumination. Unlike previous methods, which assume illumination color is white, our light mixture model encodes scene illumination with two different light types, and an automatical strategy is proposed to calculate the color of the two light types. We also adopt physical-based illumination prior to infer the distant shading component. To do so, we firstly recover the illumination distribution of the distant light sources through solving a system of linear equations with sparse and non-negative constraints. Then, the recovered illumination is used to synthesize a coarse distant shading image jointly with the depth map. Later, the synthetic image is employed as an additional constraint of distant shading component. To reduce noise disturbance from the synthetic distant shading image, a novel sampling strategy was proposed. Finally, we consider the similarity of material locally and globally, which gives reliable constraints to the reflectance component. Experimental results demonstrate the validity and flexibility of our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Barron, J.T., Malik, J.: Shape, albedo, and illumination from a single image of an unknown object. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 334–341 (2012)

  2. Barron, J.T., Malik, J.: Intrinsic scene properties from a single RGB-D image. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 17–24 (2013)

  3. Bell, M., Freeman, W.T.: Learning local evidence for shading and reflectance. In: Proceedings of the International Conference on Computer Vision, pp. 670–677 (2001)

  4. Bell, S., Bala, K., Snavely, N.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)

    Article  Google Scholar 

  5. Bell, S., Bala, K., Snavely, N.: Intrinsic images in the wild. ACM Trans. Graph. (SIGGRAPH) 33(4), 159 (2014)

    Article  Google Scholar 

  6. Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: Fitzgibbon, A. et al. (eds.) European Conference on Computer Vision (ECCV), Part IV. LNCS, vol. 7577, pp. 611–625. Springer, Berlin (2012)

  7. Chen, Q., Koltun, V.: A simple model for intrinsic image decomposition with depth cues. In: Proceedings of the International Conference on Computer Vision, pp. 241–248 (2013)

  8. Gehler, P., Rother, C., Kiefel, M., Zhang, L., Schölkopf, B.: Recovering intrinsic images with a global sparsity prior on reflectance. In: Advances in Neural Information Processing Systems (NIPS), pp. 765–773 (2011)

  9. Grosse, R., Johnson, M.K., Adelson, E.H., Freeman, W.T.: Ground-truth dataset and baseline evaluations for intrinsic image algorithms. In: International Conference on Computer Vision, pp. 2335–2342 (2009)

  10. Hsu, E., Mertens, T., Paris, S., Avidan, S., Durand, F.: Light mixture estimation for spatially varying white balance. In: Proceedings of SIGGRAPH 2008, pp. 70:1–7. Los Angeles, California, USA (2008)

  11. Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., Fitzgibbon, A.: Kinectfusion: real-time 3d reconstruction and interaction using a moving depth camera. In: Proceedings of UIST, pp. 559–568 (2011)

  12. Land, E.H., McCann, J.J.: Lightness and retinex theory. J. Opt. Soc. Am. 61(1), 1–11 (1978)

    Article  Google Scholar 

  13. Lee, K.J., Zhao, Q., Tong, X., Gong, M., Izadi, S., Lee, S.U., Tan, P., Lin, S.: Estimation of intrinsic image sequences from image+depth video. In: Proceedings of the 12th European Conference on Computer Vision, pp. 327–340 (2012)

  14. Levin, L., Weiss, Y.: A closed form solution to natural image matting. In: IEEE Computer Vision and Pattern Recognition, pp. 61–68 (2006)

  15. Mei, X., Ling, H., Jacobs, D.W.: Sparse representation of cast shadows via l1-regularized least squares. In: Proceedings of ICCV, pp. 583–590. Kyoto, Japan (2009)

  16. Shen, L., Yeo, C., Hua, B.S.: Intrinsic images decomposition using a local and global sparse representation of reflectance. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2904–2915 (2011)

  17. Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from rgbd images. In: Proc. ECCV, pp. 746–760 (2012)

  18. Sinha, P., Adelson, E.: Recovering reflectance and illumination in a world of painted polyhedra. In: Proceedings of the Fourth International Conference on Computer Vision, pp. 156–163 (1993)

  19. Tappen, M.F., Adelson, E.H., Freeman, W.T.: Estimating intrinsic component images using non-linear regression. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1992–1999 (2006)

  20. Tappen, M.F., Freeman, W.T., Adelson, E.H.: Recovering intrinsic images from a single image. IEEE Trans. Pattern Anal. Mach. Intell. 27(9), 1459–1472 (2005)

    Article  Google Scholar 

  21. Zhai, Y., Shah, M.: Visual attention detection in video sequences using spatiotemporal cues. In: Proceedings of the 14th ACM International Conference on Multimedia, pp. 815–824 (2006)

Download references

Acknowledgments

This research is supported by National Natural Science Foundation of China (Grant Nos. 61402081, 61572333), 863 Program of China (Grant No. 2015AA016405), Fundamental Research Funds for the Central Universities (Grant No. ZYGX2014J059), China Scholarship Council (Grant No. [2015]3012) and the Oversea Academic Training Funds, UESTC. Ling was supported in part by National Science Foundation (Grant Nos. 1449860, 1218156 and 1350521).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, University of Electronic Science and Technology of China (UESTC), Chengdu, China

    Guanyu Xing & Wanfa Zhang

  2. Center for Robotics, UESTC, Chengdu, China

    Guanyu Xing & Wanfa Zhang

  3. Department of Computer and Information Sciences, Center for Data Analytics and Biomedical Informatics, Temple University, Philadelphia, PA, USA

    Guanyu Xing & Haibin Ling

  4. College of Computer Science, Sichuan University, Chengdu, China

    Yanli Liu

Authors
  1. Guanyu Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanli Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wanfa Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haibin Ling
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guanyu Xing.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xing, G., Liu, Y., Zhang, W. et al. Light mixture intrinsic image decomposition based on a single RGB-D image. Vis Comput 32, 1013–1023 (2016). https://doi.org/10.1007/s00371-016-1238-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-016-1238-8

Keywords

Search

Navigation