Skip to main content
SpringerLink
Log in

Intrinsic Decomposition by Learning from Varying Lighting Conditions

  • Conference paper
  • First Online:
Advances in Visual Computing (ISVC 2019)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11844))

Included in the following conference series:

  • 2027 Accesses

Abstract

Intrinsic image decomposition describes an image based on its reflectance and shading components. In this paper we tackle the problem of estimating the diffuse reflectance from a sequence of images captured from a fixed viewpoint under various illuminations. To this end we propose a deep learning approach to avoid heuristics and strong assumptions on the reflectance prior. We compare two network architectures: one classic ‘U’ shaped Convolutional Neural Network (CNN) and a Recurrent Neural Network (RNN) composed of Convolutional Gated Recurrent Units (CGRU). We train our networks on a new dataset specifically designed for the task of intrinsic decomposition from sequences. We test our networks on MIT and BigTime datasets and outperform state-of-the-art algorithms both qualitatively and quantitatively.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Baslamisli, A.S., Le, H.-A., Gevers, T.: CNN based learning using reflection and retinex models for intrinsic image decomposition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

    Google Scholar 

  2. Bell, S., Bala, K., Snavely, N.: Intrinsic images in the wild. ACM Trans. Graph. (SIGGRAPH) 33(4), 159 (2014)

    Article  Google Scholar 

  3. Butler, D., Wulff, J., Stanley, G., Black, M.: MPI-Sintel optical flow benchmark: supplemental material. Technical report, MPI-IS-TR-006, MPI for Intelligent Systems (2012)

    Google Scholar 

  4. Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)

  5. Duchêne, S., et al.: Multi-view intrinsic images of outdoors scenes with an application to relighting. ACM Trans. Graph. 34(5) (2015)

    Google Scholar 

  6. Fan, Q., Yang, J., Hua, G., Chen, B., Wipf, D.: Revisiting deep intrinsic image decompositions. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

    Google Scholar 

  7. Goldluecke, B., Cremers, D.: An approach to vectorial total variation based on geometric measure theory. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 327–333, June 2010

    Google Scholar 

  8. Grosse, R., Johnson, M.K., Adelson, E.H., Freeman, W.T.: Ground truth dataset and baseline evaluations for intrinsic image algorithms. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 2335–2342, September 2009

    Google Scholar 

  9. Hochreiter, S., Bengio, Y., Frasconi, P.: Gradient flow in recurrent nets: the difficulty of learning long-term dependencies. In: Kolen, J., Kremer, S. (eds.) Field Guide to Dynamical Recurrent Networks. IEEE Press, Piscataway (2001)

    Google Scholar 

  10. Janner, M., Wu, J., Kulkarni, T.D., Yildirim, I., Tenenbaum, J.: Self-supervised intrinsic image decomposition. In: Advances in Neural Information Processing Systems, pp. 5938–5948 (2017)

    Google Scholar 

  11. Kong, N., Gehler, P.V., Black, M.J.: Intrinsic video. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 360–375. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10605-2_24

    Chapter  Google Scholar 

  12. Kovacs, B., Bell, S., Snavely, N., Bala, K.: Shading annotations in the wild. In: Computer Vision and Pattern Recognition (CVPR) (2017)

    Google Scholar 

  13. Laffont, P., Bazin, J.: Intrinsic decomposition of image sequences from local temporal variations. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 433–441, December 2015

    Google Scholar 

  14. Laffont, P.-Y.: Intrinsic image decomposition from multiple photographs. Ph.D. thesis, Inria/University of Nice Sophia-Antipolis, October 2012

    Google Scholar 

  15. Lee, K.J., et al.: Estimation of intrinsic image sequences from image+depth video. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 327–340. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33783-3_24

    Chapter  Google Scholar 

  16. Lettry, L., Vanhoey, K., Gool, L.V.: DARN: a deep adversarial residual network for intrinsic image decomposition. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1359–1367 (2018)

    Google Scholar 

  17. Li, Z., Snavely, N.: Learning intrinsic image decomposition from watching the world. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

    Google Scholar 

  18. Matsushita, Y., Lin, S., Kang, S.B., Shum, H.-Y.: Estimating intrinsic images from image sequences with biased illumination. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3022, pp. 274–286. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24671-8_22

    Chapter  Google Scholar 

  19. Meka, A., Fox, G., Zollhöfer, M., Richardt, C., Theobalt, C.: Live user-guided intrinsic video for static scenes. IEEE Trans. Vis. Comput. Graph. 23(11), 2447–2454 (2017)

    Article  Google Scholar 

  20. Meka, A., Zollhöfer, M., Richardt, C., Theobalt, C.: Live intrinsic video. ACM Trans. Graph. 35(4), 109:1–109:14 (2016)

    Article  Google Scholar 

  21. Narihira, T., Maire, M., Yu, S.X.: Direct intrinsics: learning albedo-shading decomposition by convolutional regression. In: Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), ICCV 2015, pp. 2992–2992. IEEE Computer Society, Washington, DC (2015)

    Google Scholar 

  22. Narihira, T., Maire, M., Yu, S.X.: Learning lightness from human judgement on relative reflectance. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2965–2973 (2015)

    Google Scholar 

  23. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  24. Shi, J., Dong, Y., Su, H., Stella, X.Y.: Learning non-lambertian object intrinsics across shapenet categories. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5844–5853. IEEE (2017)

    Google Scholar 

  25. Weiss, Y.: Deriving intrinsic images from image sequences. In: Eighth IEEE International Conference on Computer Vision, ICCV 2001, Proceedings, vol. 2, pp. 68–75. IEEE (2001)

    Google Scholar 

  26. Xingjian, S., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K., Woo, W.-C.: Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Advances in Neural Information Processing Systems, pp. 802–810 (2015)

    Google Scholar 

  27. Ye, G., Garces, E., Liu, Y., Dai, Q., Gutierrez, D.: Intrinsic video and applications. ACM Trans. Graph. 33(4), 80:1–80:11 (2014)

    Article  Google Scholar 

  28. Zhao, Q., Tan, P., Dai, Q., Shen, L., Wu, E., Lin, S.: A closed-form solution to retinex with nonlocal texture constraints. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1437–1444 (2012)

    Article  Google Scholar 

  29. Zhou, T., Krähenbühl, P., Efros, A.A.: Learning data-driven reflectance priors for intrinsic image decomposition. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 3469–3477, December 2015

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Technicolor R&I, Rennes, France

    Gregoire Nieto

  2. InterDigital R&D, 975 Avenue des Champs Blancs, 35576, Cesson-Sevigne, France

    Mohammad Rouhani & Philippe Robert

Authors
  1. Gregoire Nieto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Rouhani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Philippe Robert
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Rouhani .

Editor information

Editors and Affiliations

  1. University of Nevada, Reno, NV, USA

    George Bebis

  2. NASA Ames Research Center, Moffett Field, CA, USA

    Richard Boyle

  3. University of Nevada, Reno, NV, USA

    Bahram Parvin

  4. Desert Research Institute, Reno, NV, USA

    Darko Koracin

  5. Lawrence Berkeley National Laboratory, Berkeley, CA, USA

    Daniela Ushizima

  6. Latent AI, Palo Alto, CA, USA

    Sek Chai

  7. Texas A&M University, College Station, TX, USA

    Shinjiro Sueda

  8. Louisiana State University, Baton Rouge, LA, USA

    Xin Lin

  9. University of North Carolina at Charlotte, Charlotte, NC, USA

    Aidong Lu

  10. École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland

    Daniel Thalmann

  11. Notre Dame University, Notre Dame, IN, USA

    Chaoli Wang

  12. Bosch Research North America, Palo Alto, CA, USA

    Panpan Xu

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nieto, G., Rouhani, M., Robert, P. (2019). Intrinsic Decomposition by Learning from Varying Lighting Conditions. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2019. Lecture Notes in Computer Science(), vol 11844. Springer, Cham. https://doi.org/10.1007/978-3-030-33720-9_50

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-33720-9_50

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-33719-3

  • Online ISBN: 978-3-030-33720-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation