skip to main content
10.1145/3009977.3010046acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicvgipConference Proceedingsconference-collections
research-article

Intrinsic image decomposition using focal stacks

Published: 18 December 2016 Publication History

Abstract

In this paper, we presents a novel method (RGBF-IID) for intrinsic image decomposition of a wild scene without any restrictions on the complexity, illumination or scale of the image. We use focal stacks of the scene as input. A focal stack captures a scene at varying focal distances. Since focus depends on distance to the object, this representation has information beyond an RGB image towards an RGBD image with depth. We call our representation an RGBF image to highlight this. We use a robust focus measure and generalized random walk algorithm to compute dense probability maps across the stack. These maps are used to define sparse local and global pixel neighbourhoods, adhering to the structure of the underlying 3D scene. We use these neighbourhood correspondences with standard chromaticity assumptions as constraints in an optimization system. We present our results on both indoor and outdoor scenes using manually captured stacks of random objects under natural as well as artificial lighting conditions. We also test our system on a larger dataset of synthetically generated focal stacks from NYUv2 and MPI Sintel datasets and show competitive performance against current state-of-the-art IID methods that use RGBD images. Our method provides a strong evidence for the potential of RGBF modality in place of RGBD in computer vision.

Supplementary Material

ZIP File (a88-saini.zip)
Supplemental material.

References

[1]
Magic lantern. http://magiclantern.fm/.
[2]
Aseem Agarwala, Mira Dontcheva, Maneesh Agrawala, Steven Drucker, Alex Colburn, Brian Curless, David Salesin, and Michael Cohen. Interactive digital photomontage. In ACM Transactions on Graphics, 2004.
[3]
Jonathan T. Barron. Shape, albedo, and illumination from a single image of an unknown object. In CVPR, 2012.
[4]
Jonathan T. Barron and Jitendra Malik. Color constancy, intrinsic images, and shape estimation. In ECCV, 2012.
[5]
Jonathan T Barron, Andrew Adams, YiChang Shih, and Carlos Hernández. Fast bilateral-space stereo for synthetic defocus. In CVPR, 2015.
[6]
Sean Bell, Kavita Bala, and Noah Snavely. Intrinsic images in the wild. ACM Transactions on Graphics, 2014.
[7]
D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black. A naturalistic open source movie for optical flow evaluation. In ECCV, 2012.
[8]
Subhasis Chaudhuri and Ambasamudram N Rajagopalan. Depth from defocus: a real aperture imaging approach. Springer Science & Business Media, 2012.
[9]
Q. Chen and V. Koltun. A simple model for intrinsic image decomposition with depth cues. In ICCV, 2013.
[10]
Sylvain Duchěne, Clement Riant, Gaurav Chaurasia, Jorge Lopez Moreno, Pierre-Yves Laffont, Stefan Popov, Adrien Bousseau, and George Drettakis. Multiview intrinsic images of outdoors scenes with an application to relighting. ACM Transactions on Graphics, 2015.
[11]
Elmar Eisemann and Frédo Durand. Flash photography enhancement via intrinsic relighting. ACM Transactions on Graphics, 2004.
[12]
P. Favaro and S. Soatto. A geometric approach to shape from defocus. IEEE TPAMI, 2005.
[13]
Elena Garces, Adolfo Munoz, Jorge Lopez-Moreno, and Diego Gutierrez. Intrinsic images by clustering. Computer Graphics Forum (Proc. EGSR 2012).
[14]
Roger Grosse, Micah K. Johnson, Edward H. Adelson, and William T. Freeman. Ground-truth dataset and baseline evaluations for intrinsic image algorithms. In ICCV, 2009.
[15]
Samuel W Hasinoff and Kiriakos N Kutulakos. Light-efficient photography. IEEE TPAMI, 2011.
[16]
D. Hoiem, A. A. Efros, and M. Hebert. Geometric context from a single image. In ICCV, 2005.
[17]
Junho Jeon, Sunghyun Cho, Xin Tong, and Seungyong Lee. Intrinsic image decomposition using structure-texture separation and surface normals. In ECCV, 2014.
[18]
Vivek Kwatra, Mei Han, and Shengyang Dai. Shadow removal for aerial imagery by information theoretic intrinsic image analysis. In ICCP, 2012.
[19]
Edwin H. Land and John J. McCann. Lightness and retinex theory. J. Opt. Soc. Am., 1971.
[20]
A. Levin, D. Lischinski, and Y. Weiss. A closed-form solution to natural image matting. IEEE TPAMI, 2008.
[21]
Xiaopei Liu, Liang Wan, Yingge Qu, Tien-Tsin Wong, Stephen Lin, Chi-Sing Leung, and Pheng-Ann Heng. Intrinsic colorization. ACM Transactions on Graphics (SIGGRAPH Asia 2008 issue).
[22]
Santiago Manen, Matthieu Guillaumin, and Luc Van Gool. Prime object proposals with randomized prim's algorithm. In ICCV, 2013.
[23]
Hajime Nagahara, Sujit Kuthirummal, Changyin Zhou, and Shree K Nayar. Flexible depth of field photography. In ECCV. 2008.
[24]
Pushmeet Kohli Nathan Silberman, Derek Hoiem and Rob Fergus. Indoor segmentation and support inference from rgbd images. In ECCV, 2012.
[25]
S. K. Nayar and Y. Nakagawa. Shape from focus. IEEE TPAMI, 1994.
[26]
Carsten Rother, Martin Kiefel, Lumin Zhang, Bernhard Schölkopf, and Peter V Gehler. Recovering intrinsic images with a global sparsity prior on reflectance. In NIPS.
[27]
Parikshit Sakurikar and PJ Narayanan. Dense view interpolation on mobile devices using focal stacks. In CVPRW, 2014.
[28]
R. Shen, I. Cheng, J. Shi, and A. Basu. Generalized random walks for fusion of multi-exposure images. IEEE TIP, 2011.
[29]
Supasorn Suwajanakorn, Carlos Hernandez, and Steven M. Seitz. Depth from focus with your mobile phone. In CVPR, 2015.
[30]
M. F. Tappen, W. T. Freeman, and E. H. Adelson. Recovering intrinsic images from a single image. IEEE TPAMI, 2005.
[31]
Y. Weiss. Deriving intrinsic images from image sequences. In ICCV, 2001.
[32]
Ning Xu, Kar-Han Tan, Himanshu Arora, and Narendra Ahuja. Generating omnifocus images using graph cuts and a new focus measure. In ICPR, 2004.
[33]
Genzhi Ye, Elena Garces, Yebin Liu, Qionghai Dai, and Diego Gutierrez. Intrinsic video and applications. ACM Transactions on Graphics, 2014.

Cited By

View all
  • (2024)Specularity Factorization for Low-Light Enhancement2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00009(1-12)Online publication date: 16-Jun-2024
  • (2022)Quaternion Factorized Simulated Exposure FusionProceedings of the Thirteenth Indian Conference on Computer Vision, Graphics and Image Processing10.1145/3571600.3571604(1-9)Online publication date: 8-Dec-2022
  • (2022)Interpreting Intrinsic Image Decomposition using Concept ActivationsProceedings of the Thirteenth Indian Conference on Computer Vision, Graphics and Image Processing10.1145/3571600.3571603(1-9)Online publication date: 8-Dec-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICVGIP '16: Proceedings of the Tenth Indian Conference on Computer Vision, Graphics and Image Processing
December 2016
743 pages
ISBN:9781450347532
DOI:10.1145/3009977
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

  • Google Inc.
  • QI: Qualcomm Inc.
  • Tata Consultancy Services
  • NVIDIA
  • MathWorks: The MathWorks, Inc.
  • Microsoft Research: Microsoft Research

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 December 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. RGBF
  2. RGBF-IID
  3. focal stacks
  4. generalized random walk
  5. intrinsic image decomposition

Qualifiers

  • Research-article

Conference

ICVGIP '16
Sponsor:
  • QI
  • MathWorks
  • Microsoft Research

Acceptance Rates

ICVGIP '16 Paper Acceptance Rate 95 of 286 submissions, 33%;
Overall Acceptance Rate 95 of 286 submissions, 33%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)5
  • Downloads (Last 6 weeks)2
Reflects downloads up to 18 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Specularity Factorization for Low-Light Enhancement2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00009(1-12)Online publication date: 16-Jun-2024
  • (2022)Quaternion Factorized Simulated Exposure FusionProceedings of the Thirteenth Indian Conference on Computer Vision, Graphics and Image Processing10.1145/3571600.3571604(1-9)Online publication date: 8-Dec-2022
  • (2022)Interpreting Intrinsic Image Decomposition using Concept ActivationsProceedings of the Thirteenth Indian Conference on Computer Vision, Graphics and Image Processing10.1145/3571600.3571603(1-9)Online publication date: 8-Dec-2022
  • (2019)Intrinsic Face Image Decomposition from RGB Images with Depth CuesAdvances in Visual Informatics10.1007/978-3-030-34032-2_14(149-156)Online publication date: 2019
  • (2017)Intrinsic Decompositions for Image EditingComputer Graphics Forum10.5555/3128975.312902736:2(593-609)Online publication date: 1-May-2017

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media