research-article

Break Ames room illusion: depth from general single images

Authors:

Jiaya JiaAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 34, Issue 6

Article No.: 225, Pages 1 - 11

https://doi.org/10.1145/2816795.2818136

Published: 02 November 2015 Publication History

Abstract

Photos compress 3D visual data to 2D. However, it is still possible to infer depth information even without sophisticated object learning. We propose a solution based on small-scale defocus blur inherent in optical lens and tackle the estimation problem by proposing a non-parametric matching scheme for natural images. It incorporates a matching prior with our newly constructed edgelet dataset using a non-local scheme, and includes semantic depth order cues for physically based inference. Several applications are enabled on natural images, including geometry based rendering and editing.

References

[1]

Afonso, M. V., Bioucas-Dias, J. M., and Figueiredo, M. A. 2010. Fast image recovery using variable splitting and constrained optimization. TIP 19, 9, 2345--2356.

Digital Library

[2]

Bae, S., and Durand, F. 2007. Defocus magnification. Computer Graphics Forum 26, 3, 571--579.

[3]

Chakrabarti, A., Zickler, T., and Freeman, W. T. 2010. Analyzing spatially-varying blur. In CVPR, 2512--2519.

[4]

Chen, Q., and Koltun, V. 2013. A simple model for intrinsic image decomposition with depth cues. In ICCV, 241--248.

Digital Library

[5]

Cossairt, O., Zhou, C., and Nayar, S. 2010. Diffusion coded photography for extended depth of field. TOG 29, 4, 31.

Digital Library

[6]

Eigen, D., Puhrsch, C., and Fergus, R. 2014. Depth map prediction from a single image using a multi-scale deep network. In NIPS, 2366--2374.

[7]

Elder, J. H., and Zucker, S. W. 1998. Local scale control for edge detection and blur estimation. TPAMI 20, 7, 699--716.

Digital Library

[8]

Hoiem, D., Stein, A. N., Efros, A. A., and Hebert, M. 2007. Recovering occlusion boundaries from a single image. In ICCV, 1--8.

[9]

Jia, J., Sun, J., Tang, C.-K., and Shum, H.-Y. 2006. Drag-and-drop pasting. TOG 25, 3, 631--637.

Digital Library

[10]

Karsch, K., Liu, C., and Kang, S. B. 2012. Depth extraction from video using non-parametric sampling. In ECCV, 775--788.

Digital Library

[11]

Khoshelham, K., and Elberink, S. O. 2012. Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12, 2, 1437--1454.

[12]

Ladicky, L., Shi, J., and Pollefeys, M. 2014. Pulling things out of perspective. In CVPR, 89--96.

Digital Library

[13]

Levin, A., Fergus, R., Durand, F., and Freeman, W. T. 2007. Image and depth from a conventional camera with a coded aperture. TOG 26, 3, 70.

Digital Library

[14]

Liang, C.-K., Lin, T.-H., Wong, B.-Y., Liu, C., and Chen, H. H. 2008. Programmable aperture photography: Multiplexed light field acquisition. TOG 27, 3, 55.

Digital Library

[15]

Maini, R., and Sohal, J. 2006. Performance evaluation of prewitt edge detector for noisy images. GVIP Journal 6, 3, 39--46.

[16]

Muja, M., and Lowe, D. G. 2009. Fast approximate nearest neighbors with automatic algorithm configuration. In International Conference on Computer Vision Theory and Application, 331--340.

[17]

Rhemann, C., Hosni, A., Bleyer, M., Rother, C., and Gelautz, M. 2011. Fast cost-volume filtering for visual correspondence and beyond. In CVPR, IEEE, 3017--3024.

Digital Library

[18]

Rother, C., Kolmogorov, V., and Blake, A. 2004. Grabcut: Interactive foreground extraction using iterated graph cuts. TOG 23, 3, 309--314.

Digital Library

[19]

Saxena, A., Chung, S. H., and Ng, A. Y. 2005. Learning depth from single monocular images. In NIPS, 1--8.

[20]

Saxena, A., Sun, M., and Ng, A. 2009. Make3d: Learning 3d scene structure from a single still image. TPAMI 31, 5, 824--840.

Digital Library

[21]

Scharstein, D., and Szeliski, R. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV 47, 1--3, 7--42.

Digital Library

[22]

Schechner, Y. Y., and Kiryati, N. 2000. Depth from defocus vs. stereo: How different really are they? IJCV 39, 2, 141--162.

Digital Library

[23]

Shi, J., Xu, L., and Jia, J. 2015. Just noticeable defocus blur detection and estimation. In CVPR, 1--8.

[24]

Su, H., Huang, Q., Mitra, N. J., Li, Y., and Guibas, L. 2014. Estimating image depth using shape collections. TOG 33, 4, 37.

Digital Library

[25]

Subbarao, M., and Surya, G. 1994. Depth from defocus: a spatial domain approach. IJCV 13, 3, 271--294.

Digital Library

[26]

Tai, Y.-W., and Brown, M. S. 2009. Single image defocus map estimation using local contrast prior. In ICIP, 1797--1800.

Digital Library

[27]

Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., and Tumblin, J. 2007. Dappled photography: Mask enhanced cameras for heterodyned light fields and coded aperture refocusing. TOG 26, 3, 69.

Digital Library

[28]

Watanabe, M., and Nayar, S. K. 1998. Rational filters for passive depth from defocus. IJCV 27, 3, 203--225.

Digital Library

[29]

Wu, T.-P., Sun, J., Tang, C.-K., and Shum, H.-Y. 2008. Interactive normal reconstruction from a single image. TOG 27, 5, 119.

Digital Library

[30]

Xu, L., Yan, Q., and Jia, J. 2013. A sparse control model for image and video editing. TOG 32, 6, 197.

Digital Library

[31]

Zhou, C., and Nayar, S. 2009. What are good apertures for defocus deblurring? In ICCP, 1--8.

[32]

Zhou, C., Lin, S., and Nayar, S. 2009. Coded aperture pairs for depth from defocus. In ICCV, 325--332.

[33]

Zhu, X., Cohen, S., Schiller, S., and Milanfar, P. 2013. Estimating spatially varying defocus blur from a single image. TIP 22, 12, 4879--4891.

Digital Library

[34]

Zhuo, S., and Sim, T. 2011. Defocus map estimation from a single image. Pattern Recognition 44, 9, 1852--1858.

Digital Library

[35]

Ziou, D., and Deschênes, F. 2001. Depth from defocus estimation in spatial domain. CVIU 81, 2, 143--165.

Digital Library

Cited By

Li DJabbireddy SZhang YMetzler CVarshney A(2024)Instant-SFH: Non-Iterative Sparse Fourier Holograms Using Perlin NoiseSensors10.3390/s2422735824:22(7358)Online publication date: 18-Nov-2024
https://doi.org/10.3390/s24227358
Zhu XLin MYi MZhao H(2024)Photorealistic attention style transfer network for architectural photography photosScientific Reports10.1038/s41598-024-81249-614:1Online publication date: 28-Nov-2024
https://doi.org/10.1038/s41598-024-81249-6
Haji-Esmaeili MMontazer G(2024)Large-scale Monocular Depth Estimation in the WildEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107189127:PAOnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.engappai.2023.107189
Show More Cited By

Index Terms

Break Ames room illusion: depth from general single images
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
        Scene understanding
  2. Computer graphics
    1. Image manipulation

Recommendations

A blur estimation and detection method for out-of-focus images

With the analysis of the features of image edge based on the defocused model of optical imaging system, a blur estimation and detection method for out-of-focus images is proposed. The essential idea is to estimate the parameter of the point spread ...
Half-sweep imaging for depth from defocus

Depth from defocus (DFD) is a technique that restores scene depth based on the amount of defocus blur in the images. DFD usually captures two differently focused images, one near-focused and the other far-focused, and calculates the size of the defocus ...
Multiple depth layers and all-in-focus image generations by blurring and deblurring operations

The depth map and all-in-focus generations from a single image are proposed.The depth map bases on the characteristic curve of COC vs. the depth characteristic curve of a camera.All-in-focus image generation is from the estimated depth map.A joint ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 34, Issue 6

November 2015

944 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2816795

Issue’s Table of Contents

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 November 2015

Published in TOG Volume 34, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Research Grants Council, University Grants Committee, Hong Kong

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

42
Total Citations
View Citations
1,686
Total Downloads

Downloads (Last 12 months)62
Downloads (Last 6 weeks)4

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li DJabbireddy SZhang YMetzler CVarshney A(2024)Instant-SFH: Non-Iterative Sparse Fourier Holograms Using Perlin NoiseSensors10.3390/s2422735824:22(7358)Online publication date: 18-Nov-2024
https://doi.org/10.3390/s24227358
Zhu XLin MYi MZhao H(2024)Photorealistic attention style transfer network for architectural photography photosScientific Reports10.1038/s41598-024-81249-614:1Online publication date: 28-Nov-2024
https://doi.org/10.1038/s41598-024-81249-6
Haji-Esmaeili MMontazer G(2024)Large-scale Monocular Depth Estimation in the WildEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107189127:PAOnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.engappai.2023.107189
Fujimura YIiyama MFunatomi TMukaigawa Y(2024)Deep Depth from Focal Stack with Defocus Model for Camera-Setting InvarianceInternational Journal of Computer Vision10.1007/s11263-023-01964-x132:6(1970-1985)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s11263-023-01964-x
Phongthawee PWizadwongsa SYenphraphai JSuwajanakorn S(2023)NeX360: Real-Time All-Around View Synthesis With Neural Basis ExpansionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2022.321795745:6(7611-7624)Online publication date: 5-May-2023
https://dl.acm.org/doi/10.1109/TPAMI.2022.3217957
Basar SAli MWaheed AAhmad MMiraz M(2023)A Novel Defocus-Blur Region Detection Approach Based on DCT Feature and PCNN StructureIEEE Access10.1109/ACCESS.2023.330982011(94945-94961)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3309820
Luo XPeng JXian KWu ZCao Z(2023)Defocus to focus: Photo-realistic bokeh rendering by fusing defocus and radiance priorsInformation Fusion10.1016/j.inffus.2022.08.02389(320-335)Online publication date: Jan-2023
https://doi.org/10.1016/j.inffus.2022.08.023
Polasek TČadík MKeller YBenes B(2023)Vision UFormer: Long-range monocular absolute depth estimationComputers & Graphics10.1016/j.cag.2023.02.003111(180-189)Online publication date: Apr-2023
https://doi.org/10.1016/j.cag.2023.02.003
Liang ZFang THu YWang Y(2023)Sparse depth densification for monocular depth estimationMultimedia Tools and Applications10.1007/s11042-023-15757-483:5(14821-14838)Online publication date: 11-Jul-2023
https://doi.org/10.1007/s11042-023-15757-4
Rey-Area MYuan MRichardt C(2022)360MonoDepth: High-Resolution 360° Monocular Depth Estimation2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52688.2022.00374(3752-3762)Online publication date: Jun-2022
https://doi.org/10.1109/CVPR52688.2022.00374
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents