Skip to main content
SpringerLink
Log in
Book cover

Asian Conference on Computer Vision

ACCV 2016: Computer Vision – ACCV 2016 pp 3–18Cite as

Layered Scene Reconstruction from Multiple Light Field Camera Views

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10113))

Abstract

We propose a framework to infer complete geometry of a scene with strong reflections or hidden by partially transparent occluders from a set of 4D light fields captured with a hand-held light field camera. For this, we first introduce a variant of bundle adjustment specifically tailored to 4D light fields to obtain improved pose parameters. Geometry is recovered in a global framework based on convex optimization for a weighted minimal surface. To allow for non-Lambertian materials and semi-transparent occluders, the point-wise costs are not based on the principle of photo-consistency. Instead, we perform a layer analysis of the light field obtained by finding superimposed oriented patterns in epipolar plane image space to obtain a set of depth hypotheses and confidence scores, which are integrated into a single functional.

O. Johannsen and A. Sulc contributed equally.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Gortler, S., Grzeszczuk, R., Szeliski, R., Cohen, M.: The lumigraph. In: Proceedings of SIGGRAPH, pp. 43–54 (1996)

    Google Scholar 

  2. Bolles, R., Baker, H., Marimont, D.: Epipolar-plane image analysis: an approach to determining structure from motion. Int. J. Comput. Vis. 1, 7–55 (1987)

    Article  Google Scholar 

  3. Johannsen, O., Sulc, A., Goldluecke, B.: What sparse light field coding reveals about scene structure. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  4. Wanner, S., Goldluecke, B.: Reconstructing reflective and transparent surfaces from epipolar plane images. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 1–10. Springer, Heidelberg (2013). doi:10.1007/978-3-642-40602-7_1

    Chapter  Google Scholar 

  5. Johannsen, O., Sulc, A., Goldluecke, B.: On linear structure from motion for light field cameras. In: Proceedings of the International Conference on Computer Vision (2015)

    Google Scholar 

  6. Johannsen, O., Sulc, A., Goldluecke, B.: Variational separation of light field layers. In: Vision, Modelling and Visualization (VMV) (2015)

    Google Scholar 

  7. Wanner, S., Goldluecke, B.: Variational light field analysis for disparity estimation and super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. 36, 606–619 (2014)

    Article  Google Scholar 

  8. Chandraker, M., Reddy, D., Wang, Y., Ramamoorthi, R.: What object motion reveals about shape with unknown BRDF and lighting. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2013)

    Google Scholar 

  9. Tao, M., Wang, T.C., Malik, J., Ramamoorthi, R.: Depth estimation for glossy surfaces with light-field cameras. In: Proceedings of the European Conference on Computer Vision Workshops (2014)

    Google Scholar 

  10. Chen, C., Lin, H., Yu, Z., Kang, S.B., J., Y.: Light field stereo matching using bilateral statistics of surface cameras. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2014)

    Google Scholar 

  11. Stich, T., Tevs, A., Magnor, M.: Global depth from epipolar volumes - a general framework for reconstructing non-Lambertian surfaces. In: 3DPVT (2006)

    Google Scholar 

  12. Laurentini, A.: The visual hull concept for visual-based image understanding. IEEE Trans. Pattern Anal. Mach. Intell. 16, 150–162 (1994)

    Article  Google Scholar 

  13. Zuo, X., Du, C., Wang, S., Zheng, J., Yang, R.: Interactive visual hull refinement for specular and transparent object surface reconstruction. In: Proceedings of the International Conference on Computer Vision (2015)

    Google Scholar 

  14. Kutulakos, K.N., Seitz, S.M.: A theory of shape by space carving. Int. J. Comput. Vis. 38, 199–218 (2000)

    Article  MATH  Google Scholar 

  15. Yang, Y., Pollefeys, M., Welch, G.: Dealing with textureless regions and specular highlights - a progressive space carving scheme using a novel photo-consistency measure. In: Proceedings of the International Conference on Computer Vision (2003)

    Google Scholar 

  16. Faugeras, O., Keriven, R.: Variational principles, surface evolution, PDE’s, level set methods, and the stereo problem. IEEE Trans. Image Process. 7, 336–344 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  17. Jin, H., Soatto, S., Yezzi, A.: Multi-view stereo reconstruction of dense shape and complex appearance. Int. J. Comput. Vis. 63, 175–189 (2005)

    Article  Google Scholar 

  18. Yu, T., Narendra, A., Chen, W.C.: SDG cut: 3D reconstruction of non-Lambertian objects using graph cuts on surface distance grid. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2006)

    Google Scholar 

  19. Ihrke, I., Goldluecke, B., Magnor, M.: Reconstructing the geometry of flowing water. In: Proceedings of the International Conference on Computer Vision, pp. 1055–1060 (2005)

    Google Scholar 

  20. Goldman, D., Curless, B., Hertzmann, A., Seitz, S.: Shape and spatially-varying BRDFs from photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1060–1071 (2010)

    Article  Google Scholar 

  21. Vogiatzis, G., Hernandez, C., Cipolla, R.: Reconstruction in the round using photometric normals and silhouettes. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2006)

    Google Scholar 

  22. Kolev, K., Pock, T., Cremers, D.: Anisotropic minimal surfaces integrating photoconsistency and normal information for multiview stereo. In: Proceedings of the European Conference on Computer Vision (2010)

    Google Scholar 

  23. Cremers, D., Kolev, K.: Multiview stereo and silhouette consistency via convex functionals over convex domains. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1161–1174 (2011)

    Article  Google Scholar 

  24. Federer, H.: Geometric Measure Theory. Springer, Heidelberg (1969)

    MATH  Google Scholar 

  25. Zach, C., Pock, T., Bischof, H.: A globally optimal algorithm for robust TV-L1 range image integration. In: Proceedings of the International Conference on Computer Vision (2007)

    Google Scholar 

  26. Graber, G., Pock, T., Bischof, H.: Online 3D reconstruction using convex optimization. In: Proceedings of the International Conference on Computer Vision Workshops (2011)

    Google Scholar 

  27. Zickler, T., Belhumeur, P., Kriegman, D., Stereopsis, H.: Exploiting reciprocity for surface reconstruction. Int. J. Comput. Vis. 49, 215–227 (2002)

    Article  MATH  Google Scholar 

  28. Dansereau, D., Pizarro, O., Williams, S.: Decoding, calibration and rectification for lenselet-based plenoptic cameras. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 1027–1034 (2013)

    Google Scholar 

  29. Pock, T., Chambolle, A.: Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In: International Conference on Computer Vision (ICCV 2011) (2011)

    Google Scholar 

  30. Chan, T., Esedoglu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. SIAM J. Appl. Math. 66, 1632–1648 (2006)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgement

This work was supported by the ERC Starting Grant “Light Field Imaging and Analysis” (LIA 336978, FP7-2014).

Author information

Authors and Affiliations

  1. University of Konstanz, Konstanz, Germany

    Ole Johannsen, Antonin Sulc, Nico Marniok & Bastian Goldluecke

Authors
  1. Ole Johannsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonin Sulc
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nico Marniok
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bastian Goldluecke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Antonin Sulc .

Editor information

Editors and Affiliations

  1. National Tsing Hua University, Hsinchu, Taiwan

    Shang-Hong Lai

  2. Graz University of Technology, Graz, Austria

    Vincent Lepetit

  3. Drexel University, Philadelphia, Pennsylvania, USA

    Ko Nishino

  4. The University of Tokyo , Tokyo, Japan

    Yoichi Sato

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Johannsen, O., Sulc, A., Marniok, N., Goldluecke, B. (2017). Layered Scene Reconstruction from Multiple Light Field Camera Views. In: Lai, SH., Lepetit, V., Nishino, K., Sato, Y. (eds) Computer Vision – ACCV 2016. ACCV 2016. Lecture Notes in Computer Science(), vol 10113. Springer, Cham. https://doi.org/10.1007/978-3-319-54187-7_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-54187-7_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-54186-0

  • Online ISBN: 978-3-319-54187-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation