Skip to main content
SpringerLink
Log in

Lightfield Recovery from Its Focal Stack

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

The Focal Stack Transform integrates a 4D lightfield over a set of appropriately chosen 2D planes. The result of such integration is an image focused on a determined depth in 3D space. The set of such images is the Focal Stack of the lightfield. This paper studies the existence of an inverse for this transform. Such inverse could be used to obtain a 4D lightfield from a set of images focused on several depths of the scene. In this paper, we show that this inversion cannot be obtained for a general lightfield and introduce a subset of lightfields where this inversion can be computed exactly. We examine the numerical properties of such inversion process for general lightfields and examine several regularization approaches to stabilize the transform. Experimental results are provided for focal stacks obtained from several plenoptic cameras. From a practical point of view, results show how this inversion procedure can be used to recover, compress, and denoise the original 4D lightfield.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Levoy, M.: Light fields and computational imaging. Computer 39(8), 46–55 (2006)

    Article  Google Scholar 

  2. Adelson, E.H., Bergen, J.R.: The Plenoptic Function and the Elements of Early Vision. MIT Press, Cambridge (1991)

    Google Scholar 

  3. Levoy, M., Hanrahan, P.: Light field rendering. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, New York, pp. 31–42 (1996)

  4. Gortler, S.J., Grzeszczuk, R., Szeliski, R., Cohen, M.F.: The Lumigraph. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, New York, pp. 43–54 (1996)

  5. Ng, R.: Fourier slice photography. In: ACM SIGGRAPH: Papers, New York, 2005, pp. 735–744 (2005)

  6. Nava, F.P., Marichal-Hernández, J.G., Rodríguez-Ramos, J.M.: The discrete focal stack transform. In: Proceedings of European Signal Processing Conference, pp. 1–5 (2008)

  7. Lumsdaine, A., Georgiev, T.: The focused plenoptic camera. In: IEEE International Conference on Computational Photography (ICCP), 2009, pp. 1–8 (2009)

  8. Lüke, J.P., Pérez Nava, F., Marichal-Hernández, J.G., Rodríguez-Ramos, J.M., Rosa, F.: Near real-time estimation of super-resolved depth and all-in-focus images from a plenoptic camera using graphics processing units. Int. J. Digit. Multimed. Broadcast. 2010, e942037 (2009)

    Google Scholar 

  9. Uliyar, M., Putraya, G., Basavaraja, S.V.: Fast EPI based depth for plenoptic cameras. In: 20th IEEE International Conference on Image Processing, pp. 1–4 (2013)

  10. Tosic, I., Berkner, K.: Light field scale-depth space transform for dense depth estimation. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 441–448 (2014)

  11. Pérez, F., Pérez, A., Rodríguez, M., Magdaleno, E.: Super-resolved Fourier-slice refocusing in plenoptic cameras. J. Math. Imaging Vis. 52(2), 200–217 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  12. Georgiev, T., Chunev, G., Lumsdaine, A.: Superresolution with the focused plenoptic camera. Presented at the Proceedings of SPIE 7873, Computational Imaging IX, vol. 7873, p. 78730X–78730X-13 (2011)

  13. Perez Nava, F.: Super-resolution in plenoptic cameras by the integration of depth from focus and stereo. In: Proceedings of 19th International Conference on Computer Communications and Networks, pp. 1–6 (2010)

  14. Pérez Nava, F., Pérez Nava, A., Rodríguez Valido, M., Magdaleno Castellò, E.: Plenoptic cameras. In: Cristóbal, G., Perrinet, L., Keil, M.S. (eds.) Biologically Inspired Computer Vision, pp. 175–200. Wiley-VCH Verlag GmbH & Co, KGaA, Weinheim (2015)

    Chapter  Google Scholar 

  15. “Lytro.” https://www.lytro.com (2014). Accessed 04 Jul 2014

  16. “Raytrix.” http://www.raytrix.de/ (2014). Accessed 04 Jul 2014

  17. Pérez, F., Pérez, A., Rodríguez, M., Magdaleno, E.: A fast and memory-efficient Discrete Focal Stack Transform for plenoptic sensors. Digit. Signal Process. 38, 95–105 (2015)

    Article  Google Scholar 

  18. Stroebel, L.D. (ed.): Basic Photographic Materials and Processes, 2nd edn. Focal Press, Boston (2000)

    Google Scholar 

  19. Ferreira, P.J.S.G., Superresolution, the recovery of missing samples, and Vandermonde matrices on the unit circle. In: Proceedings of Workshop Sampling Theory Applications (SAMPTA’99) (1999)

  20. Hansen, P.C.: Rank-deficient and discrete ill-posed problems: numerical aspects of linear inversion. SIAM, Philadelphia (1998)

    Book  Google Scholar 

  21. Bertero, M., Boccacci, P.: Introduction to inverse problems in imaging. Bristol, UK?. Institute of Physics Publication, Philadelphia (1998)

    Book  MATH  Google Scholar 

  22. Reichel, L., Rodriguez, G.: Old and new parameter choice rules for discrete ill-posed problems. Numer. Algorithms 63(1), 65–87 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  23. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. Nonlinear Phenom. 60(1–4), 259–268 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  24. Afonso, M.V., Bioucas-Dias, J.M., Figueiredo, M.A.T.: Fast image recovery using variable splitting and constrained optimization. IEEE Trans. Image Process. 19(9), 2345–2356 (2010)

    Article  MathSciNet  Google Scholar 

  25. Goldstein, T., Osher, S.: The split Bregman method for L1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  26. Yang, J., Yin, W., Zhang, Y., Wang, Y.: A fast algorithm for edge-preserving variational multichannel image restoration. SIAM J. Imaging Sci. 2(2), 569–592 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  27. Dansereau, D.G., Pizarro, O., Williams, S.B.: Decoding, calibration and rectification for lenselet-based plenoptic cameras. Presented at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013, pp. 1027–1034 (2013)

  28. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  29. Shafer, S.A.: Using color to separate reflection components. Color Res. Appl. 10(4), 210–218 (1985)

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank R. Ng and Heidelberg University for lightfields that were used in the experimental results. This work has been partially supported by “Ayudas al Fomento de Nuevos Proyectos de Investigación” (Project 2013/0001339) of the University of La Laguna.

Author information

Authors and Affiliations

  1. Departamento de Estadística, Investigación Operativa y Computación, University of La Laguna, San Cristóbal de La Laguna, Canary Islands, Spain

    F. Pérez & A. Pérez

  2. Departamento de Física Fundamental, Experimental, Electrónica y Sistemas, University of La Laguna, San Cristóbal de La Laguna, Canary Islands, Spain

    M. Rodríguez & E. Magdaleno

Authors
  1. F. Pérez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Pérez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Rodríguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. E. Magdaleno
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to F. Pérez.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (docx 34 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pérez, F., Pérez, A., Rodríguez, M. et al. Lightfield Recovery from Its Focal Stack. J Math Imaging Vis 56, 573–590 (2016). https://doi.org/10.1007/s10851-016-0658-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-016-0658-4

Keywords

Search

Navigation