Abstract
The Focal Stack Transform integrates a 4D lightfield over a set of appropriately chosen 2D planes. The result of such integration is an image focused on a determined depth in 3D space. The set of such images is the Focal Stack of the lightfield. This paper studies the existence of an inverse for this transform. Such inverse could be used to obtain a 4D lightfield from a set of images focused on several depths of the scene. In this paper, we show that this inversion cannot be obtained for a general lightfield and introduce a subset of lightfields where this inversion can be computed exactly. We examine the numerical properties of such inversion process for general lightfields and examine several regularization approaches to stabilize the transform. Experimental results are provided for focal stacks obtained from several plenoptic cameras. From a practical point of view, results show how this inversion procedure can be used to recover, compress, and denoise the original 4D lightfield.












Similar content being viewed by others
References
Levoy, M.: Light fields and computational imaging. Computer 39(8), 46–55 (2006)
Adelson, E.H., Bergen, J.R.: The Plenoptic Function and the Elements of Early Vision. MIT Press, Cambridge (1991)
Levoy, M., Hanrahan, P.: Light field rendering. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, New York, pp. 31–42 (1996)
Gortler, S.J., Grzeszczuk, R., Szeliski, R., Cohen, M.F.: The Lumigraph. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, New York, pp. 43–54 (1996)
Ng, R.: Fourier slice photography. In: ACM SIGGRAPH: Papers, New York, 2005, pp. 735–744 (2005)
Nava, F.P., Marichal-Hernández, J.G., Rodríguez-Ramos, J.M.: The discrete focal stack transform. In: Proceedings of European Signal Processing Conference, pp. 1–5 (2008)
Lumsdaine, A., Georgiev, T.: The focused plenoptic camera. In: IEEE International Conference on Computational Photography (ICCP), 2009, pp. 1–8 (2009)
Lüke, J.P., Pérez Nava, F., Marichal-Hernández, J.G., Rodríguez-Ramos, J.M., Rosa, F.: Near real-time estimation of super-resolved depth and all-in-focus images from a plenoptic camera using graphics processing units. Int. J. Digit. Multimed. Broadcast. 2010, e942037 (2009)
Uliyar, M., Putraya, G., Basavaraja, S.V.: Fast EPI based depth for plenoptic cameras. In: 20th IEEE International Conference on Image Processing, pp. 1–4 (2013)
Tosic, I., Berkner, K.: Light field scale-depth space transform for dense depth estimation. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 441–448 (2014)
Pérez, F., Pérez, A., Rodríguez, M., Magdaleno, E.: Super-resolved Fourier-slice refocusing in plenoptic cameras. J. Math. Imaging Vis. 52(2), 200–217 (2015)
Georgiev, T., Chunev, G., Lumsdaine, A.: Superresolution with the focused plenoptic camera. Presented at the Proceedings of SPIE 7873, Computational Imaging IX, vol. 7873, p. 78730X–78730X-13 (2011)
Perez Nava, F.: Super-resolution in plenoptic cameras by the integration of depth from focus and stereo. In: Proceedings of 19th International Conference on Computer Communications and Networks, pp. 1–6 (2010)
Pérez Nava, F., Pérez Nava, A., Rodríguez Valido, M., Magdaleno Castellò, E.: Plenoptic cameras. In: Cristóbal, G., Perrinet, L., Keil, M.S. (eds.) Biologically Inspired Computer Vision, pp. 175–200. Wiley-VCH Verlag GmbH & Co, KGaA, Weinheim (2015)
“Lytro.” https://www.lytro.com (2014). Accessed 04 Jul 2014
“Raytrix.” http://www.raytrix.de/ (2014). Accessed 04 Jul 2014
Pérez, F., Pérez, A., Rodríguez, M., Magdaleno, E.: A fast and memory-efficient Discrete Focal Stack Transform for plenoptic sensors. Digit. Signal Process. 38, 95–105 (2015)
Stroebel, L.D. (ed.): Basic Photographic Materials and Processes, 2nd edn. Focal Press, Boston (2000)
Ferreira, P.J.S.G., Superresolution, the recovery of missing samples, and Vandermonde matrices on the unit circle. In: Proceedings of Workshop Sampling Theory Applications (SAMPTA’99) (1999)
Hansen, P.C.: Rank-deficient and discrete ill-posed problems: numerical aspects of linear inversion. SIAM, Philadelphia (1998)
Bertero, M., Boccacci, P.: Introduction to inverse problems in imaging. Bristol, UK?. Institute of Physics Publication, Philadelphia (1998)
Reichel, L., Rodriguez, G.: Old and new parameter choice rules for discrete ill-posed problems. Numer. Algorithms 63(1), 65–87 (2013)
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. Nonlinear Phenom. 60(1–4), 259–268 (1992)
Afonso, M.V., Bioucas-Dias, J.M., Figueiredo, M.A.T.: Fast image recovery using variable splitting and constrained optimization. IEEE Trans. Image Process. 19(9), 2345–2356 (2010)
Goldstein, T., Osher, S.: The split Bregman method for L1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)
Yang, J., Yin, W., Zhang, Y., Wang, Y.: A fast algorithm for edge-preserving variational multichannel image restoration. SIAM J. Imaging Sci. 2(2), 569–592 (2009)
Dansereau, D.G., Pizarro, O., Williams, S.B.: Decoding, calibration and rectification for lenselet-based plenoptic cameras. Presented at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013, pp. 1027–1034 (2013)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Shafer, S.A.: Using color to separate reflection components. Color Res. Appl. 10(4), 210–218 (1985)
Acknowledgments
The authors would like to thank R. Ng and Heidelberg University for lightfields that were used in the experimental results. This work has been partially supported by “Ayudas al Fomento de Nuevos Proyectos de Investigación” (Project 2013/0001339) of the University of La Laguna.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Pérez, F., Pérez, A., Rodríguez, M. et al. Lightfield Recovery from Its Focal Stack. J Math Imaging Vis 56, 573–590 (2016). https://doi.org/10.1007/s10851-016-0658-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10851-016-0658-4