Abstract
Plenoptic cameras are a new type of sensors that capture the four-dimensional lightfield of a scene. Processing the recorded lightfield, these cameras extend the capabilities of current commercial cameras offering the possibility of focusing the scene after the shot or obtaining 3D information. Conventional photographs focused on certain planes can be obtained through projections of the four-dimensional lightfield onto two spatial dimensions. These photographs can be efficiently computed using the Fourier Slice technique, but their resolution is limited since a plenoptic camera trades off spatial resolution for angular resolution. In order to remove this limitation, several super-resolution methods have been recently developed to increase the spatial resolution of plenoptic cameras. In this paper, we study the super-resolution problem in plenoptic cameras and show how to efficiently compute super-resolved photographs using the Fourier Slice technique. We also show how several existing super-resolution methods can be seen as particular cases of this approach. Experimental results are provided to show the validity of the approach and its extension to super-resolved all-in-focus image computation and 3D processing is studied.











Similar content being viewed by others
References
Adelson, E.H., Bergen, J.R.: The plenoptic function and the elements of early vision. In: Landy, M., Movshon, J.A. (eds.) In Computation Models of Visual Processing, pp. 3–20. MIT Press, Cambridge (1991)
Levoy, M., Hanrahan, P.: “Light field rendering”. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, pp. 31–42 (1996)
Gortler, S.J., Grzeszczuk, R., Szelinsli, R., Cohen, M.F.: The Lumigraph. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (1996)
Ives, F.: US Patent 725567 (1903)
Lipmannn, G.: Epreuves reversibles donnant la sensation du relief. J. Phys. 7(4), 821–825 (1908)
Adelson, T., Wang, J.: Single lens stereo with a plenoptic camera. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 99–106 (1992)
Okano, F., Arai, J., Hoshino, H., Yuyama, I.: Three dimensional video system based on integral photography. Opt. Eng. 38(6), 1072–1077 (1999)
Naemura, T., Yoshida, T., Harashima, H.: 3-D computer graphics based on integral photography. Opt. Express 38(6), 255–262 (2001)
Tyson, R.K.: Principles of Adaptive Optics. Academic, New York (1991)
Wilburn, B., Joshi, N., Vaish, V., Talvala, E., Antunez, E.: High performance imaging using large camera arrays. Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (1995)
Ng, R.: Fourier slice photography. ACM Trans. Graph 24, 735–744 (2005)
Lytro: www.lytro.com
Raytrix: www.raytrix.com
Pérez, F., Marichal, J.G., Rodríguez, J.M.: The discrete focal stack transform. In: Proceedings of Eusipco 08 (2008)
Lumsdaine, A., Georgiev, T.: The focused plenoptic camera. In: International Conference on Computational Photography (2009)
Georgiev, T., Chunev, G., Lumsdaine, A.: « Superresolution with the focused plenoptic camera, » In: Proceedings of SPIE 7873, Computational Imaging IX, (2011)
Pérez, F., Lüke, J.P.: Simultaneous estimation of super-resolved depth and all-in-focus images from a plenoptic camera. In: 3D Television (2009)
Bishop, T.E., Zanetti, S., Favaro, P.: Light field superresolution. In: International Conference on Computational Photography (2009)
Stroebel, L., Compton, J., Current, I., Zakia, R.: Photographic Materials and Processes. Focal, Boston (1996)
Kaiser, J. F.: Nonrecursive digital filter design using I0-sinh window function. In: IEEE International Symposiun on Circuits and Systems (1974)
Beerends, R.J., Morsche, H.G., Berg, J.C., Vrie, E.M.: Fourier and Laplace Transforms. Cambridge University Press, Cambridge (2003)
Bailey, D.H., Swarztrauber, P.N.: The fractional Fourier transform and applications. SIAM Rev. 33, 389–404 (1991)
Georgiev, T., Lumsdaine, A.: Reducing plenoptic camera artifacts. Comput. Graph. Forum 29(6), 1955–1968 (2010)
Ng, R.: « Digital light field photography. »PhD Thesis Stanford University (2006)
Pérez, F., Pérez, A., Rodríguez, M., Magdaleno, E.: « Fourier Slice Super-resolution in plenoptic cameras, »In: International Conference on Computational Photography, Seattle (2012)
Park, S.C., Park, M.K., Kang, M.G.: Super-resolution image reconstruction- a technical overview. IEEE Signal Process. Mag. 20(3), 21–36 (2003)
Farsiu, S., Robinson, D., Elad, M., Mylanfar, P.: Advances and challenges in super-resolution. Int. J. Imaging Syst. Technol. 14(2), 47–57 (2004)
Borman, S., Stevenson, R.: Super-resolution from image sequences—a review. In: Midwest Symposium on Circuits and Systems (1998)
Elad, M., Hel-Or, Y.: A fast super-resolution algorithm for pure translational motion and common space-invariant blur. IEEE Trans. Image Process. 10(8), 1187–1193 (2001)
Hansen, C.: Discrete Inverse Problems: Insight and Algorithms (Fundamentals of Algorithms). SIAM, Philadelphia (2010)
van der Shaaf, A., van Hateren, J.H.: « Modelling the power spectra of natural images: statistics and information ». Vision Res. 36(17), 2759–2770 (1996)
Tan, L.: Digital Signal Processing. Academic Press, New York (2007)
Dansereau, G., Pizarro, O., Williams, S.B.: « Decoding, calibration and rectification for lenselet-based plenoptic cameras, » In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)
Wang, Z., Bovik, A., Sheik, H., Simoncelli, E.: « Image quality assessment: from error visibility to structural similarity ». IEEE Trans. Image Process. 13(4), 600–612 (2004)
Agarwala, A., Dontcheva, M., Agrawala, M., Drucker S., Colburn, A., Curless, B., Salesin, D., Cohen, D.: Interactive digital photomontage. In: SIGGRAPH (2004)
Vaquero, D., Gelfand, N., Tico, M., Kulli, K., Turk, M.: Generalized autofocus. In: IEEE Workshop on Applications of Computer Vision (2011)
Nayar, K., Nakagawa, Y.: « Shape from focus, ». IEEE Trans. Pattern Anal. Mach. Intell. \(\text{ n }^\circ 16\). 16, 824–831 (1994)
Acknowledgments
The authors would like to thank R. Ng, T. Georgiev and Heidelberg University for lightfields that were used in the experimental results. This work has been partially supported by “Ayudas al Fomento de Nuevos Proyectos de Investigación” (Project 2013/0001339) of the University of La Laguna
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Pérez, F., Pérez, A., Rodríguez, M. et al. Super-Resolved Fourier-Slice Refocusing in Plenoptic Cameras. J Math Imaging Vis 52, 200–217 (2015). https://doi.org/10.1007/s10851-014-0540-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10851-014-0540-1