Skip to main content
SpringerLink
Log in

Assessing the quality of experience in viewing rendered decompressed light fields

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

A light field image is a sampling of the intensity and direction information of the light rays crossing the main lens of a digital light field camera. The light field information captured by a camera must be processed in order to render images of the scene to the final user at a given focal plane or viewpoint. In the area of light field imaging, efficient representation, processing, compression and quality evaluation techniques and methodologies are currently under research in order to foster the development of novel industrial, entertainment or scientific applications. This paper focuses on the problem of evaluating the quality of experience when viewing rendered decompressed light field images. A processing chain for coding and decoding a light field image is first defined as reference model. Then, a novel metric for quality evaluation of the rendered views is proposed. This metric measures the variation of structural similarity on a set of viewpoints extracted from the light field. Subjective evaluation is performed and the correlation between objective metrics and the subjective results is reported and discussed. The proposed objective quality metric resulted in a nearly strong correlation with the subjective assessment results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Adelson EH, Bergen JR (1991) The plenoptic function and the elements of early vision. In: Landy MS, Movshon AJ (eds) Computational models of visual processing. MIT Press, Cambridge, pp 3–20

  2. Bjontegaard G (2001) Calcuation of average PSNR differences between RD-curves. Doc. VCEG-m33 ITU-T Q6/16, Austin, TX, USA, 2–4 April 2001

  3. Chan SC, Ng KT, Gan ZF, Chan KL, Shum HY (2004) The plenoptic videos: capturing, rendering and compression. In: Proceedings of the 2004 international symposium on circuits and systems, 2004. ISCAS’04

  4. Chang CL, Zhu X, Ramanathan P, Girod B (2006) Light field compression using disparity-compensated lifting and shape adaptation. IEEE Trans Image Process 15(4):793–806

    Article  Google Scholar 

  5. Cho D, Lee M, Kim S, Tai YW (2013) Modeling the calibration pipeline of the lytro camera for high quality light-field image reconstruction. In: IEEE international conference on computer vision (ICCV), 2013, pp 3280–3287. https://doi.org/10.1109/ICCV.2013.407

  6. Condat L, Forster-Heinlein B, Van De Ville D (2007) H2O: reversible hexagonal-orthogonal grid conversion by 1-D filtering. In: IEEE international conference on image processing, 2007. ICIP 2007, vol 2, pp II–73–II–76. https://doi.org/10.1109/ICIP.2007.4379095

  7. Conti C, Nunes P, Soares L (2013) Inter-layer prediction scheme for scalable 3-d holoscopic video coding. IEEE Signal Process Lett 20(8):819–822. https://doi.org/10.1109/LSP.2013.2267234

    Article  Google Scholar 

  8. Conti C, Soares LD, Nunes P (2016) HEVC-Based 3D holoscopic video coding using self-similarity compensated prediction. Signal Process Image Commun 42:59–78

    Article  Google Scholar 

  9. Cui J, Liu Y, Xu Y, Zhao H, Zha H (2013) Tracking generic human motion via fusion of low-and high-dimensional approaches. IEEE Trans Syst Man Cybern Syst 43(4):996–1002

    Article  Google Scholar 

  10. Dansereau D, Pizarro O, Williams S (2013) Decoding, calibration and rectification for lenselet-based plenoptic cameras. In: IEEE conference on computer vision and pattern recognition (CVPR), 2013. https://doi.org/10.1109/CVPR.2013.137

  11. Dansereau DG, Pizarro O, Williams SB (2013) Decoding, calibration and rectification for lenselet-based plenoptic cameras. Tech. rep., Australian Centre for Field Robotics; School of Aerospace, Mechanical and Mechatronic Engineering University of Sydney, NSW, Australia

  12. Dricot A, Jung J, Cagnazzo M, Pesquet B, Dufaux F (2015) Integral images compression scheme based on view extraction. In: 23rd European signal processing conference (EUSIPCO), 2015, pp 101–105. https://doi.org/10.1109/EUSIPCO.2015.7362353

  13. Dricot A, Jung J, Cagnazzo M, Pesquet B, Dufaux F, Kovács P, Adhikarla V (2015) Subjective evaluation of super multi-view compressed contents on high-end light-field 3D displays. Signal Process Image Commun. https://doi.org/10.1016/j.image.2015.04.012

  14. Gehrig N, Dragotti PL (2007) Distributed compression of multi-view images using a geometrical coding approach. In: IEEE international conference on image processing, 2007. ICIP 2007, vol 6

  15. Gehrig N, Dragotti PL (2009) Geometry-driven distributed compression of the plenoptic function: Performance bounds and constructive algorithms. IEEE Trans Image Process 18(3):457–470

    Article  MathSciNet  MATH  Google Scholar 

  16. Gelman A, Dragotti PL, Velisavljević V (2010) Multiview image compression using a layer-based representation. In: 17th IEEE international conference on image processing (ICIP), 2010, pp 493–496

  17. Gelman A, Dragotti PL, Velisavljević V (2011) Centralized and interactive compression of multiview images. In: SPIE optical engineering+ applications, pp 81,350J–81,350J

  18. Higa RS, Chavez RFL, Leite RB, Arthur R, Iano Y (2013) Plenoptic image compression comparison between JPEG, JPEG2000 and SPITH. Cyber Journals: JSAT 3(6):1–6

    Google Scholar 

  19. ITU-R BT2022 (2012) General viewing conditions for subjective assessment of quality of sdtv and hdtv television pictures on flat panel displays. International Telecommunication Union

  20. ITU-R BT.500-13 (2012) Methodology for the subjective assessment of the quality of television pictures. International Telecommunication Union

  21. Joint photographic experts group (JPEG). http://jpeg.org/items/20150320_pleno_summary.html. Accessed: 2016-08-09

  22. Levoy M, Hanrahan P (1996) Light field rendering. In: Proceedings of the 23rd annual conference on computer graphics and interactive techniques. ACM, pp 31–42

  23. Li Y, Sjöström M, Olsson R, Jennehag U (2016) Scalable coding of plenoptic images by using a sparse set and disparities. IEEE Trans Image Process 25 (1):80–91

    Article  MathSciNet  Google Scholar 

  24. Liu Y, Cui J, Zhao H, Zha H (2012) Fusion of low-and high-dimensional approaches by trackers sampling for generic human motion tracking. In: 21st international conference on pattern recognition (ICPR), 2012. IEEE, pp 898–901

  25. Liu Y, Nie L, Han L, Zhang L, Rosenblum DS (2015) Action2activity: Recognizing complex activities from sensor data. In: IJCAI, pp 1617–1623

  26. Liu Y, Nie L, Liu L, Rosenblum DS (2016) From action to activity: sensor-based activity recognition. Neurocomputing 181:108–115

    Article  Google Scholar 

  27. Liu L, Cheng L, Liu Y, Jia Y, Rosenblum DS (2016) Recognizing complex activities by a probabilistic interval-based model. In: AAAI, vol 30, pp 1266–1272

  28. Lumsdaine A, Georgiev T (2009) The focused plenoptic camera. In: IEEE international conference on computational photography (ICCP), 2009. https://doi.org/10.1109/ICCPHOT.2009.5559008

  29. Lytro Illum specifications. https://www.lytro.com/illum/specs/. Accessed: 2016-08-09

  30. Magnor M, Girod B (2000) Data compression for light-field rendering. IEEE Trans Circuits Syst Video Technol 10(3):338–343

    Article  Google Scholar 

  31. Magnor M, Girod B (2000) Model-based coding of multiviewpoint imagery. In: Visual communications and image processing 2000, pp 14–22

  32. Murgia F, Fernández SB, Perra C, Giusto D (2014) Unfocused plenoptic camera calibration. In: IEEE 22nd telecommunications forum. Belgrade, Serbia

  33. Murgia F, Giusto D, Perra C (2015) 3D reconstruction from plenoptic image. In: IEEE 23rd telecommunications forum. Belgrade, Serbia, pp 448–451

  34. Murgia F, Perra C, Giusto D (2016) 3D point cloud reconstruction from single plenoptic image. Telfor Journal 8(1):26–31

    Article  Google Scholar 

  35. Ng R (2006) Digital light field photography. Ph.D. Thesis, Stanford University, Stanford, CA USA

  36. Ng R, Levoy M, Brédif M, Duval G, Horowitz M, Hanrahan P (2005) Light field photography with a hand-held plenoptic camera. Computer Science Technical Report CSTR 2(11):1–11

    Google Scholar 

  37. Perra C (2014) On the coding of plenoptic raw images. In: IEEE 22nd telecommunications forum. Belgrade, Serbia

  38. Perra C (2015) Lossless plenoptic image compression using adaptive block differential prediction. In: IEEE 40th international conference on acoustics, speech and signal processing. Brisbane, Australia

  39. Perra C (2016) Light field image compression based on preprocessing and high efficiency coding. In: IEEE 24th telecommunications forum. Belgrade, Serbia

  40. Perra C (2016) On the quality evaluation of lossy compressed light fields. In: IEEE 24th telecommunications forum. Belgrade, Serbia

  41. Perra C, Assuncao P (2016) High efficiency coding of light field images based on tiling and pseudo-temporal data arrangement. In: IEEE international conference on multimedia and expo (ICME). Seattle, USA

  42. Perra C, Giusto D (2017) JPEG 2000 Compression of unfocused light field images based on lenslet array slicing. In: IEEE international conference on consumer electronics. Las Vegas, USA

  43. Perra C, Giusto D (2017) Light field compression on sliced lenslet array. Intern J Internet Technol Secur Trans 8(1):1–14

    Google Scholar 

  44. Perra C, Giusto D (2017) Raw light field image compression of sliced lenslet array. In: 2017 IEEE international symposium on broadband multimedia systems and broadcasting (BMSB), pp 1–5. https://doi.org/10.1109/BMSB.2017.7986158

  45. Perra C, Murgia F, Giusto D (2016) An analysis of 3D point cloud reconstruction from light field images. In: International conference on image processing theory, tools and applications. Oulu, Finland

  46. Rerabek M, Yuan L, Authier LA, Ebrahimi T (2015) EPFL light-field image dataset. Tech. rep. ISO/IEC JTC 1/SC 29/WG1

  47. Schwiegerling J, Birch GC, Tyo JS (2012) Analysis and compression of plenoptic camera images with Zernike polynomials. In: Proc. of SPIE vol

  48. Skodras A, Christopoulos C, Ebrahimi T (2001) The JPEG 2000 still image compression standard. IEEE Signal Proc Mag 18(5):36–58. https://doi.org/10.1109/79.952804

    Article  MATH  Google Scholar 

  49. Sullivan GJ, Ohm JR, Han WJ, Wiegand T (2012) Overview of the high efficiency video coding (HEVC) standard. IEEE Trans Circuits Syst Video Technol 22 (12):1649–1668. https://doi.org/10.1109/TCSVT.2012.2221191

    Article  Google Scholar 

  50. Taguchi Y, Agrawal A, Ramalingam S, Veeraraghavan A (2010) Axial light field for curved mirrors: Reflect your perspective, widen your view. In: IEEE Conference on computer vision and pattern recognition (CVPR), 2010, pp 499–506. https://doi.org/10.1109/CVPR.2010.5540172

  51. Veeraraghavan A, Raskar R, Agrawal A, Mohan A, Tumblin J (2007) Dappled photography: Mask enhanced cameras for heterodyned light fields and coded aperture refocusing. ACM Trans. Graph 26(3). https://doi.org/10.1145/1276377.1276463

  52. Vieira A, Duarte H, Perra C, Tavora L, Assuncao P (2015) Data formats for high efficiency coding of lytro-illum light fields. In: International conference on image processing theory, tools and applications. Orléans, france

  53. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13 (4):600–612. https://doi.org/10.1109/TIP.2003.819861

    Article  Google Scholar 

  54. Wilburn B, Joshi N, Vaish V, Levoy M, Horowitz M (2004) High-speed videography using a dense camera array. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, 2004. CVPR 2004, vol 2, pp II–294–II–301. https://doi.org/10.1109/CVPR.2004.1315176

  55. Yu Z, Yu J, Lumsdaine A, Georgiev T (2012) An analysis of color demosaicing in plenoptic cameras. In: IEEE Conference on computer vision and pattern recognition (CVPR), 2012, pp 901–908. https://doi.org/10.1109/CVPR.2012.6247764

  56. Ziv J, Lempel A (1977) A universal algorithm for sequential data compression. IEEE Trans Inf Theory 23(3):337–343

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

The research activities described in this paper have been partially funded within the R&D project “DigitArch” (Top-down cluster action program, funded by the POR FESR Sardegna 2014/2020), the R&D project “Cagliari2020” (partially funded by the Italian University and Research Ministry, grant# MIUR_PON04a2_00381), and within the R&D project “CagliariPort2020” (partially funded by the MIUR, grant# SCN_00281).

Author information

Authors and Affiliations

  1. University of Cagliari, Cagliari, Italy

    Cristian Perra

Authors
  1. Cristian Perra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cristian Perra.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Perra, C. Assessing the quality of experience in viewing rendered decompressed light fields. Multimed Tools Appl 77, 21771–21790 (2018). https://doi.org/10.1007/s11042-018-5615-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-5615-3

Keywords

Search

Navigation