Skip to main content
SpringerLink
Log in

Reconstruction of compressively sampled light field by using tensor dictionaries

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

How to capture the high quality light field photography was one of important issue in computational photography. In fact, light field could be captured directly for all views or compressively reconstructed for each view just through one coded image. The latter kind of method was more feasible since only one exposure was needed for all views, among which dictionary-based light field reconstruction had been shown its effectiveness. In this paper, a more effective light field reconstruction method based on tensor dictionary was created. The proposed method is efficient because the trained tensor form dictionary can make better use of the rich structure of light field. Specifically, multiple small dictionaries were trained at the same time, and then were combined to a big dictionary using Kronecker product. Experimental results demonstrate the proposed method outperforms a state-of-the-art reconstruction method with the vector-form dictionary, in terms of higher reconstruction PSNR while reducing the scale of dictionary substantially.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Adelson T, Wang J (1992) Single lens stereo with a plenoptic camera. IEEE Transactions on Pattern Analysis & Machine Intelligence 14(2):99–106

    Google Scholar 

  2. Aharon M, Elad M, Bruckstein A (2006) K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing 54(2):4311–4322

    MATH  Google Scholar 

  3. Amarlingam M, Mishra PK, Prasad KD et al (2016) Compressed sensing for different sensors: A real scenario for WSN and IoT. In: 2016 IEEE 3rd World Forum on Internet of Things (WF-IoT), pp.289-294, Reston, VA, USA

  4. Ashok A, Neifeld M (2010) Compressive light field imaging. In: The international society for optical engineering, pp, vol 7690Q. USA, Orlando, FL

  5. Babacan D, Ansorge R, Luessi M, et al. (2012) Compressive light field sensing. IEEE Transactions on Image Processing 21(12):4746–4757

    MathSciNet  MATH  Google Scholar 

  6. Buehler C, Bosse M, Mcmillan L et al (2001) Unstructured lumigraph rendering. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 425-432, ACM

  7. Caiafa CF, Cichocki A (2012) Block sparse representations of tensors using kronecker bases. In: IEEE International conference on acoustics, speech and signal processing. Kyoto, Japan, pp 2709–2712

  8. Candès E J (2006) Compressive sampling. In: proceedings of the International Congress of Mathematicians, 3, pp 1433–1452

  9. Cao X, Geng Z, Li T (2014) Dictionary-based light field acquisition using sparse camera array. Optics express 22(20):24081–24095

    Google Scholar 

  10. Chen Y, Zhao Q, Hu X, et al. (2019) Multi-resolution parallel magnetic resonance image reconstruction in mobile computing-based IoT. IEEE Access 7:15623–15633

    Google Scholar 

  11. Davis A, Levoy M, Durand F (2012) Unstructured light fields. Computer Graphics Forum 31(2.1):305–314

    Google Scholar 

  12. Fu Y, Gao J, Sun Y, et al. (2014) Joint multiple dictionary learning for Tensor sparse coding. In: International joint conference on neural networks. Beijing, China, pp 2957–2964

  13. Gan L (2007) Block compressed sensing of natural images. In: 15Th international conference on digital signal processing. Cardiff, UK, pp 403–406

  14. Georgeiv T, Zheng KC, Curless B, et al. (2006) Spatio-angular resolution tradeoff in integral photography. In: Proceedings of the Eurographics Symposium on Rendering. Nicosia, Cyprus, pp 263–272

  15. Gortler SJ, Grzeszczuk R, Szeliski R et al (1996) The lumigraph. In: Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pp. 43-54, ACM

  16. Kamal M, Golbabaee M, Andvandergheynst P (2012) Light field compressive sensing in camera arrays. In: IEEE International conference on acoustics, speech, and signal processing. Kyoto, Japan, pp 5413–5416

  17. Kim C, Zimmer H, Pritch Y, et al. (2013) Scene reconstruction from high spatio-angular resolution light fields. ACM Trans Graph 32(4):73:1-73:12

    MATH  Google Scholar 

  18. Kolda TG, Bader BW (2009) Tensor decompositions and applications. SIAM Review 51(3):455–500

    MathSciNet  MATH  Google Scholar 

  19. Levoy M, Hanrahan P (1996) Light field rendering. In: Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pp. 31-42, ACM

  20. Li Z, Huang H, Misra S (2016) Compressed sensing via dictionary learning and approximate message passing for multimedia Internet of Things. IEEE Internet of Things Journal 4(2):505–512

    Google Scholar 

  21. Liang C, Lin T, Wong B, et al. (2008) Programmable aperture photography: multiplexed light field acquisition. ACM Transactions on Graphic 27(3):1–10

    Google Scholar 

  22. Marwah K, Wetzstein G, Bando Y, et al. (2013) Compressive light field photography using overcomplete dictionaries and optimized projections. ACM Transactions on Graphic 32(4):1–12

    MATH  Google Scholar 

  23. Memos VA, Psannis KE, Ishibashi Y, et al. (2018) An efficient algorithm for media-based surveillance system EAMSus in IoT smart city framework. Futur Gener Comput Syst 83:619–628

    Google Scholar 

  24. Ng R, Levoy M, Bredif M, et al. (2005) Light field photography with a hand-held plenoptic camera. Stanford University Computer Science Tech Report CSTR 2(11):1–11

    Google Scholar 

  25. Plageras AP, Psannis KE, Stergiou C, et al. (2018) Efficient IoT-based sensor BIG Data collection–processing and analysis in smart buildings. Futur Gener Comput Syst 82:349–357

    Google Scholar 

  26. Psannis KE, Ishibashi Y (2016) Impact of video coding on delay and jitter in 3G wireless video multicast services. EURASIP Journal on Wireless Communications and Networking 2006(1):024–616

    Google Scholar 

  27. Qi N, Shi Y, Sun X, et al. (2017) Multi-dimensional sparse models. IEEE Transactions on Pattern Analysis & Machine Intelligence 40(1):163–178

    Google Scholar 

  28. Roemer F, Galdo GD, Haardt M (2014) Tensor-based algorithms for learning multidimensional separable dictionaries. In: IEEEInternational conference on acoustics, speech, and signal processing. Florence, Italy, pp 3963–3967

  29. Shi L, Hassanieh H, Davis A, et al. (2014) Light field reconstruction using sparsity in the continuous Fourier domain. ACM Transactions on Graphics 34(1):1–13

    Google Scholar 

  30. Shidanshidi H, Safaei F, Li W (2015) Estimation of signal distortion using effective sampling density for light field-based free viewpoint video. IEEE Transactions on Multimedia 17(10):1677–1693

    Google Scholar 

  31. Shon KW, Park IK (2016) Spatio-angular consistent editing framework for 4D light field images. Multimedia Tools and Applications 75(23):16615–16631

    Google Scholar 

  32. Stergiou C, Psannis KE, Kim BG, et al. (2018) Secure integration of IoT and cloud computing. Future Generation Computer Systems 78:964–975

    Google Scholar 

  33. Stergiou C, Psannis KE, Plageras AP, et al. (2018) Algorithms for efficient digital media transmission over IoT and cloud networking. Journal of Multimedia Information System 5(1):27–34

    Google Scholar 

  34. Vagharshakyan S, Bregovic R, Gotchev A (2018) Light field reconstruction using shearlet transform. IEEE Transactions on Pattern Analysis and Machine Intelligence 40(1):133–147

    Google Scholar 

  35. Veeraraghavan A, Raskar R, Agrawal A, et al. (2007) Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing. ACM Transactions on Graphic 26(3):1–12

    Google Scholar 

  36. Wang Y, Wang L, Kong D, et al. (2015) High-resolution light field capture with coded aperture. IEEE Transactions on Image Processing 24(12):5609–5618

    Google Scholar 

  37. Wilburn B, Joshi N, Vaish V, et al. (2005) High performance imaging using large camera arrays. ACM Transactions on Graphics 24(3):765–776

    Google Scholar 

  38. Xu Z, Ke J, Lam EY (2012) High-resolution lightfield photography using two masks. Optics Express 20(10):10971–10983

    Google Scholar 

  39. Yang JC, Everett M, Buehler C, et al. (2002) A real-time distributed light field camera. Pisa, Italy

    Google Scholar 

  40. Zhang C, Hou G, Zhang Z, et al. (2018) Efficient auto-refocusing for light field camera. Pattern Recognition 81:176–189

    Google Scholar 

  41. Zubair S, Wang W (2013) Tensor dictionary learning with sparse tucker decomposition. In: 18Th international conference on digital signal processing. Fira, Greece, pp 1–6

Download references

Acknowledgements

We would like to thank the referees and editors for their helpful comments and suggestions. Yuping Wang would like thank to research fund support in 2019 from Capital University of Economics and Business. Jungfei Zhang would like thank to the support from Natural Science Foundation of China (NSFC NO. 11871488), the foundation ”the Fundamental Research Funds for the Central Universities” (NO. QL18010) and School of Statistics and Mathematics of CUFE.

Author information

Authors and Affiliations

  1. Capital University of Economics and Business, School of Statistics, 121 Zhangjialukou, Huaxiang Fengtai District, Beijing, China, 100070

    Yuping Wang

  2. Central University of Finance and Economics, School of Statistics and Mathematics, 39 South College Road, Haidian District, Beijing, China, 100081

    Junfei Zhang

Authors
  1. Yuping Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junfei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Junfei Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Zhang, J. Reconstruction of compressively sampled light field by using tensor dictionaries. Multimed Tools Appl 79, 20449–20460 (2020). https://doi.org/10.1007/s11042-020-08903-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08903-9

Keywords

Search

Navigation