skip to main content
10.1145/3304109.3306228acmconferencesArticle/Chapter ViewAbstractPublication PagesmmsysConference Proceedingsconference-collections
research-article

Adaptive two-layer light field compression scheme based on sparse reconstruction

Published: 18 June 2019 Publication History

Abstract

As a new form of volumetric media, the technology of light field and its compression has gradually become the research hotspots in academia. The scheme of compressing using the sparsity of the light field is a very promising idea, which has the characteristics of high compression rate and is not affected by the occlusion of scene objects. However, the instability of the reconstruction algorithm's performance on different datasets limits the further application of this solution. Since the quality of the decompression outputs will be limited below the reconstruction result, the poor performance of the reconstruction algorithm on some light field images will result in a very low PSNR upper limit for the compression scheme. This paper finds that the main reason for this performance problem is the poor ability of the algorithm to process the high-frequency components of the light field. And in order to solve it, an adaptive two-layer light field compression scheme is presented. The proposed scheme separates the high-frequency components and the low-frequency components of the light field so that they can be independently compressed. Through the adaptive adjustment, the data of different frequency component can adopt different compression strategies, so that the performance of the proposed scheme can be optimal. Experiments with multiple datasets1 show that the proposed scheme can break the upper limit of PSNR caused by sparse reconstruction and is capable to provide decompression results above 40 dB. It also achieves significant improvement in compression efficiency under diverse requirements.

References

[1]
Amar Aggoun. 2011. Compression of 3D integral images using 3D wavelet transform. Journal of Display Technology 7, 11 (2011), 586--592.
[2]
H. Amirpour, M. Pereira, and A. Pinheiro. 2018. High Efficient Snake Order Pseudo-Sequence Based Light Field Image Compression. In 2018 Data Compression Conference. 397--397.
[3]
Stanford University Computer Graphics Laboratory. 2008. Light Field Datasets. http://lightfield.stanford.edu/lfs.html.
[4]
Caroline Conti, Paulo Nunes, and Luís Ducla Soares. 2013. New HEVC prediction modes for 3D holoscopic video coding. In IEEE International Conference on Image Processing. 1325--1328.
[5]
Caroline Conti, Paulo Nunes, and Luis Ducla Soares. 2016. HEVC-based light field image coding with bi-predicted self-similarity compensation. In IEEE International Conference on Multimedia & Expo Workshops. 1--4.
[6]
Caroline Conti, Luís Ducla Soares, and Paulo Nunes. 2016. HEVC-based 3D holoscopic video coding using self-similarity compensated prediction. Signal Processing Image Communication 42 (2016), 59--78.
[7]
D. G. Dansereau, O. Pizarro, and S. B. Williams. 2013. Decoding, Calibration and Rectification for Lenselet-Based Plenoptic Cameras. In 2013 IEEE Conference on Computer Vision and Pattern Recognition. 1027--1034.
[8]
Bichuan Guo, Yuxing Han, and Jiangtao Wen. 2018. Convex Optimization Based Bit Allocation for Light Field Compression under Weighting and Consistency Constraints. (2018).
[9]
Harini Priyadarshini Hariharan, Tobias Lange, and Thorsten Herfet. 2017. Low complexity light field compression based on pseudo-temporal circular sequencing. In IEEE International Symposium on Broadband Multimedia Systems and Broadcasting. 1--5.
[10]
Fatma Hawary, Christine Guillemot, Dominique Thoreau, and Guillaume Boisson. 2017. Scalable light field compression scheme using sparse reconstruction and restoration. In Image Processing (ICIP), 2017 IEEE International Conference on. IEEE, 3250--3254.
[11]
Xinjue Hu. 2019. Simulation projects on Github. https://github.com/hxj121314/Two-Layer_LF_Compression.git.
[12]
Xiaoran Jiang, Mikaël Le Pendu, Reuben A. Farrugia, and Christine Guillemot. 2017. Light Field Compression with Homography-based Low Rank Approximation. IEEE Journal of Selected Topics in Signal Processing PP, 99 (2017), 1--1.
[13]
Shinjini Kundu. 2012. Light field compression using homography and 2D warping. In IEEE International Conference on Acoustics, Speech and Signal Processing. 1349--1352.
[14]
Li Li, Zhu Li, Bin Li, Dong Liu, and Houqiang Li. 2017. Pseudo-Sequence-Based 2-D Hierarchical Coding Structure for Light-Field Image Compression. IEEE Journal of Selected Topics in Signal Processing 11, 7 (2017), 1107--1119.
[15]
Yun Li, Marten Sjostrom, Roger Olsson, and Ulf Jennehag. 2014. Efficient intra prediction scheme for light field image compression. In IEEE International Conference on Acoustics, Speech and Signal Processing. 539--543.
[16]
Chia-Kai Liang, Yi-Chang Shih, and Homer H Chen. 2011. Light field analysis for modeling image formation. IEEE Transactions on Image Processing 20, 2 (2011), 446--460.
[17]
Xing Lin, Jiamin Wu, Guoan Zheng, and Qionghai Dai. 2015. Camera array based light field microscopy. Biomedical optics express 6, 9 (2015), 3179--3189.
[18]
Dong Liu, Lizhi Wang, Li Li, Zhiwei Xiong, Feng Wu, and Wenjun Zeng. 2016. Pseudo-sequence-based light field image compression. In IEEE International Conference on Multimedia and Expo Workshops. 1--4.
[19]
Luís F. R. Lucas, Caroline Conti, Paulo Nunes, Luís Ducla Soares, Nuno M. M. Rodrigues, Carla L. Pagliari, Eduardo A. B. Da Silva, and Sérgio M. M. De Faria. 2014. Locally linear embedding-based prediction for 3D holoscopic image coding using HEVC. In Signal Processing Conference. 11--15.
[20]
Ricardo Monteiro, Luis Lucas, Caroline Conti, Paulo Nunes, Nuno Rodrigues, Sergio Faria, Carla Pagliari, Eduardo Da Silva, and Luis Soares. 2016. Light field HEVC-based image coding using locally linear embedding and self-similarity compensated prediction. In IEEE International Conference on Multimedia and Expo Workshops. 1--4.
[21]
C. Perra and P. Assuncao. 2016. High efficiency coding of light field images based on tiling and pseudo-temporal data arrangement. In IEEE International Conference on Multimedia and Expo Workshops. 1--4.
[22]
Cristian Perra and Daniele Giusto. 2017. JPEG 2000 compression of unfocused light field images based on lenslet array slicing. In IEEE International Conference on Consumer Electronics. 27--28.
[23]
Lixin Shi, Haitham Hassanieh, Abe Davis, Dina Katabi, and Fredo Durand. 2014. Light field reconstruction using sparsity in the continuous fourier domain. ACM Transactions on Graphics (TOG) 34, 1 (2014), 12.
[24]
Kartik Venkataraman, Dan Lelescu, Jacques Duparré, Andrew McMahon, Gabriel Molina, Priyam Chatterjee, Robert Mullis, and Shree Nayar. 2013. Picam: An ultra-thin high performance monolithic camera array. ACM Transactions on Graphics (TOG) 32, 6 (2013), 166.
[25]
Irene Viola, Hermina Petric Maretic, Pascal Frossard, and Touradj Ebrahimi. 2018. A graph learning approach for light field image compression. Applications of Digital Image Processing XLI Spie-Int Soc Optical Engineering (2018), 12.
[26]
Gaochang Wu, Belen Masia, Adrian Jarabo, Yuchen Zhang, Liangyong Wang, Qionghai Dai, Tianyou Chai, and Yebin Liu. 2017. Light field image processing: An overview. IEEE Journal of Selected Topics in Signal Processing 11, 7 (2017), 926--954.
[27]
Shan Xu, Zhi-Liang Zhou, and Nicholas Devaney. 2014. Multi-view image restoration from plenoptic raw images. In Asian Conference on Computer Vision. Springer, 3--15.
[28]
Wei Zhang, Dong Liu, Zhiwei Xiong, and Jizheng Xu. 2018. SIFT-based adaptive prediction structure for light field compression. In Visual Communications and Image Processing. 1--4.
[29]
Xiang Zhang, Philip A. Chou, Ming Ting Sun, Maolong Tang, Shanshe Wang, Siwei Ma, and Wen Gao. 2018. Surface Light Field Compression using a Point Cloud Codec. (2018).

Cited By

View all
  • (2023)Multiple Description Coding for Best-Effort Delivery of Light Field Video Using GNN-Based CompressionIEEE Transactions on Multimedia10.1109/TMM.2021.312991825(690-705)Online publication date: 1-Jan-2023
  • (2022)Geometry-guided compact compression for light field image using graph convolutional networksProceedings of the 32nd Workshop on Network and Operating Systems Support for Digital Audio and Video10.1145/3534088.3534345(15-21)Online publication date: 17-Jun-2022
  • (2022)An Untrained Neural Network Prior for Light Field CompressionIEEE Transactions on Image Processing10.1109/TIP.2022.321737431(6922-6936)Online publication date: 2022
  • Show More Cited By

Index Terms

  1. Adaptive two-layer light field compression scheme based on sparse reconstruction

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      MMSys '19: Proceedings of the 10th ACM Multimedia Systems Conference
      June 2019
      374 pages
      ISBN:9781450362979
      DOI:10.1145/3304109
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      In-Cooperation

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 18 June 2019

      Permissions

      Request permissions for this article.

      Check for updates

      Badges

      Author Tags

      1. compression
      2. light field
      3. optimization
      4. reconstruction

      Qualifiers

      • Research-article

      Conference

      MMSys '19
      Sponsor:
      MMSys '19: 10th ACM Multimedia Systems Conference
      June 18 - 21, 2019
      Massachusetts, Amherst

      Acceptance Rates

      MMSys '19 Paper Acceptance Rate 40 of 82 submissions, 49%;
      Overall Acceptance Rate 176 of 530 submissions, 33%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)16
      • Downloads (Last 6 weeks)2
      Reflects downloads up to 14 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2023)Multiple Description Coding for Best-Effort Delivery of Light Field Video Using GNN-Based CompressionIEEE Transactions on Multimedia10.1109/TMM.2021.312991825(690-705)Online publication date: 1-Jan-2023
      • (2022)Geometry-guided compact compression for light field image using graph convolutional networksProceedings of the 32nd Workshop on Network and Operating Systems Support for Digital Audio and Video10.1145/3534088.3534345(15-21)Online publication date: 17-Jun-2022
      • (2022)An Untrained Neural Network Prior for Light Field CompressionIEEE Transactions on Image Processing10.1109/TIP.2022.321737431(6922-6936)Online publication date: 2022
      • (2021)Angular‐spatial analysis of factors affecting the performance of light field reconstructionIET Image Processing10.1049/ipr2.1220316:4(1027-1035)Online publication date: 4-Apr-2021
      • (2020)QoE-Fair DASH Video Streaming Using Server-side Reinforcement LearningACM Transactions on Multimedia Computing, Communications, and Applications10.1145/339722716:2s(1-21)Online publication date: 21-Jun-2020
      • (2020)A Practical Learning-based Approach for Viewer Scheduling in the Crowdsourced Live StreamingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/339722616:2s(1-22)Online publication date: 21-Jun-2020
      • (2020)An Adaptive Two-Layer Light Field Compression Scheme Using GNN-Based ReconstructionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/339562016:2s(1-23)Online publication date: 21-Jun-2020
      • (2020)RequetACM Transactions on Multimedia Computing, Communications, and Applications10.1145/339449816:2s(1-28)Online publication date: 10-Jul-2020
      • (2020)The Impact of Motion and Delay on Selecting Game Targets with a MouseACM Transactions on Multimedia Computing, Communications, and Applications10.1145/339046416:2s(1-24)Online publication date: 21-Jun-2020
      • (2020)Evaluation of Shared Resource Allocation Using SAND for ABR StreamingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/338892616:2s(1-18)Online publication date: 10-Jul-2020
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media