skip to main content
10.1145/2964284.2967260acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
short-paper

Multiview Video Super-Resolution via Information Extraction and Merging

Published: 01 October 2016 Publication History

Abstract

Multiview video super-resolution provides a promising solution to the contradiction between the huge data size of multiview video and the degraded video quality due to mixed-resolution compression. This algorithm consists of two different functional layers. An information extraction layer draws relevant high-frequency information from the high-resolution views via depth-image-based rendering and interpolation. A merging layer fuses multiview high-frequency information to refine the low-resolution view. In this paper, we introduce kernel regression and non-local means to improve the two layers, respectively. Kernel regression adapts to the local image structure and thus outperforms basic interpolation methods. Non-local means exploits the similarity between different views of multiview videos to restore the high-frequency component of a low-resolution image. We constrain non-local means by limiting the pixels used to restore a pixel. The experimental results show the effectiveness of the proposed algorithm.

References

[1]
C. Ancuti, C. O. Ancuti, and P. Bekaert. Video super-resolution using high quality photographs. In Proc. ICASSP, pages 862 -- 865, 2010.
[2]
F. Brandi, R. d. Queiroz, and D. Mukherjee. Super-resolution of video using key frames and motion estimation. In Proc. ICIP, pages 321 -- 324, 2010.
[3]
C. Fehn. Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV. Proc. SPIE, 5291:93 -- 104, 2004.
[4]
W. T. Freeman, T. R. Jones, and E. C. Pasztor. Example-based super-resolution. IEEE Comput. Graph. Appl., 22(2):56 -- 65, 2001.
[5]
Z. Fu, Y. Li, J. Xu, H. Wu, and Y. Lai. Super resolution for multiview mixed resolution images in transform-domain with optimal weight. Multimed. Tools Appl., pages 1--15, 2016.
[6]
D. C. Garcia, C. Dorea, and R. L. de Queiroz. Super resolution for multiview images using depth information. IEEE Trans. Circuits Syst. Video Technol., 22(9):1249 -- 1256, 2012.
[7]
B. Goldluecke and D. Cremers. Superresolution texture maps for multiview reconstruction. In Proc. CVPR, pages 1677 -- 1684, 2009.
[8]
T. Hiroyuki, F. Sina, and M. Peyman. Kernel regression for image processing and reconstruction. IEEE Trans. Image Process., 16(2):349 -- 366, 2007.
[9]
T. S. Huang and R. Y. Tsai. Multi-frame image restoration and registration. Adv. Comput. Vis. Image Process., 1:317 -- 339, 1984.
[10]
Z. Jin, T. Tillo, C. Yao, and J. Xiao. Virtual view assisted video super-resolution and enhancement. IEEE Trans. Circuits Syst. Video Technol., PP(99):1, 2015.
[11]
P. Matan, E. Michael, T. Hiroyuki, and M. Peyman. Generalizing the nonlocal-means to super-resolution reconstruction. IEEE Trans. Image Process., 18(1):36 -- 51, 2009.
[12]
T. Richter, J. Seiler, W. Schnurrer, and A. Kaup. Robust super-resolution for mixed-resolution multiview image plus depth data. IEEE Trans. Circuits Syst. Video Technol., PP(99):1, 2015.
[13]
H. Su, L. Tang, Y. Wu, D. Tretter, and J. Zhou. Spatially adaptive block-based super-resolution. IEEE Trans. Image Process., 21(3):1031 -- 1045, 2012.
[14]
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process., 13(4):600 -- 612, 2004.
[15]
H. Zhang, J. Yang, Y. Zhang, and T. S. Huang. Image and video restorations via nonlocal kernel regression. IEEE Trans. Cybern., 43(3):1035 -- 1046, 2013.
[16]
K. Zhang, X. Gao, D. Tao, and X. Li. Single image super-resolution with non-local means and steering kernel regression. IEEE Trans. Image Process., 21(11):4544 -- 4556, 2012.
[17]
C. L. Zitnick, S. B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski. High-quality video view interpolation using a layered representation. SIGGRAPH, 23(3):600 -- 608, 2004.

Cited By

View all
  • (2022)Geometry-Aware Reference Synthesis for Multi-View Image Super-ResolutionProceedings of the 30th ACM International Conference on Multimedia10.1145/3503161.3547947(6083-6093)Online publication date: 10-Oct-2022
  • (2022)Video super-resolution based on deep learning: a comprehensive surveyArtificial Intelligence Review10.1007/s10462-022-10147-y55:8(5981-6035)Online publication date: 1-Dec-2022
  • (2021)Low-Rank Constrained Super-Resolution for Mixed-Resolution Multiview VideoIEEE Transactions on Image Processing10.1109/TIP.2020.304206430(1072-1085)Online publication date: 2021
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MM '16: Proceedings of the 24th ACM international conference on Multimedia
October 2016
1542 pages
ISBN:9781450336031
DOI:10.1145/2964284
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 October 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. kernel regression
  2. multiview video processing
  3. non-local means
  4. super-resolution

Qualifiers

  • Short-paper

Conference

MM '16
Sponsor:
MM '16: ACM Multimedia Conference
October 15 - 19, 2016
Amsterdam, The Netherlands

Acceptance Rates

MM '16 Paper Acceptance Rate 52 of 237 submissions, 22%;
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)21
  • Downloads (Last 6 weeks)0
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Geometry-Aware Reference Synthesis for Multi-View Image Super-ResolutionProceedings of the 30th ACM International Conference on Multimedia10.1145/3503161.3547947(6083-6093)Online publication date: 10-Oct-2022
  • (2022)Video super-resolution based on deep learning: a comprehensive surveyArtificial Intelligence Review10.1007/s10462-022-10147-y55:8(5981-6035)Online publication date: 1-Dec-2022
  • (2021)Low-Rank Constrained Super-Resolution for Mixed-Resolution Multiview VideoIEEE Transactions on Image Processing10.1109/TIP.2020.304206430(1072-1085)Online publication date: 2021
  • (2019)3D Appearance Super-Resolution With Deep Learning2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR.2019.00990(9663-9672)Online publication date: Jun-2019
  • (2017)Beyond Human-level License Plate Super-resolution with Progressive Vehicle Search and Domain Priori GANProceedings of the 25th ACM international conference on Multimedia10.1145/3123266.3123422(1618-1626)Online publication date: 23-Oct-2017

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media