Skip to main content
SpringerLink
Log in

Super resolution for multiview mixed resolution images in transform-domain with optimal weight

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Mixed resolution framework has been applied in stereoscopic video coding system to reduce data-size and bit rate. Recently, Super Resolution (SR) problem in mixed resolution format has drawn a lot of attention in order to provide the users with high quality 3D visual experience. In this paper we present a novel SR scheme, which not only borrows the high frequency content from neighboring full resolution frame, but also makes use of all high resolution (HR) warped images to enhance low resolution (LR) view image. The correlation between the target LR view and auxiliary HR views determines the weight of these high frequency components, which will be merged into LR frame to construct the output HR image. The DCT is used for frequency decomposition and frequency integration. Experimental results show that the proposed method achieves higher performance in both subjective and objective evaluations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Ancuti C, Ancuti CO, Bekaert P (2010) Video super-resolution using high quality photographs. In: Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, pp 862–865

  2. Brandi F, De Queiroz R, Mukherjee D (2008) Super-resolution of video using key frames and motion estimation. In: Proceedings of the IEEE International Conference on Image Processing, pp 321–324

  3. Capel D, Zisserman A (2003) Computer vision applied to super-resolution. IEEE Signal Process Mag 20(3):75–86

    Article  Google Scholar 

  4. David C, Zisserman A (2001) Super-resolution from multiple views using learnt image models. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 627–634

  5. Duchon CE (1979) Lanczos filtering in one and two dimensions. J Appl Meteorol 18(8):1016–1022

    Article  Google Scholar 

  6. Freeman WT, Jones TR, Pasztor EC (2001) Example-based super-resolution. IEEE Comput Graph Appl 22(2):56–65

    Article  Google Scholar 

  7. Fehn C (2004) Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV. Proc SPIE-Int Soc Opt Eng 5291:93–104

    Google Scholar 

  8. Garcia DC, Dorea C, de Queiroz RL (2010) Super-resolution for multiview images using depth information. In: Proceedings of the International Conference on Image Processing, pp 1793–1796

  9. Garcia DC, Dorea C, de Queiroz RL (2012) Super resolution for multiview images using depth information. IEEE Trans Circuits Syst Video Technol 22(9):1249–1256

    Article  Google Scholar 

  10. Hung EM, Garcia DC (2011) Transform domian semi-super resolution. In: Proceedings of the International Conference on Image Processing, pp 1193–1196

  11. Hung EM, Dorea C, Garcia DC, de Queiroz RL (2011) Transform-domain super-resolution for multiview images using depth information. In: Proceedings of the European Signal Processing Conference, pp 398–401

  12. Lengyel R, Reza Soroushmehr SM, Shirani S (2014) Multi-view video super-resolution for hybrid cameras using modified NLM and adaptive thresholding. In: Proceedings of the IEEE International Conference on Image Processing, pp 5437 – 5441

  13. Merkle P, Müller K, Wiegand T (2010) 3D video: acquisition, coding, and display. IEEE Trans Consum Electron 56(2):946–950

    Article  Google Scholar 

  14. Müller K, Merkle P, Wiegand T (2011) 3-D video representation using depth maps. Proc IEEE 99(4):643–656

    Article  Google Scholar 

  15. Najafi S, Shirani S (2012) Regularization function for video super-resolution using auxiliary high resolution still images. In: Proceedings of the 2012 Conference Record of the Forty Sixth Asilomar Conference on Signals, Systems and Computers, pp 1713–1717

  16. Ndjiki-Nya P, Köppel M, Doshkov D, Lakshman H, Merkle P, Müller K, Wiegand T (2011) Depth image-based rendering with advanced texture synthesis for 3-D video. IEEE Trans Multimedia 13(3):453–465

    Article  Google Scholar 

  17. Richter T, Kaup A (2012) Multiview super-resolution using high-frequency synthesis in case of low-framerate depth information. In: Proceedings of the IEEE Visual Communications and Image Processing, pp 1–6

  18. Richter T, Seiler J, Schnurrer W, Kaup A (2012) Robust super-resolution in a multiview setup based on refined high-frequency synthesis. In: Proceedings of the 2012 IEEE 14th International Workshop on Multimedia Signal Processing, pp 7–12

  19. Richter T, Seiler J, Schnurrer W, Kaup A (2015) Robust super-resolution for mixed-resolution multiview image plus depth data. IEEE Transactions on Circuits and Systems for Video Technology

  20. Richter T, Jonscher M, Schnurrer W, Seiler J, Kaup A (2014) Reconstruction of multiview images taken with non-regular sampling sensors. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, pp 5789–5793

  21. Seiler J, Kaup A (2010) Complex-valued frequency selective extrapolation for fast image and video signal extrapolation. IEEE Signal Process Lett 17(11):949–952

    Article  Google Scholar 

  22. Takeda H, Milanfar P, Protter M, Elad M (2009) Super-resolution without explicit subpixel motion estimation. IEEE Trans Image Process 18(9):1958–1975

    Article  MathSciNet  MATH  Google Scholar 

  23. Tsai RY, Huang TS (1984) Multiframe image restoration and registration. In: Proceedings of the Advances in Computer Vision and Image Processing, vol 1, pp 317–339

  24. Vetro A, Tourapis AM, Muller K, Chen T (2011) 3D-TV content storage and transmission. IEEE Trans Broadcast 57(2):384–394

    Article  Google Scholar 

  25. Wu Z, Yu H, Chen CW (2010) A new hybrid DCT-Wiener-based interpolation scheme for video intra frame up-sampling. IEEE Signal Process Lett 17(10):827–830

    Article  Google Scholar 

  26. Zhang J, Cao Y, Wang Z (2013) A simultaneous method for 3D video super-resolution and high-quality depth estimation. In: Proceedings of the IEEE International Conference on Image Processing, pp 1346–1350

  27. Zitnick C, Kang S, Uyttendaele M, Winder S, Szeliski R (2004) High-quality video view interpolation using a layered representation. ACM Trans Graph 23(3):600–608

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank the Interactive Visual Media Group of Microsoft Research for providing the B a l l e t and B r e a k d a n c e r s data sets.

Author information

Authors and Affiliations

  1. School of Communication and Information Engineering, University of Electronic and Science Technology of China, Chengdu, Sichuan, 611731, China

    Zhizhong Fu, Yuan Li, Jin Xu & Yongwen Lai

  2. The Second Research Institute of CAAC, Chengdu, Sichuan, 610041, China

    Honggang Wu

Authors
  1. Zhizhong Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jin Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Honggang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yongwen Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhizhong Fu.

Additional information

This work was supported by the Natural Science Foundation of China(61075013) and Civil Aviation Administration of China(61139003).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fu, Z., Li, Y., Xu, J. et al. Super resolution for multiview mixed resolution images in transform-domain with optimal weight. Multimed Tools Appl 76, 3031–3045 (2017). https://doi.org/10.1007/s11042-016-3258-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3258-9

Keywords

Search

Navigation