Skip to main content
SpringerLink
Log in

A spatiotemporal super-resolution algorithm for a hybrid stereo video system

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

This paper presents a spatiotemporal super-resolution method to enhance both the spatial resolution and the frame rate in a hybrid stereo video system. In this system, a scene is captured by two cameras to form two videos, including a low spatial resolution with high-frame-rate video and a high spatial resolution with low-frame-rate video. For the low-spatial-resolution video, the low-resolution frames are spatially super-resolved by the high-resolution video via the stereo matching, the bilateral overlapped block motion estimation, and the adaptive overlapped block motion compensation algorithms, while for the low-frame-rate video, those missed frames are interpolated using the high-resolution frames obtained by fusing the disparity compensation and the motion compensation frame rate up-conversion. Experimental results demonstrate that the proposed mixed spatiotemporal super-resolution method has a more significant contribution to both the subjective and objective qualities than the pure spatial super-resolution or the frame rate up-conversion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Stoykova, E., Alatan, A.A., Benzie, P., Grammalidis, N., Malassiotis, S., Ostermann, J., Piekh, S., Sainov, V., Theobalt, C., Thevar, T., Zabulis, X.: 3-D time-varying scene capture technologies—a survey. IEEE Trans. Circuits Syst. Video Technol. 17(11), 1568–1586 (2007)

    Article  Google Scholar 

  2. Ozkalayci, B., Gedik, S., Alatan, A.A.: Multi-view video coding via dense depth estimation. In: Proceedings of the 3DTV Conference, pp. 1–4. Kos Island (2007)

  3. Aflaki, P., Hannuksela, M.M., Gabbouj, M.: Subjective quality assessment of asymmetric stereoscopic 3D video. Signal Image Video Process. 9(2), 331–345 (2015)

    Article  Google Scholar 

  4. Tian, J., Ma, K.K.: A state-space super-resolution approach for video reconstruction. Signal Image Video Process. 3(3), 217–240 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  5. Nasrollahi, K., Moeslund, T.B.: Super-resolution: a comprehensive survey. Mach. Vis. Appl. 25(6), 1423–1468 (2014)

    Article  Google Scholar 

  6. Tian, J., Ma, K.K.: A survey on super-resolution imaging. Signal Image Video Proces. 5(3), 329–342 (2011)

    Article  MathSciNet  Google Scholar 

  7. Brandi, F., de Queiroz, R., Mukherjee, D.: Super resolution of video using key frames. In: Proceeding of International Symposium on Circuits and Systems, pp. 1608–1611. Seattle (2008)

  8. Brandi, F., de Queiroz, R., Mukherjee, D.: Super-resolution of video using key frames and motion estimation. In: Proceeding of International Conference on Image Processing, pp. 321–324. San Diego (2008)

  9. Song, B.C., Jeong, S.C., Choi, Y.: Key frame-based video super-resolution using bi-directional overlapped block motion compensation and trained dictionary. In: Proceeding of International Conference on Image Processing Theory Tools and Applications, pp. 181–186. Paris (2010)

  10. Song, B.C., Jeong, S.C., Choi, Y.: Video super-resolution algorithm using bi-directional overlapped block motion compensation and on-the-fly dictionary training. IEEE Trans. Circuits Syst. Video Technol. 21(3), 274–285 (2011)

    Article  Google Scholar 

  11. Ge, J., Liu, J., Ge, C., Yang, X.: A robust video super-resolution based on adaptive overlapped block motion compensation. In: Proceeding of International Conference on Signal-Image Technology and Internet-Based Systems, pp. 187–194. Kyoto (2013)

  12. Schierl, T., Narasimhan, S.: Transport and storage systems for 3-D video using MPEG-2 systems, RTP, and ISO file format. Proc. IEEE 99(4), 671–683 (2011)

    Article  Google Scholar 

  13. Vetro, A., Tourapis, A.M., Muller, K., Chen, T.: 3D-tv content storage and transmission. IEEE Trans. Broadcast. 57(2), 384–394 (2011)

    Article  Google Scholar 

  14. Lee, Y.L., Nguyen, T.: Method and architecture design for motion compensated frame interpolation in high-definition video processing. In: Proceeding of IEEE International Symposium on Circuits and Systems, pp. 1633–1636. Taipei (2009)

  15. Zhang, B., Liu, J., Ge, J., Chen, C., Yuan, H., Liu, W.: A super resolution reconstruction scheme for mixed spatio-temporal stereo video. In: Proceeding of International Conference on Audio, Language and Image Processing, pp. 490–496. Shanghai (2012)

  16. Le, H.V., Seetharaman, G.: A super-resolution imaging method based on dense subpixel-accurate motion fields. J. VLSI Signal Proces. Syst. Signal Image Video Technol. 42(1), 79–89 (2006)

    Article  Google Scholar 

  17. Hung, E.M., de Queiroz, R.L., Brandi, F., de Oliveira, K.F., Mukherjee, D.: Video super-resolution using codebooks derived from key-frames. IEEE Trans. Circuits Syst. Video Technol. 22(9), 1321–1331 (2012)

    Article  Google Scholar 

  18. Chen, Y.K., Vetro, A., Sun, H., Kung, S.Y.: Frame-rate up-conversion using transmitted true motion vectors. In: Proceeding of IEEE Second Workshop on Multimedia Signal Processing, pp. 622–627. Redondo Beach (1998)

  19. De Haan, G., Biezen, P.W.A.C., Huijgen, H., Ojo, O.A.: True-motion estimation with 3-d recursive search block matching. IEEE Trans. Circuits Syst. Video Technol. 3(5), 368–379 (1993)

    Article  Google Scholar 

  20. Liu, C.: Beyond pixels: exploring new representations and applications for motion analysis. Doctoral Thesis, Massachusetts Institute of Technology (2009)

  21. Gu, K., Zhai, G., Yang, X., Zhang, W., Liu, M.: Structural similarity weighting for image quality assessment. In: Proceeding of IEEE International Conference on Multimedia and Expo Workshops, pp. 1–6. San Jose (2013)

  22. Gu, K., Zhai, G., Yang, X., Zhang, W.: An efficient color image quality metric with local-tuned-global model. In: Proceeding of IEEE International Conference on Image Processing, pp. 506–510. Paris (2014)

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grants (61201211), in part by the Excellent Youth Scientist Award Foundation of Shandong Province under Grants BS(2012DX021), in part by the China Postdoctoral Science Foundation Funded Project under Grant (2013M530320), in part by the open research fund of Sciences Key Laboratory of Wireless Sensor Network and Communication under Grant (2013002) and in part by the Cultivation Fund the Key Scientific and Technical Innovation Project (708059).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Shandong University, Jinan, 250100, China

    Jing Ge, Ju Liu, Hui Yuan & Chuan Ge

  2. Hisense State Key Laboratory of Digital Multi-Media Technology, Qingdao, 266061, China

    Ju Liu

  3. Key Laboratory of Wireless Sensor Network and Communication, Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences, Shanghai, 200050, China

    Hui Yuan

  4. China Mobile Group Shandong Company Limited, Jinan, 250001, China

    Boyang Zhang

Authors
  1. Jing Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ju Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hui Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chuan Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Boyang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ju Liu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ge, J., Liu, J., Yuan, H. et al. A spatiotemporal super-resolution algorithm for a hybrid stereo video system. SIViP 10, 559–566 (2016). https://doi.org/10.1007/s11760-015-0774-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-015-0774-4

Keywords

Search

Navigation