Skip to main content
SpringerLink
Log in

Intra prediction based on geometry padding for omnidirectional video coding

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Omnidirectional videos or 360 degree videos play an important role in virtual reality (VR) applications. In order to employ the existing video coding standards, omnidirectional videos are firstly projected onto the 2-Dimension (2D) plane, which generates the discontinuity at boundaries and may result in unexpected artifacts when lossy coding is applied. In this paper, we propose a new intra prediction method to deal with the above coding artefacts in omnidirectional videos. Different from the conventional intra prediction using the left and top reference samples, the right reference samples are derived from the reconstructed samples on the left boundary and padded for intra prediction. The proposed method applies to the planar, DC and partial angular prediction modes in intra prediction. The experimental results demonstrate a Bjøntegaard-Delta-rate reduction of up to 2.97% using weighted spherical peak-signal to noise ratio (WSPSNR) quality metric for the coding tree units (CTUs) at the right boundary.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. 360lib-4.0, [Online] Available: https://jvet.hhi.fraunhofer.de/svn/svn 360Lib/tags/

  2. Albrecht M, Bartnik C, Bosse S et al. (2018) Description of sdr, hdr and 360 video coding technology proposal by fraunhoferhhi, JVET-J0014

  3. Alshina E, Boyce J, Abbas A, Ye Y (2016) Jvet common test conditions and software reference configurations, JVET-D1030

  4. Azevedo RGDA, Birkbeck N et al. (2019) Visual Distortions in 360-degree Videos, IEEE Transactions on Circuits and Systems for Video Technology pp.1–7

  5. Bordes P, Chen Y, Chevance C et al. (2018) Description of sdr, hdr and 360 video coding technology proposal by qualcommand technicolor medium complexity version, JVET-J0018

  6. Chen Y, Chien W-J, H.-C, et al. (2018) Description of sdr, hdr and 360 video coding technology proposal by qualcomm and technicolor low and high complexity versions, JVET-J0021

  7. Chen H, Chen J, Chernyak R et al. (2018) Description of sdr, hdr and 360 video coding technology proposal by huawei, gopro, hisilicon, and samsung general application scenario, JVET-J0025

  8. der Auwera GV, Seregin V, Said A, Karczewicz M (2018) Extension of simplified pdpc to diagonal intra modes, JVET-J0069

  9. Fu C, Wan L, Wong T, Leung C (2009) The rhombic dodecahedron map: an efficient scheme for encoding panoramic video. IEEE Transactions on Multimedia 11(4):634–644

    Article  Google Scholar 

  10. Gisle B (2001) Calculation of average psnr differences between rd-curves, ITU-T VCEG-M33

  11. Hanhart P, He Y-W, Ye Y (2016) Reference samples derivation using geometry padding for intra coding, JVET-D0092

  12. High efficiency video coding (hevc) reference software hm, [Online] Available: https://hevc.hhi.fraunhofer.de

  13. Hsu C-W, Chen C-Y, Chuang T-D et al. (2018) Description of sdr video coding technology proposal by mediatek, JVET-J0018

  14. Kamisli F (2013) Intra prediction based on markov process modeling of images. IEEE Trans Image Process 22(10):3916–3925

    Article  MathSciNet  Google Scholar 

  15. Kamisli F (2015) Block-based spatial prediction and transforms based on 2d markov processes for image and video compression. IEEE Trans Image Process 24(4):1247–1260

    Article  MathSciNet  Google Scholar 

  16. Kang J, Lee H, Lim S-C et al. (2018) Description of sdr video coding technology proposal by etri and sejong university, JVET-J0013

  17. Lee S, Kim S, Yip E, Choi B, Song J, Ko S (2017) Omnidirectional video coding using latitude adaptive down-sampling and pixel rearrangement. Electron Lett 53(10):655–657

    Article  Google Scholar 

  18. Li J, Wen Z, Li S, Zhao Y, Guo B, Wen J (2016) Novel tile segmentation scheme for omnidirectional video, 2016 IEEE International Conference on Image Processing (ICIP), pp. 370–374

  19. Li N, Wan S, Yang F (2018) Reference samples padding for intra-frame coding of omnidirectional video, 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, pp. 1987–1990

  20. Li X, Xu X, Zhao X et al. (2018) Description of sdr video coding technology proposal by tencent, JVET-J0029

  21. Lin P-H, Sung P, Lin C-C (2018) Multiple reference line intra prediction based on jem7.0, JVET-J0070

  22. Lin J et al (2019) Efficient projection and coding tools for 360 video. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 9(1):84–97

    Article  Google Scholar 

  23. Matsuda I, Ohtake Y, Mochizuki S, Fukai H, Itoh S (2013) Interpolative intra prediction by adapting processing order in block-based image coding, 2013 IEEE International Conference on Image Processing, pp. 1646–1650

  24. Matt Y, Haricharan L, Bernd G (2015) Content adaptive representations of omnidirectional videos for cinematic virtual reality, Proceedings of the 3rd International Workshop on Immersive Media Experiences, pp.1–6

  25. Misra K, Zhao J, Segall A et al. (2018) Description of sdr and hdr video coding technology proposal by sharp and foxconn, JVET-J0026

  26. Ng K-T, Chan S-C, Shum H-Y (2005) Data compression and transmission aspects of panoramic videos. IEEE Transactions on Circuits and Systems for Video Technology 15(1):82–95

    Article  Google Scholar 

  27. Ohm J, Sullivan GJ, Schwarz H, Tan TK, Wiegand T (2012) Comparison of the coding efficiency of video coding standard-sincluding high efficiency video coding (hevc). IEEE Transactions on Circuits and Systems for Video Technology 22(12):1669–1684

    Article  Google Scholar 

  28. Shiodera T, Tanizawa A, Chujoh T (2007) Block based extra/inter-polating prediction for intra coding, 2007 IEEE International Conference on Image Processing, pp. 445–448

  29. Snyder JP (1997) Flattening the earth: two thousand years of map projections, University of Chicago Press

  30. Sullivan GJ, Ohm J, Han W, Wiegand T (2012) Overview of the high efficiency video coding (HEVC) standard. IEEE Transactions on Circuits and Systems for Video Technology 22(10):1649–1668

    Article  Google Scholar 

  31. Sun Y, Yu L (2017) Coding optimization based on weighted-to-spherically-uniform quality metric for 360 video, 2017 IEEE Visual Communications and Image Processing (VCIP), pp. 1–4

  32. Suzuki T, Ikeda M, Sharman K et al. (2018) Description of sdr and hdr video coding technology proposal by sony, JVET-J0028

  33. Sze V, Budagavi M (2014) High efficiency video coding (HEVC) algorithms and architectures, Springer

  34. Tan TK, Weerakkody R, Mrak M, Ramzan N, Baroncini V, Ohm J, Sullivan GJ (2016) Video quality evaluation methodology and verification testing of hevc compression performance. IEEE Transactions on Circuits and Systems for Video Technology 26(1):76–90

    Article  Google Scholar 

  35. Tang M, Zhang Y, Wen J, Yang S (2017) Optimized video coding for omnidirectional videos, 2017 IEEE International Conference on Multimedia and Expo (ICME), pp. 799–804

  36. Toma T et al. (2018) Description of sdr video coding technology proposal by panasonic, JVET-J0020

  37. Wiegand T, Sullivan GJ, Bjontegaard G, Luthra A (2003) Overview of the h.264/avc video coding standard. IEEE Trans-actions on Circuits and Systems for Video Technology 13(7):560–576

    Article  Google Scholar 

  38. Xu M, Li C, Zhang S, Callet PL (2020) State-of-the-art in 360 video/image processing: perception. Assessment and Compression, IEEE Journal of Selected Topics in Signal Processing 14(1):5–26

    Article  Google Scholar 

  39. Ye Y et al. (2017) Algorithm description of projection format conversion in 360lib, JVET-E0084

  40. Youvalari RG, Aminlou A, Hannuksela MM (2016) Analysis of regional down-sampling methods for coding of omnidirectional video, 2016 Picture Coding Symposium (PCS), pp. 1–5

  41. Youvalari RG, Aminlou A, Hannuksela MM, Gabbouj M (2016) Efficient coding of 360-degree pseudo-cylindrical panoramic video for virtual reality applications, 2016 IEEE International Symposium on Multimedia (ISM), pp. 525–528

  42. Zhang X, Liu S, Lei S (2012) Intra mode coding in hevc standard, 2012 Visual Communications and Image Processing, pp. 1–6

  43. Zheng A, Au OC, Yuan Y, Yang H, Pang J, Ling Y (2014) Intra prediction with adaptive cu processing order in hevc, 2014 IEEE International Conference on Image Processing (ICIP), pp. 3724–3728

  44. Zheng A, Yuan Y, Zhou J, Guo Y, Yang H, Au OC (2016) Adaptive block coding order for intra prediction in hevc. IEEE Transactions on Circuits and Systems for Video Technology 26(11):2152–2158

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronics and Information, Northwestern Polytechnical University, Xi’an, China

    Ning Li & Shuai Wan

Authors
  1. Ning Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuai Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuai Wan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, N., Wan, S. Intra prediction based on geometry padding for omnidirectional video coding. Multimed Tools Appl 80, 3409–3424 (2021). https://doi.org/10.1007/s11042-020-09544-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09544-8

Keywords

Search

Navigation