Skip to main content
Log in

A new tile boundary artifact removal method for tile-based viewport-adaptive streaming in 360 videos

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In recent years, 360 video is becoming popular with the development of the virtual reality technology. However, the 360 video streaming faces many challenging issues since such video contains a large volume of data and requires streaming omnidirectional view. The Tile-based viewport-adaptive streaming method is used to deliver 360 videos more efficiently. In this method, a set of high and low-resolution tiles, corresponding to the viewport and non-viewport parts of the scene, is generated. These tiles are encoded and decoded independently using the Motion Constrained Tile Set (MCTS) technique. However, coding and decoding each tile independently, may create artifacts around the tile boundaries. In this paper, a novel method, called Resizing, is proposed to reduce the tile boundary artifact in tile-based viewport-adaptive streaming. In this method, the adjacent tiles are overlapped before encoding. Then, the overlapping tiles are downsampled to keep the resolution of video constant. This way, the mixed-resolution packing of 6K equirectangular (ERP) content, defined in MPEG Omnidirectional Media Format (OMAF) standard Annex D, can be used so that the output bitstream complies with the 4K-decoding constraint and the High Efficiency Video Coding (HEVC) standard. Finally, the video is upsampled after decoding and the overlapping parts are removed. Then, a novel method to remove the overlapping areas at the decoder side is proposed that could increase the perceived quality considerably. Simulation results indicate that our proposed approach achieves a 14.14% reduction in bandwidth in the Bjøntegaard-Bitrate scale, compared to the state-of-the-art methods that use variants of filtering for tile boundary artifact removal.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Bjøntegaard G (2001) Calculation of average PSNR differences between R-D curves. ITU T SG16/Q6, Doc. VCEG-M33

  2. Blumenberg C, Palomino D, Bampi S, Zatt B (2013) Adaptive content-based Tile partitioning algorithm for the HEVC standard. In: Picture coding symposium (PCS), San Jose, CA, SA. https://doi.org/10.1109/PCS.2013.6737714

  3. Boyce J, Alshina E, Abbas A, Ye Y (2017) JVET common test conditions and evaluation procedures for 360 video. In: Joint video exploration team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 8th Meeting: Macao

  4. Cho S, Kim H, Kim HY, Kim M (2015) Efficient in-loop filtering across tile boundaries for multi-core HEVC hardware decoders with 4 K/8 K-UHD video applications. IEEE Trans Multimed 17(6):778–791. https://doi.org/10.1109/TMM.2015.2418995

    Article  Google Scholar 

  5. de la Fuente YS, Bhullar GS, Skupin R, Hellge C, Schierl T (2019) Delay impact on MPEG OMAF’s tile-based viewport-dependent 360 video streaming. IEEE J Emerg Select Topics Circ Syst 9 (1):18–28. https://doi.org/10.1109/JETCAS.2019.2899516

    Article  Google Scholar 

  6. Gankhuyag G, Jeong J, Kim Y-H (2019) Advanced motion-constrained av1 encoder for 8K 360 VR tiled streaming. In: 2019 International conference on information and communication technology convergence (ICTC), Korea (South). https://doi.org/10.1109/ICTC46691.2019.8939730

  7. Hannuksela M, Wang YK, Hourunranta A (2019) An overview of the OMAF standard for 360 video. https://doi.org/10.1109/DCC.2019.00050

  8. High Efficiency Video Coding (HEVC) reference software, HM. 2020. Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute. https://hevc.hhi.fraunhofer.de/

  9. https://ffmpeg.org/ffmpeg-resampler.html, Accessed on January 2021

  10. Sauer J, Wien M, Schneider J, Bläser M (2018) Geometry-corrected deblocking filter for 360 video coding using cube representation. In: 2018 Picture coding symposium (PCS), San Francisco, CA, USA

  11. Shafi R, Shuai W, Younus MU (2020) 360-degree video streaming: a survey of the state of the art. Symmetry 12(9):1491. https://doi.org/10.3390/sym12091491

    Article  Google Scholar 

  12. Son J, Jang D, Ryu E-S (2018) Implementing motion-constrained tile and viewport extraction for VR streaming. In: NOSSDAV ’18 proceedings of the 28th ACM SIGMM workshop on network and operating systems support for digital audio and video, Amsterdam, Netherlands. https://doi.org/10.1145/3210445.3210455, pp 61–66

  13. Sun Y, Wang B, Yu L (2019) Adaptive QP with Tile partition and padding to remove boundary artifacts for 360 video coding. In: 2019 Picture coding symposium (PCS), Ningbo, China. https://doi.org/10.1109/PCS48520.2019.8954549

  14. Sze V, Budagavi M (2014) High efficiency video coding (HEVC): algorithms and architectures. Springer, Cham

    Book  Google Scholar 

  15. Wang Y-K (2019) Omnidirectional media format (OMAF). ISO/IEC JTC1/SC 29/WG 11 MPEG. ISO/IEC 23090–2

  16. You Y, Hourunranta A, Aksu E (2019) OMAF4CLOUD: standard-enabled 360 video creation as a service. In: IBC 2019. Amsterdam, Netherlands

  17. Zare A, Aminlou A, Hannuksela M, Gabbouj M (2016) HEVC-compliant Tile-based streaming of panoramic video for virtual reality applications. In: MM ’16 Proceedings of the 24th ACM international conference on multimedia, Amsterdam, Netherlands. https://doi.org/10.1145/2964284.2967292, pp 601–605

  18. Zare A, Aminlou A, Hannuksela M (2018) 6K effective resolution with 4K HEVC decoding capability for OMAF-compliant 360 video streaming. In: Proceedings of the 23rd packet video workshop. Amsterdam, Netherland. https://doi.org/10.1145/3210424.3210425, pp 72–77

  19. Zou W, Zhang W, Yang F (2021) Modeling the perceptual quality for viewport-adaptive omnidirectional video streaming considering dynamic quality boundary artifact. IEEE Trans Circ Syst Vid Technol

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hoda Roodaki.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahrar, A.M., Roodaki, H. A new tile boundary artifact removal method for tile-based viewport-adaptive streaming in 360 videos. Multimed Tools Appl 80, 29785–29803 (2021). https://doi.org/10.1007/s11042-021-11173-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11173-8

Keywords

Navigation