Abstract
In recent years, 360∘ video is becoming popular with the development of the virtual reality technology. However, the 360∘ video streaming faces many challenging issues since such video contains a large volume of data and requires streaming omnidirectional view. The Tile-based viewport-adaptive streaming method is used to deliver 360∘ videos more efficiently. In this method, a set of high and low-resolution tiles, corresponding to the viewport and non-viewport parts of the scene, is generated. These tiles are encoded and decoded independently using the Motion Constrained Tile Set (MCTS) technique. However, coding and decoding each tile independently, may create artifacts around the tile boundaries. In this paper, a novel method, called Resizing, is proposed to reduce the tile boundary artifact in tile-based viewport-adaptive streaming. In this method, the adjacent tiles are overlapped before encoding. Then, the overlapping tiles are downsampled to keep the resolution of video constant. This way, the mixed-resolution packing of 6K equirectangular (ERP) content, defined in MPEG Omnidirectional Media Format (OMAF) standard Annex D, can be used so that the output bitstream complies with the 4K-decoding constraint and the High Efficiency Video Coding (HEVC) standard. Finally, the video is upsampled after decoding and the overlapping parts are removed. Then, a novel method to remove the overlapping areas at the decoder side is proposed that could increase the perceived quality considerably. Simulation results indicate that our proposed approach achieves a 14.14% reduction in bandwidth in the Bjøntegaard-Bitrate scale, compared to the state-of-the-art methods that use variants of filtering for tile boundary artifact removal.
Similar content being viewed by others
References
Bjøntegaard G (2001) Calculation of average PSNR differences between R-D curves. ITU T SG16/Q6, Doc. VCEG-M33
Blumenberg C, Palomino D, Bampi S, Zatt B (2013) Adaptive content-based Tile partitioning algorithm for the HEVC standard. In: Picture coding symposium (PCS), San Jose, CA, SA. https://doi.org/10.1109/PCS.2013.6737714
Boyce J, Alshina E, Abbas A, Ye Y (2017) JVET common test conditions and evaluation procedures for 360∘ video. In: Joint video exploration team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 8th Meeting: Macao
Cho S, Kim H, Kim HY, Kim M (2015) Efficient in-loop filtering across tile boundaries for multi-core HEVC hardware decoders with 4 K/8 K-UHD video applications. IEEE Trans Multimed 17(6):778–791. https://doi.org/10.1109/TMM.2015.2418995
de la Fuente YS, Bhullar GS, Skupin R, Hellge C, Schierl T (2019) Delay impact on MPEG OMAF’s tile-based viewport-dependent 360∘ video streaming. IEEE J Emerg Select Topics Circ Syst 9 (1):18–28. https://doi.org/10.1109/JETCAS.2019.2899516
Gankhuyag G, Jeong J, Kim Y-H (2019) Advanced motion-constrained av1 encoder for 8K 360 VR tiled streaming. In: 2019 International conference on information and communication technology convergence (ICTC), Korea (South). https://doi.org/10.1109/ICTC46691.2019.8939730
Hannuksela M, Wang YK, Hourunranta A (2019) An overview of the OMAF standard for 360∘ video. https://doi.org/10.1109/DCC.2019.00050
High Efficiency Video Coding (HEVC) reference software, HM. 2020. Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute. https://hevc.hhi.fraunhofer.de/
https://ffmpeg.org/ffmpeg-resampler.html, Accessed on January 2021
Sauer J, Wien M, Schneider J, Bläser M (2018) Geometry-corrected deblocking filter for 360∘ video coding using cube representation. In: 2018 Picture coding symposium (PCS), San Francisco, CA, USA
Shafi R, Shuai W, Younus MU (2020) 360-degree video streaming: a survey of the state of the art. Symmetry 12(9):1491. https://doi.org/10.3390/sym12091491
Son J, Jang D, Ryu E-S (2018) Implementing motion-constrained tile and viewport extraction for VR streaming. In: NOSSDAV ’18 proceedings of the 28th ACM SIGMM workshop on network and operating systems support for digital audio and video, Amsterdam, Netherlands. https://doi.org/10.1145/3210445.3210455, pp 61–66
Sun Y, Wang B, Yu L (2019) Adaptive QP with Tile partition and padding to remove boundary artifacts for 360 video coding. In: 2019 Picture coding symposium (PCS), Ningbo, China. https://doi.org/10.1109/PCS48520.2019.8954549
Sze V, Budagavi M (2014) High efficiency video coding (HEVC): algorithms and architectures. Springer, Cham
Wang Y-K (2019) Omnidirectional media format (OMAF). ISO/IEC JTC1/SC 29/WG 11 MPEG. ISO/IEC 23090–2
You Y, Hourunranta A, Aksu E (2019) OMAF4CLOUD: standard-enabled 360∘ video creation as a service. In: IBC 2019. Amsterdam, Netherlands
Zare A, Aminlou A, Hannuksela M, Gabbouj M (2016) HEVC-compliant Tile-based streaming of panoramic video for virtual reality applications. In: MM ’16 Proceedings of the 24th ACM international conference on multimedia, Amsterdam, Netherlands. https://doi.org/10.1145/2964284.2967292, pp 601–605
Zare A, Aminlou A, Hannuksela M (2018) 6K effective resolution with 4K HEVC decoding capability for OMAF-compliant 360∘ video streaming. In: Proceedings of the 23rd packet video workshop. Amsterdam, Netherland. https://doi.org/10.1145/3210424.3210425, pp 72–77
Zou W, Zhang W, Yang F (2021) Modeling the perceptual quality for viewport-adaptive omnidirectional video streaming considering dynamic quality boundary artifact. IEEE Trans Circ Syst Vid Technol
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Ahrar, A.M., Roodaki, H. A new tile boundary artifact removal method for tile-based viewport-adaptive streaming in 360∘ videos. Multimed Tools Appl 80, 29785–29803 (2021). https://doi.org/10.1007/s11042-021-11173-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-021-11173-8