skip to main content
10.1145/3394171.3413550acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Tile Rate Allocation for 360-Degree Tiled Adaptive Video Streaming

Published: 12 October 2020 Publication History

Abstract

360-degree video streaming commonly encodes and transmits the video as independently-decodable tiles to conserve bandwidth of regions out of the viewer's field of view (FoV). The bitrate of the tiles, however, can vary significantly across the tiles, complicating the choice of the representation to download for each tile in each segment to adapt to the bandwidth dynamics. In this paper, we model the tile rate allocation problem as a multiclass knapsack problem with a dynamic profit function that is a function of the FoV and the buffer occupancy. Experiments show that our approach can reduce bandwidth wastage by up to 41%, the number of stalls by up to 31%, stall durations by up to 26.5%, switches in quality by up to 20%, without sacrificing the quality of the tiles within the FoV, even when there are significant head movement and changes in FoV during streaming.

Supplementary Material

MP4 File (3394171.3413550.mp4)
360-degree video streaming commonly encodes and transmits the video as independently-decodable tiles to conserve bandwidth of regions out of the viewer?s field of view (FoV). The bitrate of the tiles, however, can vary significantly across the tiles, complicating the choice of the representation to download for each tile in each segment to adapt to the bandwidth dynamics. In this paper, we model the tile rate allocation problem as a multiclass knapsack problem with a dynamic profit function that is a function of the FoV and the buffer occupancy. Experiments show that our approach can reduce bandwidth wastage by up to 41%, the number of stalls by up to 31%, stall durations by up to 26.5%, switches in quality by up to 20%, without sacrificing the quality of the tiles within the FoV, even when there are significant head movement and changes in FoV during streaming.

References

[1]
ISO/IEC 23009--1:2014/Amd 2:2015. 2015. Spatial relationship description, generalized URL parameters and other extensions. Technical Report. ISO/IEC.
[2]
A Deniz Aladagli, Erhan Ekmekcioglu, Dmitri Jarnikov, and Ahmet Kondoz. 2017. Predicting head trajectories in 360° virtual reality videos. In Proceedings of the 2017 International Conference on 3D Immersion (IC3D). IEEE, San Francisco, CA, 1--6.
[3]
Mathias Almquist, Viktor Almquist, Vengatanathan Krishnamoorthi, Niklas Carlsson, and Derek Eager. 2018. The prefetch aggressiveness tradeoff in 360° video streaming. In Proceedings of the 9th ACM Multimedia Systems Conference (MMSys '18). ACM, Amsterdam, The Netherlands, 258--269.
[4]
Abdelhak Bentaleb, Ali C Begen, and Roger Zimmermann. 2016. SDNDASH: Improving QoE of HTTP adaptive streaming using software defined networking. In Proceedings of the 24th ACM International Conference on Multimedia (MM '16). ACM, Amsterdam, The Netherlands, 1296--1305.
[5]
Abdelhak Bentaleb, Ali C Begen, and Roger Zimmermann. 2018. QoE-aware bandwidth broker for HTTP adaptive streaming flows in an SDN-enabled HFC network. IEEE Transactions on Broadcasting, Vol. 64, 2 (2018), 575--589.
[6]
Cyril Concolato, Jean Le Feuvre, Franck Denoual, Frédéric Mazé, Eric Nassor, Naël Ouedraogo, and Jonathan Taquet. 2018. Adaptive streaming of HEVC tiled videos using MPEG-DASH. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 28, 8 (2018), 1981--1992.
[7]
Xavier Corbillon, Francesca De Simone, and Gwendal Simon. 2017a. 360-degree video head movement dataset. In Proceedings of the 8th ACM Multimedia Systems Conference (MMSys '17). ACM, Taipei, Taiwan, 199--204.
[8]
Xavier Corbillon, Gwendal Simon, Alisa Devlic, and Jacob Chakareski. 2017b. Viewport-adaptive navigable 360-degree video delivery. In Proceedings of the 2017 IEEE International Conference on Communications (ICC). IEEE, Paris, France, 1--7.
[9]
Luca De Cicco, Vito Caldaralo, Vittorio Palmisano, and Saverio Mascolo. 2013. ELASTIC: A client-side controller for dynamic adaptive streaming over HTTP (DASH). In 20th International Packet Video Workshop. IEEE, San Jose, CA, 1--8.
[10]
Luca De Cicco and Saverio Mascolo. 2014. An adaptive video streaming control system: Modeling, validation, and performance evaluation. IEEE/ACM Transactions on Networking (TON), Vol. 22, 2 (2014), 526--539.
[11]
Ran Dubin, Ofer Hadar, and Amit Dvir. 2013. The effect of client buffer and MBR consideration on DASH adaptation logic. In Proceedings of the 2013 IEEE Wireless Communications and Networking Conference (WCNC). IEEE, Shanghai, China, 2178--2183.
[12]
Omar Eltobgy, Omar Arafa, and Mohamed Hefeeda. 2020. Mobile Streaming of Live 360-Degree Videos. IEEE Transactions on Multimedia (2020).
[13]
Ching-Ling Fan, Wen-Chih Lo, Yu-Tung Pai, and Cheng-Hsin Hsu. 2019. A survey on 360 video streaming: Acquisition, transmission, and display. ACM Computing Surveys (CSUR), Vol. 52, 4 (2019), 1--36.
[14]
Wu-chi Feng, Thanh Dang, John Kassebaum, and Tim Bauman. 2008. Supporting region-of-interest cropping through constrained compression. In Proceedings of the 16th ACM International Conference on Multimedia (MM '08). ACM, Vancouver, British Columbia, Canada, 745--748. https://doi.org/10.1145/1459359.1459476
[15]
DASH Industry Forum. 2014. Guidelines for implementation: DASH-AVC/264 test cases and vectors. Web page. http://dashif.org/guidelines/ Retrieved May 25, 2018 from
[16]
Stephen Hemminger. 2005. Network emulation with NetEm. In Proceedings of Australia's National Linux Conference. Canberra, Australia, 18--23.
[17]
Mohammad Hosseini and Viswanathan Swaminathan. 2016a. Adaptive 360 VR video streaming based on MPEG-DASH SRD. In Proceedings of the 2016 IEEE International Symposium on Multimedia (ISM '16). IEEE, San Jose, California, USA, 407--408.
[18]
Mohammad Hosseini and Viswanathan Swaminathan. 2016b. Adaptive 360 VR video streaming: Divide and conquer. In Proceedings of the 2016 IEEE International Symposium on Multimedia (ISM '16). IEEE, San Jose, California, USA, 107--110.
[19]
Shenghong Hu, Lingfen Sun, Chao Gui, Emmanuel Jammeh, and Is-Haka Mkwawa. 2014. Content-aware adaptation scheme for QoE optimized DASH applications. In 2014 IEEE Global Communications Conference (GLOBECOM '14). IEEE, Austin, Texas, USA, 1336--1341.
[20]
Te-Yuan Huang, Ramesh Johari, Nick McKeown, Matthew Trunnell, and Mark Watson. 2015. A buffer-based approach to rate adaptation: Evidence from a large video streaming service. ACM SIGCOMM Computer Communication Review, Vol. 44, 4 (2015), 187--198.
[21]
Parikshit Juluri, Venkatesh Tamarapalli, and Deep Medhi. 2015. SARA: Segment aware rate adaptation algorithm for dynamic adaptive streaming over HTTP. In Proceedings of the 2015 IEEE International Conference on Communication Workshop (ICCW). IEEE, London, UK, 1765--1770.
[22]
Kvazaar. 2018. Kvazaar. Web page. https://github.com/ultravideo/kvazaar
[23]
Jean Le Feuvre and Cyril Concolato. 2016. Tiled-based adaptive streaming using MPEG-DASH. In Proceedings of the 7th ACM Multimedia Systems Conference (MMSys '16). ACM, Klagenfurt, Austria, Article 41, bibinfonumpages3 pages.
[24]
Jean Le Feuvre and Cyril Concolato. 2018. GPAC. Web page. https://gpac.wp.imt.fr Retrieved April 2, 2019 from
[25]
Zhi Li, Xiaoqing Zhu, Joshua Gahm, Rong Pan, Hao Hu, Ali C Begen, and David Oran. 2014. Probe and adapt: Rate adaptation for HTTP video streaming at scale. IEEE Journal on Selected Areas in Communications, Vol. 32, 4 (2014), 719--733.
[26]
Wen-Chih Lo, Ching-Ling Fan, Jean Lee, Chun-Ying Huang, Kuan-Ta Chen, and Cheng-Hsin Hsu. 2017. 360 video viewing dataset in head-mounted virtual reality. In Proceedings of the 8th ACM Multimedia Systems Conference (MMSys '17). ACM, Taipei, Taiwan, 211--216.
[27]
Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with Pensieve. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication (SIGCOMM '17). ACM, Los Angeles, California, USA, 197--210.
[28]
Britta Meixner, Jan Willem Kleinrouweler, and Pablo Cesar. 2018. 4G/LTE channel quality reference signal trace data set. In Proceedings of the 9th ACM Multimedia Systems Conference (MMSys '18). ACM, Amsterdam, The Netherlands, 387--392.
[29]
Konstantin Miller, Abdel-Karim Al-Tamimi, and Adam Wolisz. 2016. QoE-based low-delay live streaming using throughput predictions. ACM Transactions on Multimedia Computing, Communications, and Application, Vol. 13, 1, Article 4 (Oct. 2016), 24 pages. https://doi.org/10.1145/2990505
[30]
Ricky KP Mok, Xiapu Luo, Edmond WW Chan, and Rocky KC Chang. 2012. QDASH: A QoE-aware DASH system. In Proceedings of the 3rd ACM Multimedia Systems Conference (MMSys '12). ACM, Chapel Hill, North Carolina, USA, 11--22.
[31]
Christopher Müller and Christian Timmerer. 2011. A VLC media player plugin enabling dynamic adaptive streaming over HTTP. In Proceedings of the 19th ACM International Conference on Multimedia (MM '11). ACM, Scottsdale, Arizona, USA, 723--726.
[32]
Afshin Taghavi Nasrabadi, Anahita Mahzari, Joseph D Beshay, and Ravi Prakash. 2017. Adaptive 360-degree video streaming using scalable video coding. In Proceedings of the 25th ACM International Conference on Multimedia (MM '17). ACM, Mountain View, California, USA, 1689--1697.
[33]
Anh Nguyen, Zhisheng Yan, and Klara Nahrstedt. 2018. Your attention is unique: Detecting 360-degree video saliency in head-mounted display for head movement prediction. In Proceedings of the 26th ACM International Conference on Multimedia (MM '18). ACM, Seoul, Republic of Korea, 1190--1198.
[34]
Omar A Niamut, Emmanuel Thomas, Lucia D'Acunto, Cyril Concolato, Franck Denoual, and Seong Yong Lim. 2016. MPEG DASH SRD: Spatial relationship description. In Proceedings of the 7th ACM Multimedia Systems Conference (MMSys '16). ACM, Klagenfurt, Austria, 5.
[35]
Stefano Petrangeli, Viswanathan Swaminathan, Mohammad Hosseini, and Filip De Turck. 2017. An HTTP/2-based adaptive streaming framework for 360 virtual reality videos. In Proceedings of the 25th ACM International Conference on Multimedia (MM '17). ACM, Mountain View, California, USA, 306--314.
[36]
David Pisinger. 1995. A minimal algorithm for the multiple-choice knapsack problem. European Journal of Operational Research, Vol. 83, 2 (1995), 394--410.
[37]
Feng Qian, Bo Han, Qingyang Xiao, and Vijay Gopalakrishnan. 2018. Flare: Practical viewport-adaptive 360-degree video streaming for mobile devices. In Proceedings of the 24th Annual International Conference on Mobile Computing and Networking (MobiCom '18). ACM, New Delhi, India, 99--114.
[38]
Feng Qian, Lusheng Ji, Bo Han, and Vijay Gopalakrishnan. 2016. Optimizing 360 video delivery over cellular networks. In Proceedings of the 5th Workshop on All Things Cellular: Operations, Applications and Challenges. ACM, London, United Kingdom, 1--6.
[39]
Ngo Quang Minh Khiem, Guntur Ravindra, Axel Carlier, and Wei Tsang Ooi. 2010. Supporting zoomable video streams with dynamic region-of-interest cropping. In Proceedings of the First Annual ACM Multimedia Systems Conference (MMSys '10). ACM, Phoenix, Arizona, USA, 259--270. https://doi.org/10.1145/1730836.1730868
[40]
Benjamin Rainer, Stefan Lederer, Christopher Müller, and Christian Timmerer. 2012. A seamless Web integration of adaptive HTTP streaming. In Proceedings of the 20th European Signal Processing Conference (EUSIPCO '12). IEEE, Bucharest, Romania, 1519--1523.
[41]
Hideaki Shimazaki and Shigeru Shinomoto. 2007. A method for selecting the bin size of a time histogram. Neural computation, Vol. 19, 6 (2007), 1503--1527.
[42]
Prabhakant Sinha and Andris A Zoltners. 1979. The multiple-choice knapsack problem. Operations Research, Vol. 27, 3 (1979), 503--515.
[43]
Kevin Spiteri, Ramesh Sitaraman, and Daniel Sparacio. 2018. From theory to practice: Improving bitrate adaptation in the DASH reference player. In Proceedings of the 9th ACM Multimedia Systems Conference (MMSys '18). ACM, Amsterdam, The Netherlands, 123--137.
[44]
Kevin Spiteri, Rahul Urgaonkar, and Ramesh K Sitaraman. 2016. BOLA: Near-optimal bitrate adaptation for online videos. In Proceedings of the 35th Annual IEEE International Conference on Computer Communications (INFOCOM '16). IEEE, San Francisco, CA, USA, 1--9.
[45]
Kashyap Kammachi Sreedhar, Alireza Aminlou, Miska M Hannuksela, and Moncef Gabbouj. 2016. Viewport-adaptive encoding and streaming of 360-degree video for virtual reality applications. In Proceedings of the 2016 IEEE International Symposium on Multimedia (ISM '16). IEEE, San Jose, California, USA, 583--586.
[46]
Liyang Sun, Fanyi Duanmu, Yong Liu, Yao Wang, Yinghua Ye, Hang Shi, and David Dai. 2018. Multi-path multi-tier 360-degree video streaming in 5G networks. In Proceedings of the 9th ACM Multimedia Systems Conference (MMSys '18). ACM, Amsterdam, The Netherlands, 162--173.
[47]
Afshin TaghaviNasrabadi, Anahita Mahzari, Joseph D Beshay, and Ravi Prakash. 2017. Adaptive 360-degree video streaming using layered video coding. In Proceedings of the 2017 IEEE Virtual Reality (VR '17). IEEE, Los Angeles, California, USA, 347--348.
[48]
Truong Cong Thang, Quang-Dung Ho, Jung Won Kang, and Anh T Pham. 2012. Adaptive streaming of audiovisual content using MPEG DASH. IEEE Transactions on Consumer Electronics, Vol. 58, 1 (2012), 78--85.
[49]
Hui Wang, Vu-Thanh Nguyen, Wei Tsang Ooi, and Mun Choon Chan. 2014. Mixing tile resolutions in tiled video: A perceptual quality assessment. In Proceedings of Network and Operating System Support on Digital Audio and Video Workshop (NOSSDAV '14). ACM, Singapore, Singapore, 25--30. https://doi.org/10.1145/2597176.2578267
[50]
XAUT. 2018. Formerly Xutomation. Web page. http://xautomation.sourceforge.net
[51]
Mengbai Xiao, Shuoqian Wang, Chao Zhou, Li Liu, Zhenhua Li, Yao Liu, and Songqing Chen. 2018. MiniView layout for bandwidth-efficient 360-degree video. In Proceedings of the 26th ACM International Conference on Multimedia (MM '18). ACM, Seoul, Republic of Korea, 914--922.
[52]
Mengbai Xiao, Chao Zhou, Yao Liu, and Songqing Chen. 2017. Optile: Toward optimal tiling in 360-degree video streaming. In Proceedings of the 25th ACM International Conference on Multimedia (MM '17). ACM, Mountain View, California, USA, 708--716.
[53]
Shichang Xu, Subhabrata Sen, Z Morley Mao, and Yunhan Jia. 2017. Dissecting VOD services for cellular: Performance, root causes and best practices. In Proceedings of the 2017 Internet Measurement Conference. ACM, London, United Kingdom, 220--234.
[54]
Praveen Kumar Yadav. 2020. A queueing theory approach to adaptive video streaming. Ph.D. Dissertation. School of Computing, National University of Singapore.
[55]
Praveen Kumar Yadav, Arash Shafiei, and Wei Tsang Ooi. 2017. QUETRA: A queuing theory approach to DASH rate adaptation. In Proceedings of the 25th ACM International Conference on Multimedia (MM '17). ACM, Mountain View, California, USA, 1130--1138.
[56]
Xiaoqi Yin, Abhishek Jindal, Vyas Sekar, and Bruno Sinopoli. 2015. A control-theoretic approach for dynamic adaptive video streaming over HTTP. ACM SIGCOMM Computer Communication Review, Vol. 45, 4 (2015), 325--338.
[57]
YouTube. 2018. YouTube: Recommended upload encoding settings. Web page. https://support.google.com/youtube/answer/1722171 Retrieved April 2, 2019 from
[58]
Alireza Zare, Alireza Aminlou, Miska M. Hannuksela, and Moncef Gabbouj. 2016. HEVC-compliant tile-based streaming of panoramic video for virtual reality applications. In Proceedings of the 24th ACM International Conference on Multimedia (MM '16). ACM, Amsterdam, The Netherlands, 601--605.

Cited By

View all
  • (2024)TileClipperProceedings of the 2024 USENIX Conference on Usenix Annual Technical Conference10.5555/3691992.3692051(967-984)Online publication date: 10-Jul-2024
  • (2024)DACOD360: Deadline-Aware Content Delivery for 360-Degree Video Streaming Over MEC NetworksIEEE Transactions on Multimedia10.1109/TMM.2023.332143926(4168-4182)Online publication date: 1-Jan-2024
  • (2024)Achieving QoE Fairness in Bitrate Allocation of 360° Video StreamingIEEE Transactions on Multimedia10.1109/TMM.2023.327728626(1169-1178)Online publication date: 2024
  • Show More Cited By

Index Terms

  1. Tile Rate Allocation for 360-Degree Tiled Adaptive Video Streaming

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MM '20: Proceedings of the 28th ACM International Conference on Multimedia
    October 2020
    4889 pages
    ISBN:9781450379885
    DOI:10.1145/3394171
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 12 October 2020

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. 360-degree video streaming
    2. dash
    3. multiclass knapsack
    4. tile rate allocation
    5. tiled video streaming

    Qualifiers

    • Research-article

    Funding Sources

    • NExT++ research, supported by the National Research Foundation, Prime Minister's Office, Singapore under its IRC@SG Funding Initiative

    Conference

    MM '20
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)47
    • Downloads (Last 6 weeks)5
    Reflects downloads up to 27 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)TileClipperProceedings of the 2024 USENIX Conference on Usenix Annual Technical Conference10.5555/3691992.3692051(967-984)Online publication date: 10-Jul-2024
    • (2024)DACOD360: Deadline-Aware Content Delivery for 360-Degree Video Streaming Over MEC NetworksIEEE Transactions on Multimedia10.1109/TMM.2023.332143926(4168-4182)Online publication date: 1-Jan-2024
    • (2024)Achieving QoE Fairness in Bitrate Allocation of 360° Video StreamingIEEE Transactions on Multimedia10.1109/TMM.2023.327728626(1169-1178)Online publication date: 2024
    • (2024)HiVAT: Improving QoE for Hybrid Video Streaming Service With Adaptive TranscodingIEEE Transactions on Mobile Computing10.1109/TMC.2024.339939823:12(11209-11226)Online publication date: Dec-2024
    • (2024)Synergistic Temporal-Spatial User-Aware Viewport Prediction for Optimal Adaptive 360-Degree Video StreamingIEEE Transactions on Broadcasting10.1109/TBC.2024.337411970:2(453-467)Online publication date: Jun-2024
    • (2024)Online tile dispatching framework with guarantees for 360-degree video streaming in wireless networksWireless Networks10.1007/s11276-024-03871-6Online publication date: 18-Nov-2024
    • (2024)Tile-size aware bitrate allocation for adaptive 360$$^{\circ }$$ video streamingMultimedia Tools and Applications10.1007/s11042-024-19486-0Online publication date: 5-Jun-2024
    • (2024)Modeling the non-uniform retinal perception for viewport-dependent streaming of immersive videoMultimedia Systems10.1007/s00530-024-01434-530:4Online publication date: 5-Aug-2024
    • (2023)Adaptive Anti-jitter Optimization for Immersive Streaming in Metaverse ScenariosProceedings of the 2023 7th International Conference on High Performance Compilation, Computing and Communications10.1145/3606043.3606087(310-319)Online publication date: 17-Jun-2023
    • (2023)Advanced Predictive Tile Selection Using Dynamic Tiling for Prioritized 360° Video VR StreamingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/360314620:1(1-28)Online publication date: 1-Jun-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media