research-article

16K Cinematic VR Streaming

Authors:

Patrice Rondao Alface,

Jean-Francois MacqAuthors Info & Claims

MM '17: Proceedings of the 25th ACM international conference on Multimedia

Pages 1105 - 1112

https://doi.org/10.1145/3123266.3123307

Published: 19 October 2017 Publication History

Get Access

Abstract

We present an end-to-end system for streaming Cinematic Virtual Reality (VR) content (also called 360 or omnidirectional content). Content is captured and ingested at a resolution of 16K at 25Hz and streamed towards untethered mobile VR devices. Besides the usual navigation interactions such as panning and tilting offered by common VR systems, we also provide a zooming interactivity. This allows the VR client to fetch high quality pixels captured at a spatial resolution of 16K that greatly increase perceived quality compared to a 4K VR streaming solution. Since current client devices are not capable of receiving and decoding a 16K video, several optimizations are provided to only stream the required pixels for the current viewport of the user, while meeting strict latency and bandwidth requirements for a qualitative VR immersive experience.

References

[1]

Earnshaw, Rae A. (ed.). 2014. Virtual Reality Systems. Academic Press.

Google Scholar

[2]

Wexelblat, A. (ed.). 2014. Virtual reality: applications and explorations. Academic Press.

Google Scholar

[3]

Rondao Alface, P., Macq, J.-F. and Verzijp, N. 2012. Interactive Omnidirectional Video Delivery: A Bandwidth-Effective Approach. Bell Labs Tech. J., 16: 135--147.

Digital Library

Google Scholar

[4]

M. Yu, H. Lakshman, and B. Girod. 2015. Content adaptive representations of omnidirectional videos for cinematic virtual reality. Proceedings of the 3rd International Workshop on Immersive Media Experiences. ACM. pp. 1--6.

Digital Library

Google Scholar

[5]

Van Brandenburg, R., Niamut, O., Prins, M., and Stokking, H. 2011. Spatial segmentation for immersive media delivery. In Intelligence in Next Generation Networks (ICIN), 2011 15th International Conference on, IEEE, pp. 151--156.

Google Scholar

[6]

A. Mavlankar and B. Girod. 2009. Background Extraction and Long-Term Memory Motion-Compensated Prediction for Spatial-Random-Access-Enabled Video Coding. Proc. Picture Coding Symp. (PCS '09), Chicago, IL.

Digital Library

Google Scholar

[7]

C. W. Fu, L. Wan, T. T. Wong and C. S. Leung. 2009. The rhombic dodecahedron map: An efficient scheme for encoding panoramic video. IEEE Transactions on Multimedia. 11(4), pp. 634--644.

Digital Library

Google Scholar

[8]

Mavlankar, A., and Girod, B. 2010. Video streaming with interactive pan/tilt/zoom. In High-Quality Visual Experience. Springer Berlin Heidelberg. pp. 431--455.

Google Scholar

[9]

Skupin, R., Sanchez, Y., Hellge, C., and Schierl, T. (2016, December). Tile Based HEVC Video for Head Mounted Displays. In Multimedia (ISM), 2016 IEEE International Symposium on, pp. 399--400. IEEE.

Google Scholar

[10]

Vanne, J., Viitanen, M., Hamalainen, T. D., and Hallapuro, A. 2012. Comparative rate-distortion-complexity analysis of HEVC and AVC video codecs. IEEE Transactions on Circuits and Systems for Video Technology. 22(12), 1885--1898.

Digital Library

Google Scholar

[11]

Wiegand, T., Schwarz, H., Joch, A., Kossentini, F., and Sullivan, G. J. 2003. Rate-constrained coder control and comparison of video coding standards. IEEE Transactions on circuits and systems for video technology. 13(7), 688--703.

Digital Library

Google Scholar

[12]

G. Bjontegaard. Calculation of average PSNR differences between RD-curves. Technical Report VCEG-M33, ITU-T SG16/Q6, Austin, Texas, USA, 2001.

Google Scholar

Cited By

View all

Li ZZhong PHuang JGao FWang J(2024)Achieving QoE Fairness in Bitrate Allocation of 360° Video StreamingIEEE Transactions on Multimedia10.1109/TMM.2023.327728626(1169-1178)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3277286
Yu ZLo C(2023)Mapping the Viewer Experience in Cinematic Virtual Reality: A Systematic ReviewPRESENCE: Virtual and Augmented Reality10.1162/pres_a_0040932(205-229)Online publication date: 1-Dec-2023
https://doi.org/10.1162/pres_a_00409
Li JZhang HMa H(2023)DRL-based transmission control for QoE guaranteed transmission efficiency optimization in tile-based panoramic video streamingMultimedia Systems10.1007/s00530-023-01129-329:5(2761-2777)Online publication date: 30-Jun-2023
https://doi.org/10.1007/s00530-023-01129-3
Show More Cited By

Index Terms

16K Cinematic VR Streaming
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Virtual reality
2. Information systems
  1. Information systems applications
    1. Multimedia information systems
      1. Multimedia streaming

Recommendations

DreamStream: Immersive and Interactive Spectating in VR
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

Today spectating and streaming virtual reality (VR) activities typically involves spectators viewing a 2D stream of the VR user’s view. Streaming 2D videos of the game play is popular and well-supported by platforms such as Twitch. However, the generic ...
Immersive virtual reality with multimodal interaction and streaming technology
ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal Interaction

In this demo, we present an immersive virtual reality (VR) system which integrates multimodal interaction sensors (i.e., smartphone, Kinect v2, and Myo armband) and streaming technology to improve the VR experience. The integrated system solves the ...
VR Grabbers: Ungrounded Haptic Retargeting for Precision Grabbing Tools
UIST '18: Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology

Haptic feedback in VR is important for realistic simulation in virtual reality. However, recreating the haptic experience for hand tools in VR traditionally requires hardware with precise actuators, adding complexity to the system. We propose Ungrounded ...

Comments

Information & Contributors

Information

Published In

MM '17: Proceedings of the 25th ACM international conference on Multimedia

October 2017

2028 pages

ISBN:9781450349062

DOI:10.1145/3123266

General Chairs:
Qiong Liu
FXPAL, USA
,
Rainer Lienhart
Universität Augsburg, Germany
,
Haohong Wang
TCL America, USA
,
Program Chairs:
Sheng-Wei "Kuan-Ta" Chen
Academia Sinica, Taiwan
,
Susanne Boll
University of Oldenburg, Germany
,
Phoebe Chen
La Trobe University, Australia
,
Gerald Friedland
Lawrence Livermore National Lab, USA
,
Jia Li
Google, USA
,
Shuicheng Yan
Qihoo 360, China

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

VLAIO

Conference

MM '17

Sponsor:

SIGMM

MM '17: ACM Multimedia Conference

October 23 - 27, 2017

California, Mountain View, USA

Acceptance Rates

MM '17 Paper Acceptance Rate 189 of 684 submissions, 28%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
571
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)1

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Li ZZhong PHuang JGao FWang J(2024)Achieving QoE Fairness in Bitrate Allocation of 360° Video StreamingIEEE Transactions on Multimedia10.1109/TMM.2023.327728626(1169-1178)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3277286
Yu ZLo C(2023)Mapping the Viewer Experience in Cinematic Virtual Reality: A Systematic ReviewPRESENCE: Virtual and Augmented Reality10.1162/pres_a_0040932(205-229)Online publication date: 1-Dec-2023
https://doi.org/10.1162/pres_a_00409
Li JZhang HMa H(2023)DRL-based transmission control for QoE guaranteed transmission efficiency optimization in tile-based panoramic video streamingMultimedia Systems10.1007/s00530-023-01129-329:5(2761-2777)Online publication date: 30-Jun-2023
https://doi.org/10.1007/s00530-023-01129-3
Samiei APrakash R(2021)Improving 360-Degree Video Field-of-View Prediction and Edge Caching2021 IEEE International Symposium on Multimedia (ISM)10.1109/ISM52913.2021.00009(9-16)Online publication date: Nov-2021
https://doi.org/10.1109/ISM52913.2021.00009
He DJiang JMa TYang GWestphal CGarcia-Luna-Aceves JXia S(2021)CUBIST: High-Quality 360-Degree Video Streaming Services via Tile-based Edge Caching and FoV-Adaptive Prefetching2021 IEEE International Conference on Web Services (ICWS)10.1109/ICWS53863.2021.00039(208-218)Online publication date: Sep-2021
https://doi.org/10.1109/ICWS53863.2021.00039
Wijnants MCoppers SRovelo Ruiz GQuax PLamotte WAmsaleg LHuet BLarson MGravier GHung HNgo CTsang Ooi W(2019)Talking Video HeadsProceedings of the 27th ACM International Conference on Multimedia10.1145/3343031.3351045(2449-2458)Online publication date: 15-Oct-2019
https://dl.acm.org/doi/10.1145/3343031.3351045
Fan CLo WPai YHsu C(2019)A Survey on 360° Video StreamingACM Computing Surveys10.1145/332911952:4(1-36)Online publication date: 30-Aug-2019
https://dl.acm.org/doi/10.1145/3329119
Shi SHwang MGupta VJana RSong JKim MLane NBalan R(2019)Latency Adaptive Streaming of 8K 360 Degree Video to Mobile VR Headsets (demo)Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services10.1145/3307334.3328566(651-652)Online publication date: 12-Jun-2019
https://dl.acm.org/doi/10.1145/3307334.3328566
Shi SGupta VJana RSong JKim MLane NBalan R(2019)FreedomProceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services10.1145/3307334.3326087(130-141)Online publication date: 12-Jun-2019
https://dl.acm.org/doi/10.1145/3307334.3326087
Shi SHwang MGupta VHan BGopalakrishnan VJana RZink MToni LBegen A(2019)Real time streaming of 8K 360 degree video to mobile VR headsetsProceedings of the 10th ACM Multimedia Systems Conference10.1145/3304109.3323840(328-329)Online publication date: 18-Jun-2019
https://dl.acm.org/doi/10.1145/3304109.3323840
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

DreamStream: Immersive and Interactive Spectating in VR

Immersive virtual reality with multimodal interaction and streaming technology

VR Grabbers: Ungrounded Haptic Retargeting for Precision Grabbing Tools

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations