research-article

Is Foveated Rendering Perceivable in Virtual Reality?: Exploring the Efficiency and Consistency of Quality Assessment Methods

Authors:

Cheng-Hsin Hsu,

Chun-Ying Huang,

Chin-Laung Lei,

Kuan-Ta ChenAuthors Info & Claims

MM '17: Proceedings of the 25th ACM international conference on Multimedia

Pages 55 - 63

https://doi.org/10.1145/3123266.3123434

Published: 19 October 2017 Publication History

Abstract

Foveated rendering leverages human visual system to increase video quality under limited computing resources for Virtual Reality (VR). More specifically, it increases the frame rate and the video quality of the foveal vision via lowering the resolution of the peripheral vision. Optimizing foveated rendering systems is, however, not an easy task, because there are numerous parameters that need to be carefully chosen, such as the number of layers, the eccentricity degrees, and the resolution of the peripheral region. Furthermore, there is no standard and efficient way to evaluate the Quality of Experiment (QoE) of foveated rendering systems. In this paper, we propose a framework to compare the performance of different subjective assessment methods on foveated rendering systems. We consider two performance metrics: efficiency and consistency, using the perceptual ratio, which is the probability of the foveated rendering is perceivable by users. A regression model is proposed to model the relationship between the human perceived quality and foveated rendering parameters. Our comprehensive study and analysis reveal several insights: 1) there is no absolute superior subjective assessment method, 2) subjects need to make more observations to confirm the foveated rendering is imperceptible than perceptible, 3) subjects barely notice the foveated rendering with an eccentricity degree of 7.5 degrees+ and peripheral region of a resolution of 540p+, and 4) QoE levels are highly dependent on the individuals and scenes. Our findings are crucial for optimizing the foveated rendering systems for future VR applications.

References

[1]

Maria Schultheis and Albert Rizzo. The application of Virtual Reality technology in rehabilitation. Rehabilitation Psychology, 46(3):296, 2001.

[2]

Tariq Mujber, Tamas Szecsi, and Mohammed Hashmi. Virtual Reality applications in manufacturing process simulation. Journal of materials processing technology, 155:1834--1838, 2004.

[3]

Christine Youngblut. Educational uses of Virtual Reality technology. Technical report, DTIC Document, 1998.

[4]

Daniel Guttentag. Virtual Reality: Applications and implications for tourism. Tourism Management, 31(5):637--651, 2010.

[5]

Doug Bowman and Ryan McMahan. Virtual Reality: How much immersion is enough? Computer, 40(7):36--43, 2007.

Digital Library

[6]

Asynchronous Timewarp on Oculus Rift. https://developer3.oculus.com/blog/asynchronous-timewarp-on-oculus-rift/.

[7]

Brian Wandell. Foundations of Vision. Sinauer Associates, 1995.

[8]

Markus Fiedler, Tobias Hossfeld, and Phuoc Tran-Gia. A generic quantitative relationship between Quality of Experience and Quality of Service. IEEE Network, 24(2):36--41, 2010.

Digital Library

[9]

Karlene Ball, Bettina Beard, Daniel Roenker, Richard Miller, and David Griggs. Age and visual search: Expanding the useful field of view. JOSA A, 5(12):2210--2219, 1988.

[10]

Hans Strasburger, Ingo Rentschler, and Martin Jüttner. Peripheral vision and pattern recognition: A review. Journal of Vision, 11(5):13--13, 2011.

[11]

Eyal Reingold, Lester Loschky, George McConkie, and David Stampe. Gaze-contingent multiresolutional displays: An integrative review. Human Factors: The Journal of the Human Factors and Ergonomics Society, 45(2):307--328, 2003.

[12]

Brian Guenter, Mark Finch, Steven Drucker, Desney Tan, and John Snyder. Foveated 3D graphics. ACM Transactions on Graphics, 31(6):164:1--164:10, 2012.

Digital Library

[13]

Nicholas Swafford, José Iglesias-Guitian, Charalampos Koniaris, Bochang Moon, Darren Cosker, and Kenny Mitchell. User, metric, and computational evaluation of foveated rendering methods. In Proceedings of the ACM Symposium on Applied Perception, pages 7--14, 2016.

Digital Library

[14]

Anjul Patney, Marco Salvi, Joohwan Kim, Anton Kaplanyan, Chris Wyman, Nir Benty, David Luebke, and Aaron Lefohn. Towards foveated rendering for gaze-tracked Virtual Reality. ACM Transactions on Graphics, 35(6):179:1--179:12, 2016.

Digital Library

[15]

Pietro Lungaro and Konrad Tollmar. Eye-gaze based service provision and QoE optimization. In Proceedings of the Workshop on Perceptual Quality of Systems, pages 6--10, 2016.

[16]

Sanghoon Lee, Marios Pattichis, and Alan Bovik. Foveated video quality assessment. IEEE Transactions on Multimedia, 4(1):129--132, 2002.

Digital Library

[17]

Recommendation ITU-R BT.500--13, 2012.

[18]

Rafał Mantiuk, Anna Tomaszewska, and Radosław Mantiuk. Comparison of four subjective methods for image quality assessment. Computer Graphics Forum, 31(8):2478--2491, 2012.

Digital Library

[19]

George Gescheider. Psychophysics: The Fundamentals. Psychology Press, 3rd edition, 1997.

[20]

Wanmin Wu, Ahsan Arefin, Gregorij Kurillo, Pooja Agarwal, Klara Nahrstedt, and Ruzena Bajcsy. Color-plus-depth level-of-detail in 3D tele-immersive video: A psychophysical approach. In Proceedings of the ACM International Conference on Multimedia, pages 13--22, 2011.

Digital Library

Cited By

Wu NLiu KCheng RHan BZhou POkoshi TKo JLiKamWa R(2024)Theia: Gaze-driven and Perception-aware Volumetric Content Delivery for Mixed Reality HeadsetsProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661858(70-84)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661858
Kim JKim JJung MKwon TKim K(2024)Individualized foveated rendering with eye-tracking head-mounted displayVirtual Reality10.1007/s10055-023-00931-828:1Online publication date: 19-Jan-2024
https://dl.acm.org/doi/10.1007/s10055-023-00931-8
Surace LWernikowski MTursun CMyszkowski KMantiuk RDidyk P(2023)Learning GAN-Based Foveated Reconstruction to Recover Perceptually Important Image FeaturesACM Transactions on Applied Perception10.1145/358307220:2(1-23)Online publication date: 21-Apr-2023
https://dl.acm.org/doi/10.1145/3583072
Show More Cited By

Index Terms

Is Foveated Rendering Perceivable in Virtual Reality?: Exploring the Efficiency and Consistency of Quality Assessment Methods
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
      1. Perception
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies
    2. Interaction paradigms
      1. Virtual reality

Recommendations

User, metric, and computational evaluation of foveated rendering methods
SAP '16: Proceedings of the ACM Symposium on Applied Perception

Perceptually lossless foveated rendering methods exploit human perception by selectively rendering at different quality levels based on eye gaze (at a lower computational cost) while still maintaining the user's perception of a full quality render. We ...
Perceptually-based foveated virtual reality
SIGGRAPH '16: ACM SIGGRAPH 2016 Emerging Technologies

Humans have two distinct vision systems: foveal and peripheral vision. Foveal vision is sharp and detailed, while peripheral vision lacks fidelity. The difference in characteristics of the two systems enable recently popular foveated rendering systems, ...
Towards foveated rendering for gaze-tracked virtual reality

Foveated rendering synthesizes images with progressively less detail outside the eye fixation region, potentially unlocking significant speedups for wide field-of-view displays, such as head mounted displays, where target framerate and resolution is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '17: Proceedings of the 25th ACM international conference on Multimedia

October 2017

2028 pages

ISBN:9781450349062

DOI:10.1145/3123266

General Chairs:
Qiong Liu
FXPAL, USA
,
Rainer Lienhart
Universität Augsburg, Germany
,
Haohong Wang
TCL America, USA
,
Program Chairs:
Sheng-Wei "Kuan-Ta" Chen
Academia Sinica, Taiwan
,
Susanne Boll
University of Oldenburg, Germany
,
Phoebe Chen
La Trobe University, Australia
,
Gerald Friedland
Lawrence Livermore National Lab, USA
,
Jia Li
Google, USA
,
Shuicheng Yan
Qihoo 360, China

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Ministry of Science and Technology of Taiwan

Conference

MM '17

Sponsor:

SIGMM

MM '17: ACM Multimedia Conference

October 23 - 27, 2017

California, Mountain View, USA

Acceptance Rates

MM '17 Paper Acceptance Rate 189 of 684 submissions, 28%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
700
Total Downloads

Downloads (Last 12 months)65
Downloads (Last 6 weeks)4

Reflects downloads up to 27 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wu NLiu KCheng RHan BZhou POkoshi TKo JLiKamWa R(2024)Theia: Gaze-driven and Perception-aware Volumetric Content Delivery for Mixed Reality HeadsetsProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661858(70-84)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661858
Kim JKim JJung MKwon TKim K(2024)Individualized foveated rendering with eye-tracking head-mounted displayVirtual Reality10.1007/s10055-023-00931-828:1Online publication date: 19-Jan-2024
https://dl.acm.org/doi/10.1007/s10055-023-00931-8
Surace LWernikowski MTursun CMyszkowski KMantiuk RDidyk P(2023)Learning GAN-Based Foveated Reconstruction to Recover Perceptually Important Image FeaturesACM Transactions on Applied Perception10.1145/358307220:2(1-23)Online publication date: 21-Apr-2023
https://dl.acm.org/doi/10.1145/3583072
Lisboa TMacêdo HPorcino TOliveira ETrevisan DClua E(2023)Is Foveated Rendering Perception Affected by Users’ Motion?2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00127(1104-1112)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR59233.2023.00127
Rolff TSchmidt SLi KSteinicke FFrintrop S(2023)VRS-NeRF: Accelerating Neural Radiance Field Rendering with Variable Rate Shading2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00039(243-252)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR59233.2023.00039
Wang LShi XLiu Y(2023)Foveated rendering: A state-of-the-art surveyComputational Visual Media10.1007/s41095-022-0306-49:2(195-228)Online publication date: 3-Jan-2023
https://doi.org/10.1007/s41095-022-0306-4
Fan CHung THsu C(2022)Modeling the User Experience of Watching 360° Videos with Head-Mounted DisplaysACM Transactions on Multimedia Computing, Communications, and Applications10.1145/346382518:1(1-23)Online publication date: 27-Jan-2022
https://dl.acm.org/doi/10.1145/3463825
Huong THa DTran HViet NTien BThanh NThang TNam P(2022)An Effective Foveated 360° Image Assessment Based on Graph Convolution NetworkIEEE Access10.1109/ACCESS.2022.320476610(98165-98178)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3204766
Cha SMa JSeo JKim JHan S(2022)Empirical comparison of spatial experience between photo-based IVE and real spaceArchitectural Science Review10.1080/00038628.2022.213408966:1(1-16)Online publication date: 17-Oct-2022
https://doi.org/10.1080/00038628.2022.2134089
Vlahovic SSuznjevic MSkorin-Kapov L(2022)A survey of challenges and methods for Quality of Experience assessment of interactive VR applicationsJournal on Multimodal User Interfaces10.1007/s12193-022-00388-016:3(257-291)Online publication date: 29-Apr-2022
https://doi.org/10.1007/s12193-022-00388-0
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten