Conor Keighrey
Niall Murray
ABSTRACT
Virtual reality (VR) and augmented reality (AR) applications are gaining significant attention in industry and academia as potential avenues to support truly immersive and interactive multimedia experiences. Understanding the user perceived quality of immersive multimedia experiences is critical to the success of these technologies. However, this is a multidimensional and multifactorial problem. The user quality of experience (QoE) is influenced by human, context and system factors. Attempts to understand QoE via multimedia quality assessment has typically involved users reporting their experiences via post-test questionnaires. More recently, efforts have been made to automatically collect objective metrics that can quantitatively reflect user QoE in terms of physiological measurement methods. In this context, this paper presents a novel comparison of objective quality measures of immersive AR and VR applications through physiological: (electrodermal activity (EDA) and heart rate (HR)); and interaction (response times (RT), incorrect responses, and miss-click) metrics. The analysis shows consistency in terms of physiological ratings and miss-click metrics between the AR and VR groups. Interestingly, the AR group reported lower response times and less incorrect responses compared to the VR group. The difference between the AR and VR groups was statistically significant for the incorrect response metric and in 45.5% of the cases tested for response times metric, they were statistically significant with 95% confidence levels.
- C. Timmerer, M. Waltl, R. Benjamin and N. Murray, "Sensory Experience: Quality of Experience beyond Audio-Visual," in Quality of Experience: Advanced Concepts, Applications and Methods, Springer, 2014, pp. 351--365.Google Scholar
- J. Steuer, "Defining Virtual Reality: Dimensions Determining Telepresence," Journal of Communication, vol. 42, no. 4, pp. 73--93, 1992.Google ScholarCross Ref
- R. Azuma, Y. Baillot, S. Feiner, S. Julier, B. MacIntyre and R. Behringer, "Recent Advances in Augmented Reality," IEEE Computer Graphics and Applications, vol. 21, no. 6, pp. 34--47, 2001. Google ScholarDigital Library
- S. Möller and A. Raake, "Quality and Quality of Experiance," in Quality of Experience: Advanced Concepts, Applications and Methods, Springer, 2014, p. 19.Google Scholar
- J. Puig, A. Perkis, F. Lindseth and T. Ebrahimi, "Towards an Efficient Methodology for Evaluation of Quality of Experience in Augmented Reality," in Quality of Multimedia Experience (QoMEX), 2012.Google Scholar
- J. Cha, M. Eid, A. Barghout, A. M. Rahm and A. El Saddik, "HugMe: Synchronous Haptic Teleconferencing," in ACM international conference on Multimedia, 2009. Google ScholarDigital Library
- M. Obrist, C. Velasco, C. Vi, N. Ranasinghe, A. Israr, A. Cheok, C. Spence and P. Gopalakrishnakone, "Sensing the future of HCI: touch, taste, and smell user interfaces," Sussex Research Online, 2016.Google Scholar
- A. Drachen, L. E. Nacke, G. Yannakakis and A. L. Pedersen, "Correlation between Heart Rate, Electrodermal Activity and Player Experience," in SIGGRAPH Symposium on Video Games, 2010. Google ScholarDigital Library
- T. Hoßfeld, R. Schatz and S. Egger, "SOS: The Mos Is Not Enough!," in Quality of Multimedia Experience (QoMEX), 2011.Google Scholar
- E. Kroupi, P. Hanhart, J.-S. Lee, M. Rerabek and T. Ebrahimi, "Modeling Immersive Media Experiences by Sensing Impact on Subjects," Multimedia Tools and Applications, vol. 75, p. 12409--12429, 2016. Google ScholarDigital Library
- U. Engelke, D. P. Darcy, G. H. Mulliken, S. Bosse, M. G. Martini, S. Arndt, J.-N. Antons, K. Y. Chat, N. Ramzan and K. Brunnström, "Psychophysiology-Based QoE Assessment: A Survey," IEEE Journal of Selected Topics in Signal Processing, 2017.Google Scholar
- K. Swinburn, G. Porter and D. Howard, Comprehensive Aphasia Test, Psychology Press, 2004.Google Scholar
- "ITU-T BT.500: Methodology for the subjective assessment of the quality of television pictures," 2017. {Online}. Available: https://www.itu.int/rec/R-REC-BT.500. {Accessed 07 04 2017}.Google Scholar
- "ITU-T P.910: Subjective video quality assessment methods for multimedia applications," {Online}. Available: https://www.itu.int/rec/T-REC-P.910-199909-S/en. {Accessed 01 02 2017}.Google Scholar
- D. Egan, S. Brennan, J. Barrett, Y. Qiao, C. Timmerer and N. Murray, "An evaluation of Heart Rate and ElectroDermal Activity as an objective QoE evaluation method for immersive virtual reality environments," in Quality of Multimedia Experience (QoMEX), 2016.Google Scholar
- P. D. Ritsos, D. P. Ritsos and A. S. Gougoulis, "Standards for Augmented Reality: a User Experience perspective," in International AR Standards Meeting, 2011.Google Scholar
- "ISO 9241-210:2010: Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems," International Organization for Standardization, {Online}. Available: https://www.iso.org/standard/52075.html. {Accessed 04 07 2017}.Google Scholar
- Committee on Vision, Assembly of Behavioral and Social Sciences, National Research Council, "Procedures for Testing Color Vision: Report of," NATIONAL ACADEMY PRESS, 1981.Google Scholar
- C. H. Graham, N. R. Bartlett, J. L. Brown, C. G. Mueller, Y. Hsia and L. A. Riggs, Vision and Visual Perception, John Wiley & Sons Inc., 1965.Google Scholar
- C. Keighrey, R. Flynn, S. Murray and N. Murray, "A QoE Evaluation of Immersive Augmented and Virtual Reality Speech & Language Assessment Applications," in QoMEX 2017 - 9th International Conference on Quality of Multimedia, Erfurt, Germany, 2017.Google Scholar
- "Fitbit Charge HR," Fitbit, {Online}. Available: https://www.fitbit.com/chargehr. {Accessed 10 02 2017}.Google Scholar
- "PIP Biosensor," PIP, {Online}. Available: https://thepip.com/en-eu/. {Accessed 01 02 2017}.Google Scholar
- "ACM MM: Subjective Questionnaire," {Online}. Available: http://bit.ly/ACM-Multimedia-2017. {Accessed 11 04 2017}.Google Scholar
- ITU-T, "ITU-T P.913: Methods for the subjective assessment of video quality, audio quality and audiovisual quality of Internet video and distribution quality television in any environment," {Online}. Available: https://www.itu.int/rec/T-REC-P.913/en. {Accessed 29 04 2017}.Google Scholar
- "Unity - Game Engine," Unity3D, {Online}. Available: https://unity3d.com/. {Accessed 07 04 2017}.Google Scholar
- "Microsoft HoloLens," Microsoft, {Online}. Available: https://www.microsoft.com/microsoft-hololens/en-us. {Accessed 02 02 2017}.Google Scholar
- "Oculus Rift Development Kit 2," Oculus, {Online}. Available: https://www.oculus.com/en-us/dk2/. {Accessed 02 02 2016}.Google Scholar
- "Leap Motion," Leap Motion, {Online}. Available: https://www.leapmotion.com/. {Accessed 07 04 2017}.Google Scholar
- D. G. Kilpatrick, "Differential responsiveness of two electrodermal indices to psychological stress and performance of a complex cognitive task," Psychophysiology, vol. 9, no. 2, pp. 218--226, 1972.Google ScholarCross Ref
- Fitbit, "Help article: How accurate are Fitbit trackers?," Fitbit, 25 05 2017. {Online}. Available: https://help.fitbit.com/articles/en_US/Help_article/1136#wrist. {Accessed 25 05 2017}.Google Scholar
- "IBM SPSS - IBM Analytics," IBM, {Online}. Available: https://www.ibm.com/analytics/us/en/technology/spss/. {Accessed 07 04 2017}.Google Scholar
- I. Goradia, J. Doshi and L. Kurup, "A Review Paper on Oculus Rift & Project Morpheus," International Journal of Current Engineering and Technology, vol. 4, no. 5, pp. 3196--3200, 2014.Google Scholar
Recommendations
Exhibition approach using an AR and VR pillar
SA '17: SIGGRAPH Asia 2017 Mobile Graphics & Interactive ApplicationsThis demonstration presents a development of an Augmented Reality (AR) and Virtual Reality (AR) pillar, a novel approach for showing AR and VR content in a public setting. A pillar in a public exhibition venue was converted to a four-sided AR and VR ...
Requirements for Virtualization of AR Displays within VR Environments
Proceedings, Part I, of the 6th International Conference on Virtual, Augmented and Mixed Reality. Designing and Developing Virtual and Augmented Environments - Volume 8525Everybody has been talking about new emerging products in augmented reality AR and their potential to enhance our daily life and work. The AR technology has been around for quite a while and various use cases have been thought and tested. Clearly, the ...
The First Workshop on Multiple Input Modalities and Sensations for VR/AR Interactions (MIMSVAI)
UbiComp/ISWC '21 Adjunct: Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable ComputersWith the advance of VR/AR technology, more and more VR/AR applications are emerging and have been popular among new users. Users interact with both physical or virtual worlds through multiple input modalities and sensations to create an immersive VR/AR ...
Comments