Abstract
Much effort has gone into exploring the concept of presence in virtual environments. One of the reasons for this is the possible link between presence and performance, which has also received a fair amount of attention. However, the performance side of this equation has been largely ignored. That is, without much discussion, researchers tend to equate performance with measured performance on some specific task. But if presence is a measure of overall acceptance of the virtual experience can we get away with any less than assessing all aspects of user performance? This is a problem as we can neither measure everything subjects do nor determine the weights or importance of all subtasks carried out during task completion by any given user. The way around this may be to ask subjects to assess their own performance, call it subjective performance. This notion is explored in the context of an experiment that investigated the presence–performance relationship by decoupling the variation in spatial audio fidelity (the one independent variable) from its relevance to the search task at hand (the other independent variable). It was found that while presence showed no correlation with task time presence did exhibit a fairly strong correlation with subjective performance.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Originally constructed, using the layout from the classic board game Cluedo, by A. Steed, UCL London, for use in experiments by the collaborative virtual environments (COVEN) project [9].
SAS Institute’s statistics package was used for the variance analysis [14].
The ‘I Mastered The VE’ questionnaire has most items in common with the Witmer and Singer Questionnaire. These items have been eliminated from the Witmer and Singer PQ before calculating the correlation. If one does not do this, one has p=0.0003.
As the Witmer and Singer questionnaire gauges such factors this has let to claims that it is not a proper presence questionnaire [12, 13]. Engaging in this discussion is beyond the scope of this paper. But whether a good presence measure or not, it does capture a number of interesting facets of the subjective perception of VEs.
An oft noted requirement that is invalid, or irrelevant, if the input devices were properly registered within the VE.
References
Welch R (1999) How can we determine if the sense of presence affects task performance. Presence 8.5:574–577
Witmer B, Singer M (1998) Measuring presence in virtual environments: a presence questionnaire. Presence 7.3:225–240
Bailey J, Witmer B (1994) Learning and transfer of spatial knowledge in a virtual environment. proc. In: Human factors and ergonomics society 38th annual meeting, pp 1158–1162
Swedish Institute Of Computer Science (1999) DIVE 3.3, Distributed virtual environment. http://www.sics.se/dive
Adler D (1996) Virtual audio: three-dimensional audio in virtual environments. Technical report T96:03, Swedish Institute of Computer Science (ISRN: SICS-T–96/03-SE). http://www.ftp.sics.se/pub/SICS-reports/Reports/SICS-T-96-03-SE.ps.Z
Freeman J, Avons S, Pearson D, IJsselsteijn W (1999) Effects of sensory information and prior experience on direct subjective ratings of presence. Presence 8.1:1–13
Bormann K (2002) Context calibration. Virtual Real Res Dev Appl 6.1:45–55
Bormann K (2005) Presence and the utility of audio spatialization. Presence 15.3:278–297
Normand V (1999) The COVEN project: Exploring applicative, technical, and usage dimensions of collaborative virtual environments. Presence 8.2:218–236
Uno S, Slater M (1997) The sensitivity of presence to collision response. In: Proc VRAIS’97, pp 95–103
Tromp J, Bullock A, Steed A, Sadagic A, Slater M, Frecon E (1998) Small group behavior experiments in the Coven project. Comput Graph Appl Nov/Dec:53–63
Slater M (1999) Measuring presence: a response to the witmer and singer presence questionnaire. Presence 8.5:560–565
Singer M (1999) On selecting the right yardstick. Presence 8.5:566–573
The SAS System 8e, SAS Institute 2001
Banerjee A, Banerjee P, Ye N, Dech F (1999) Assembly Planning Effectiveness Using Virtual Reality. Presence 8.2:204–217
Sun H, Hujun B, Man T, Fai W (1999) Interactive Task Planning in Virtual Assembly. Proc. VRST’99, 174–175
Loomis J (1992) Distal attribution and presence. Presence 1.1:113
Sheridan T (1992) Musings on telepresence and virtual presence. Presence 1.1:120
Zahorik P, Jenison R (1998) Presence as being-in-the-world. Presence 7.1:78
Gibson J (1979) The ecological approach to visual perception. Houghton-Muffin
Warren W (1984) Perceiving affordances: Visual guidance of stair climbing. J Exp Psychol Hum Percept Perform 10.5:683–703
Mark L (1987) Eyeheight scaled information about affordances: a study of sitting and stair climbing. J Exp Psychol Hum Percept Perform 13:361–370
Smets G, Stappers P, Overbeeke K, der Mast C (1994) Designing in virtual reality: implementing perception-action coupling with affordances. In: Proceedings of VRST’94, p 97
Smets G, Stappers P, Overbeeke K, van der Mast C (1995) Designing in virtual reality: perception–action coupling and affordances. In: Carr K, England R (eds) Simulated and virtual realities—elements of perception. Taylor & Francis, London
Author information
Authors and Affiliations
Appendix
Appendix
1.1 Questionnaire
Questionnaire item | Subgroup(s) |
|---|---|
1. Overall, how enjoyable did you find the game? | |
2. To what extent was there a sense of actually searching a (real) house to find dominoes? | SlaterPQ |
3. How responsive was the virtual environment to your actions? | W and S PQ |
4. How natural did your interactions with the computer-generated environment feel? | W and S PQ |
5. Were you able to anticipate what would happen in response to the actions that you performed? | W and S PQ |
6. How, much delay did you experience between your actions and the (expected) outcomes? | W and S PQ |
7. How natural was the mechanism which controlled movement through the environment? | W and S PQ |
8. How compelling was your sense of moving through space? | W and S PQ |
9. Was the experience of the house more like watching TV or more like being in a house? | Slater PQ |
10. How proficient in moving through and interacting with the virtual environment did you feel (at the end of the experience)? | W and S PQ |
11. How much did your experiences in the virtual environment seem consistent with your real-world experiences? | W and S PQ |
12. How completely were you able to actively survey or search the environment using vision? | W and S PQ |
13. How closely were you able to examine objects? | W and S PQ |
14. How well could you examine objects from multiple viewpoints? | W and S PQ |
15. How much did the visual display quality interfere or distract you from performing assigned tasks or required activities? | W and S PQ |
16. To what extent were there times during the game when the computer-generated environment became reality for you, and you almost forgot about the the (real) world around | SlaterPQ |
17. How well could you localize sounds (the playing radio)? | W and S PQ |
18. To what extent was the playing radio useful for establishing your orientation in the virtual world? | |
19. Rate the usefulness of visuals versus audio as regards its utility in task completion | |
20. If ‘1’ signifies that, during the game, you felt totally embedded in the real world, and ‘7’ signifies that you felt totally embedded in the computer model/game world, then, by a number between 1 and 7, signify whether you felt most embedded in the real or in the game world. | Slater PQ |
21. How much did the auditory aspects of the environment involve you? | W and S PQ |
22. How much did the visual aspects of the environment involve you? | W and S PQ |
23. How involved were you in the game? | W and S PQ |
24. How quickly did you adjust to the virtual environment? | W and S PQ |
25. How well could you concentrate on the assigned task rather than on the mechanisms used to perform the task (i.e. the computer interface)? | W and S PQ |
26. Is the software well suited for the game task? | |
27. Is the virtual house in which the game took place remembered in a way more similar to somewhere visited than to images seen on a (computer) screen? | SlaterPQ |
28. How mentally alert do you feel at the present time? | |
29. How physically fit do you feel at the present time? | |
30. Was the experiment mentally tiring? | |
31. In the introduction to the game, two search strategies were mentioned; a) move fast to quickly search the house, and b) try to remember which dominoes were seen where, in order to search more efficiently. Which of the two strategies is closest to the strategy you chose? | |
32. To what extent did you remember which room’s dominoes had already been found, thus avoiding these rooms when searching for another domino? | |
33. To what extent did you remember that the domino you were searching for had already been seen, thus making you able to search for a (known) room, rather than for a domino? | |
34. To what extent were you able to remember the layout of the house, so that you, generally, could keep your orientation (and, if you were looking for a particular room; had an idea about where to find it)? |
Rights and permissions
About this article
Cite this article
Bormann, K. Subjective performance. Virtual Reality 9, 226–233 (2006). https://doi.org/10.1007/s10055-006-0019-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10055-006-0019-5