Skip to main content
SpringerLink
Log in

The influence of different sensory cues as selection feedback and co-location in presence and task performance

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

For some applications based on virtual reality technology, presence and task performance are important factors to validate the experience. Different approaches have been adopted to analyse the extent to which certain aspects of a computer-generated environment may enhance these factors, but mainly in 2D graphical user interfaces. This study explores the influence of different sensory modalities on performance and the sense of presence experienced within a 3D environment. In particular, we have evaluated visual, auditory and active haptic feedback for indicating selection of virtual objects. The effect of spatial alignment between proprioceptive and visual workspaces (co-location) has also been analysed. An experiment has been made to evaluate the influence of these factors in a controlled 3D environment based on a virtual version of the Simon game. The main conclusions obtained indicate that co-location must be considered in order to determine the sensory needs during interaction within a virtual environment. This study also provides further evidence that the haptic sensory modality influences presence to a higher extent, and that auditory cues can reduce selection times. Conclusions obtained provide initial guidelines that will help designers to set out better selection techniques for more complex environments, such as training simulators based on VR technology, by highlighting different optimal configurations of sensory feedback.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Bailey JH, Witmer BG (1994) Learning and transfer of spatial knowledge in a virtual environment. In Proc. of the Human Factors and Ergonomics Society 38th Annual Meeting, pp 1158–1162

  2. Bertelson P (1999) Ventriloquism: a case of crossmodal perceptual grouping. In: Aschersleben G, Bachmann T, Musseler J (eds) Cognitive contributions to the perception of spatial and temporal events. Elsevier, North-Holland, pp 347–363

    Chapter  Google Scholar 

  3. Biocca F, Kim J, Choi Y (2001) Visual touch in virtual environments: an exploratory study of presence, multimodal interfaces, and cross-modal sensory illusions. Presence-Teleoper Virtual Env 10:247–265. doi:10.1162/105474601300343595

    Article  Google Scholar 

  4. Bormann K (2006) Subjective performance. Virtual Reality 9:226–233. doi:10.1007/s10055-006-0019-5

    Article  Google Scholar 

  5. Bowman DA, Hodges LF (1997) An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments. In Proc. of Symposium on Interactive 3D Graphics, pp. 35–38. ACM, NY, USA

  6. Burke JL, Prewett MS, Gray AA, Yang L, Stilson FR, Coovert MD, Elliot LR, Redden E (2006) Comparing the effects of visual-auditory and visual-tactile feedback on user performance: a meta-analysis. In Proc. of the 8th international Conference on Multimodal interfaces, pp. 108–117, ACM, NY, USA

  7. Chen J, Bowman DA (2009) Domain-specific design of 3D interaction techniques: an approach for designing useful virtual environment applications. Presence-Teleoper Virtual Env 18:370–386. doi:10.1162/pres.18.5.370

    Article  Google Scholar 

  8. Coelho C, Tichon JG, Hine TJ, Wallis GM, Riva G (2006) Media presence and inner presence: the sense of presence in virtual reality technologies. In: Riva G, Anguera MT, Wiederhold BK, Mantovani F (eds) From communication to presence: cognition, emotions and culture towards the ultimate communicative experience. Ios Press, Amsterdam, pp 25–45

    Google Scholar 

  9. Congedo M, Lécuyer A, Gentaz E (2006) The Influence of spatial de-location on perceptual integration of vision and touch. Presence-Teleoper Virtual Env 15:353–357. doi:10.1162/pres.15.3.353

    Article  Google Scholar 

  10. Cumming G, Finch S (2005) Inference by eye: How to read pictures of data. Am Psychol 60:170–180

    Article  Google Scholar 

  11. Durlach P, Fowlkes J, Metevier C (2005) Effects of variations in sensory feedback on performance in a virtual reaching task. Presence-Teleoper Virtual Env 14:450–462

    Article  Google Scholar 

  12. Ellis SR, Dorighi NS, Menges BM, Adelstein BD, Jacoby RH (1997) In search of equivalence classes in subjective scales of reality. In: Smith MJ, Salvendy G, Koubek RJ (eds) Design of computing systems: social and ergonomic considerations. Elsevier, Amsterdam, pp 873–876

    Google Scholar 

  13. Ferris TK, Sarter NB (2008) Cross-modal links among vision, audition, and touch in complex environments. Hum Factors 50:17–26

    Article  Google Scholar 

  14. Hecht D, Reiner M (2006) Multimodal virtual environments: response times, attention and presence. Presence-Teleoper Virtual Env 15:515–523

    Article  Google Scholar 

  15. Hendrix C, Barfield W (1996) Presence within virtual environments as a function of visual display parameters. Presence-Teleoper Virtual Env 5:274–289

    Google Scholar 

  16. Jacko JA, Barnard L, Kongnakorn T, Moloney KP, Edwards PJ, Emery VK, Sainfort F (2004) Isolating the effects of visual impairment: exploring the effect of AMD on the utility of multimodal feedback. In Proc. of ACM CHI. 7:311–318. doi:10.1145/985692.985732

  17. Keetels M, Stekelenburg JJ, Vroomen J (2007) Auditory grouping occurs prior to intersensory pairing: evidence from temporal ventriloquism. Exp Brain Res 180:449–456. doi:10.1007/s00221-007-0881-8

    Article  Google Scholar 

  18. Kim T, Biocca F (1997) Telepresence via television: two dimensions of telepresence may have different connections to memory and persuasion. J Comput-Mediat Comm 3. doi:10.1111/j.1083-6101.1997.tb00073.x

  19. Loomis JM (1992) Distal attribution and presence. Presence-Teleoper Virtual Env 1:113–119

    Google Scholar 

  20. Ma R, Kaber DB (2006) Presence, workload and performance effects of synthetic environment design factors. Int J Hum- Comput Stud 64:541–552

    Article  Google Scholar 

  21. Maida J, Aldridge A, Novak J (1997) Effects of lighting on human performance in training. In Proc. of Seventh International Conference on Human-Computer Interaction, pp. 877–880

  22. Mori M (1970) Bukini no tani (The Uncanny Valley). Energy 7:33–35

    Google Scholar 

  23. Navarra J, Soto-Faraco S, Spence C (2007) Adaptation to audiotactile asynchrony. Neurosci Lett 413:72–76. doi:10.1016/j.neulet.2006.11.027

    Article  Google Scholar 

  24. Nunez D (2004) Working memory and presence: reconsidering the role of attention in presence. In Proc. of 7th International Workshop in Presence, pp. 44–48

  25. Pausch R, Proffitt D, Williams G (1997) Quantifying immersion in virtual reality. In Proc. of the 24th annual conference on Computer graphics and interactive techniques, pp. 13–18

  26. Poupyrev I, Weghorst S, Billinghurst M, Ichikawa T (1998) Egocentric object manipulation in virtual environments: empirical evaluation of interaction techniques. Comput Graph Forum 17:41–52

    Article  Google Scholar 

  27. Sheik-Nainar MA, Kaber DB, Chow M (2005) Control gain adaptation in virtual reality mediated human-telerobot interaction. Hum Factors Ergon Manuf 15:259–274

    Article  Google Scholar 

  28. Singer MJ, Ehrlich JA, Cinq-Mars S, Papin JP (1995) Task performance in virtual environments: Stereoscopic vs. monoscopic displays and head-coupling. Technical Report 1034, Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Science.

  29. Singer MJ, Allen RC, McDonald DP, Gildea JP (1997) Terrain appreciation in virtual environments: Spatial knowledge acquisition. Technical Report 1056. Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Science

  30. Slater M, Usoh M, Steed A (1995) Taking steps: the influence of a walking technique on presence in virtual reality. ACM Trans Comp Hum Interact 2:201–219

    Article  Google Scholar 

  31. Slater M, Steed A, McCarthy J, Maringelli F (1998) The influence of body movement on subjective presence in virtual environments. Hum Factors 40:469–477

    Article  Google Scholar 

  32. Spence C (2007) Audiovisual multisensory integration. Acoust Sci Tech 28:61–70

    Article  MathSciNet  Google Scholar 

  33. Stanney K, Salvendy G, Deisinger J, Dizio P, Ellis P, Ellison J, Fogleman G, Gallimore J, Singer M, Hettinger L, Kennedy R, Lackner J, Lawson B, Maida J, Mead A, Mon-Williams M, Newman D, Piantanida T, Reeves L, Riedel O, Stoffregen T, Wann J, Welch R, Wilson J, Witmer B (1998) Aftereffects and sense of presence in virtual environments: formulation of a research and development agenda. Int J Hum-Comput Interact 10:135–187. doi:10.1207/s15327590ijhc1002DOI:dx.doi.org

    Article  Google Scholar 

  34. Szameitat AJ, Rummel J, Szameitat DP, Sterr A (2009) Behavioral and emotional consequences of brief delays in human-computer interaction. Int J Hum-Comput Stud 67:561–570. doi:10.1016/j.ijhcs.2009.02.004

    Article  Google Scholar 

  35. Teder-Salejarvi WA, Di Russo F, McDonald JJ, Hillyard SA (2005) Effects of spatial congruity on audio-visual multimodal integration. J Cog Neurosci 17:1396–1409. doi:10.1162/0898929054985383

    Article  Google Scholar 

  36. Viciana-Abad R, Reyes-Lecuona A, García-Berdonés C, Díaz-Estrella A (2004) The importance of significant information in presence and stress within a virtual reality experience. In Proc. of Cybertherapy Conference, pp 25-32. Interactive Media Institute, CA, USA

  37. Viciana-Abad R, Reyes-Lecuona A, Cañadas-Quesada F (2005) Difficulties using passive haptic augmentation in the interaction within a virtual environment. In Proc. of Presence, pp. 287–292

  38. Viciana-Abad R, Reyes-Lecuona A (2008) Effects of Co-location and crossmodal interaction between Haptic, Auditory and visual cues in presence. Haptics: Perception, Devices and Scenarios, LNCS 5024: 832-837, Berlin: Springer. doi: 10.1007/978-3-540-69057-3

  39. Viciana-Abad R, Reyes-Lecuona A, Poyade M (2010) The influence of passive haptic feedback and difference interaction metaphors on presence and task performance. Presence-Teleoper Virtual Env 19:197–212. doi:10.1162/pres.19.3.197

    Article  Google Scholar 

  40. Viciana-Abad R, Reyes-Lecuona A, Poyade M, Escolano J (2011) The role of mismatches in the sensory feedback provided to indicate selection within a virtual environment. Multimedia Tools Appl 56:353–378. doi:10.1007/s11042-010-0551-x

    Google Scholar 

  41. Vitense HS, Jacko JA, Emery VK (2002) Foundation for improved interaction by individuals with visual impairments through multimodal feedback. Univers Access Inform Soc 2:76–87. doi:10.1007/s10209-002-0038-2

    Article  Google Scholar 

  42. Wall SA, Paynter K, Shillito AM, Wright M, Scali S (2002) The Effect of Haptic Feedback and Stereo Graphics in a 3D Target Acquisition Task. In Proc. of Eurohaptics, pp 23–28. Springer, Berlin, Germany

  43. Wallach H, Safir M, Samana R (2009) Personality variables and presence. Virtual Reality 14:3–13. doi:10.1007/s10055-009-0124-3

    Article  Google Scholar 

  44. Welch RB (1999) How can we determine if the sense of presence affects task performance? Presence-Teleoper Virtual Env 8:574–577. doi:10.1162/105474699566387

    Article  Google Scholar 

  45. Welch RB, Warren DH (1980) Immediate perceptual response to intersensory discrepancy. Psychol Bull 88:638–667. doi:10.1037/0033-2909.88.3.638

    Article  Google Scholar 

  46. Welch RB, Blackmon TT, Liu A, Mellers BA, Stark LW (1996) The effects of pictorial realism, delay of visual feedback, and observer in teractivity on the subjective sense of presence. Presence-Teleoper Virtual Env 5:263–273

    Google Scholar 

  47. Whitelock D, Brna P, Romano D, Anne J (2000) Perfect presence: what does this mean for the design of virtual environments? Educ Inform Tech 5:277–289

    Article  Google Scholar 

  48. Wickens CD (2002) Multiple resources and performance prediction. Theor Issues Ergon Sci 3:159–177. doi:10.1080/14639220210123806

    Article  Google Scholar 

  49. Wickens CD (2008) Multiple resources and mental workload. Hum Factors 50:449–455. doi:10.1234/12345678

    Article  Google Scholar 

  50. Witmer BG, Singer MJ (1994) Measuring immersion in virtual environments. Technical Report No. 1014. U.S. Army Research Institute for the Behavioral and Social Sciences, Alexandria, VA

    Google Scholar 

  51. Witmer BG, Singer MJ (1998) Measuring presence in virtual environments: a presence questionnaire. Presence-Teleoper Virtual Env 7:225–240. doi:/10.1162/105474698565686

    Article  Google Scholar 

Download references

Acknowledgments

This work has been partially supported by DIANA research group (PAI: TIC-171) and the Junta de Andalucía, under the project P11-TIC-8231. The authors would like to thank the participants of this experiment for their collaboration and comments.

Author information

Authors and Affiliations

  1. Department of Telecommunication Engineering, University of Jaén, Jaén, Spain

    Raquel Viciana-Abad & José Manuel Pérez-Lorenzo

  2. Department of Electronic Technology, University of Málaga, Málaga, Spain

    Arcadio Reyes-Lecuona & Alejandro Rosa-Pujazón

Authors
  1. Raquel Viciana-Abad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arcadio Reyes-Lecuona
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alejandro Rosa-Pujazón
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. José Manuel Pérez-Lorenzo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Raquel Viciana-Abad.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Viciana-Abad, R., Reyes-Lecuona, A., Rosa-Pujazón, A. et al. The influence of different sensory cues as selection feedback and co-location in presence and task performance. Multimed Tools Appl 68, 623–639 (2014). https://doi.org/10.1007/s11042-012-1070-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-012-1070-8

Keywords

Search

Navigation