Skip to main content
SpringerLink
Log in

The contribution of virtual reality to research on sensory feedback in remote control

  • Original Article
  • Published:
Virtual Reality Aims and scope Submit manuscript

Abstract

Here we consider research on the kinds of sensory information most effective as feedback during remote control of machines, and the role of virtual reality and telepresence in that research. We argue that full automation is a distant goal and that remote control deserves continued attention and improvement. Visual feedback to controllers has developed in various ways but autostereoscopic displays have yet to be proven. Haptic force feedback, in both real and virtual settings, has been demonstrated to offer much to the remote control environment and has led to a greater understanding of the kinesthetic and cutaneous components of haptics, and their role in multimodal processes, such as sensory capture and integration. We suggest that many displays using primarily visual feedback would benefit from the addition of haptic information but that much is yet to be learned about optimizing such displays.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.

Similar content being viewed by others

References

  1. Schmitt VR, Morris JW, Jenney GD (1998) Fly-by-wire. Society of Automotive Engineers Philadelphia

    Google Scholar 

  2. Wallace JC, Vodanovich SJ, Restino R (2003) Predicitng cognitive failures from boredom proneness and daytime sleepiness scores: an investigation within military and undergraduate samples. Pers Individ Dif 34:635–644

    Article  Google Scholar 

  3. Diolaiti N, Melchiorri C (2002) Tele-operation of a mobile robot through haptic feedback. In: HAVE, IEEE international workshop on haptic virtual environments and their applications

  4. Bjelland HV, Roed BK, Hoff T (2005) Studies on throttle sticks in high speed crafts—haptics in mechanical, electronic and haptic feedback interfaces. In: Proceedings of world haptics conference. IEEE Computer Society, Los Alamitos, pp 509–510

  5. Rastogi A, Milgram P, Drascic D (1996) Telerobotic control with stereoscopic augmented reality. In: Bolas M, Fisher S, Merritt J (eds) Stereoscopic displays and virtual reality systems, vol III. Proc SPIE 2635:115–122

  6. Gupta R, Sheridan T, Whitney D (1997) Experiments using multimodal virtual environments in design for assembly analysis. Presence Teleop Virtual Environ 6(3):318–338

    Google Scholar 

  7. Hainsworth DW (2001) Teleoperation user interfaces for mining robotics. Auton Robots 11(1):19–28

    Article  MATH  Google Scholar 

  8. Elhajj I, Xi N, Fung WK, Liu YH, Li WJ, Kaga T, Fukuda T (2001) Haptic information in internet-based teleoperation. IEEE/ASME Transactions on Mechatronics 6(3): 295–304

    Article  Google Scholar 

  9. Roberts JW, Slattery OT, Swope B, Volker M, Comstock T (2002) Small-scale tactile graphics for virtual reality systems. In: Woods AJ, Merrit JO, Benton SA, Bolas MT (eds) Stereoscopic displays and virtual reality systems, IX. Proc SPIE 4660:422–429

  10. Grohn M (2002) Is audio useful in immersive visualization? In: Woods AJ, Merrit JO, Benton SA, Bolas MT (eds) Stereoscopic displays and virtual reality systems, IX. Proc SPIE 4660:411–421

  11. Edwards GW, Barfield W, Nussbaum MA (2004) The use of force feedback and auditory cues for performance of an assembly task in an immersive virtual environment. Virtual Real 7:112–119

    Article  Google Scholar 

  12. Sallnas EL Rassmus-Grohn K, Sjostrom C (2001) Supporting presence in collaborative environments by haptic feedback. ACM Trans Comput–Hum Interact 7(4):461–476

    Article  Google Scholar 

  13. Kazi A (2001) Operator performance in surgical telemanipulation. Presence 10:495–510

    Article  Google Scholar 

  14. Raspolli M, Avizzano CA, Facenza G, Bergamasco M (2005) HERMES: an angioplasty surgery simulator. In: Proceedings of world haptics conference. IEEE Computer Society, Los Alamitos, pp 148–156

  15. Jiang L, Girotra R, Cutkosky MR, Ullrich C (2005) Reducing error rates with low-cost haptic feedback in virtual reality-based training applications. In: Proceedings of world haptics conference. IEEE Computer Society, Los Alamitos, pp 420–425

  16. White BW, Saunders FA, Scadden L, Bach-y-Rita P, Collins CC (1970) Seeing with the skin. Percept Psychophys 7:23–27

    Google Scholar 

  17. Kennedy JM, Richardson BL, Magee LE (1980) The nature of haptics. In: Hagen M (ed) The perception of pictures. Academic, New York

    Google Scholar 

  18. Baier H, Buss M, Freyberger F, Hoogen J, Kammermeier P, Schmidt G (1999) Distributed PC-based haptic, visual and acoustic telepresence system experiments in virtual and remote environments. In: Proceedings of IEEE virtual reality conference, p 118

  19. Miner N, Gillespie B, Caudell T (1996) Examining the influence of audio and visual stimuli on a haptic interface. In: Proceedings IMAGE conference, pp 23–35

  20. Grane C, Bengtsson P (2005) Menu selection with a rotary device founded on haptic and/or graphic information. In: Proceedings of world haptics conference. IEEE Computer Society, Los Alamitos, pp 475–476

  21. Palmer S (2002) Vision sciences. Bradford Books Cambridge

    Google Scholar 

  22. Goldstein BE (1999) Sensation and perception 5th edn. Brooks Cole, Pacific Grove

    Google Scholar 

  23. Bradshaw MF, Elliot KM, Watt SJ, Davies IR (2002) Do observers exploit binocular disparity information in motor tasks within dynamic telepresence environments? In: Woods AJ, Merrit JO, Benton SA, Bolas MT (eds) Stereoscopic displays and virtual reality systems, IX. Proc SPIE 4660:331–342

  24. Schmit A, Grasnik A (2002) Multiviewpoint autostereoscopic displays from 4D-vision. In: Woods AJ, Merrit JO, Benton SA, Bolas MT (eds) Stereoscopic displays and virtual reality systems, IX. Proc SPIE 4660:212–221

  25. Perlin K, Paxia S, Kollin J (2000) An autostereoscopic display. In: Proceedings of SIG-GRAPH, ACM conference on computer graphics and interactive techniques, pp 319–326

  26. McKnight S, Melder N, Barrow AL, Harwin WS, Wann JP (2005) Perceptual cues for orientation in a two finger haptic grasp task. In: Proceedings of World Haptics Conference. IEEE Computer Society, Los Alamitos, pp 549–550

  27. Richardson BL, Wuillemin DB, Symmons MA (2004) Sensory feedback and remote control of machines in mining and extraterrestrial environments. J Aust Inst Mining Metall 2:53–56

    Google Scholar 

  28. Woods A (2003) Seeing in depth at depth. Newsletter of the Centre for Marine Science & Technology, Sept. Available at: http://www.cmst.curtin.edu.au/brochures/cmstnewsletter4.pdf

  29. Kugah DA (1972) Experiments evaluating compliance and force feedback effect on manipulator performance. Genral Elec Corp NASA – CR 128605 Philadelphia

  30. Gunn C, Hutchins M, Adcock M, Hawkins R (2003) Trans-world Haptic collaboration, In: Proceedings of the SIGGRAPH Conference, Sketches and Applications, p 1

  31. Oakley I, O’Modhrain S (2005) Tilt to scroll: evaluating a motion based vibrotactile mobile interface. In: Proceedings of World Haptics Conference. IEEE Computer Society, Los Alamitos, pp 40–49

  32. Nojima T, Funabiki K (2005) Cockpit display using tactile sensations. In: Proceedings of World Haptics Conference. IEEE Computer Society, Los Alamitos, pp 501–502

  33. Bhargava A, Scott M, Traylor R, Chung R, Mrozek K, Wolter J, Tan HZ (2005) Effect of cognitive load on tactor location identification in zero-g. In: Proceedings of World Haptics Conference. IEEE Computer Society, Los Alamitos, pp 56–62

  34. Rauterberg M (1999) New directions in user-system interaction: augmented reality, ubiquitous and mobile computing. In: Proceedings of IEEE symposium on human interfacing, pp 105–133

  35. Lindeman RW, Sibert JL, Mendez-Mendez E, Patil S, Phifer D (2005) Effectiveness of directional vibrotactile cuing on a building-clearing task. In: Proceedings of ACM CHI, pp 271–280

  36. Richardson BL, Wuillemin DB, Saunders F (1978) Tactile discrimination of competing sounds. Percept Psychophys 24: 546–550

    PubMed  Google Scholar 

  37. Hoffman H, Groen J, Rousseau S, Hollander A, Winn W, Wells M, Furness T (1996) Tactile augmentation: enhancing presence in virtual reality with tactile feedback from real objects. In: Meeting of the American Psychological Society, San Francisco. Available at: http://www.hitl.washington.edu/publications/p-96–1/

  38. McGrath BJ, Estrada A, Braithwaite MG, Raj AK, Rupert AH, (2004). Tactile situation awareness system flight demonstration final report USAARL Report 2004–10, March

  39. See details of cybergrasp at: http://www.immersion.com/3d/products/cyber_grasp.php

  40. Richardson BL, Wuillemin DB, Symmons MA, Accardi R (2005) The Exograsp delivers tactile and kinaesthetic information about virtual objects. In: IEEE Tencon conference, November, Melbourne

  41. Kammermeier P, Kron A, Hoogen J, Schmit G (2004) Display of holistic sensation by combined tactile and kinaesthetic feedback. Presence 13(1):1–15

    Article  Google Scholar 

  42. Seay AF, Krum DM, Hodges L, Ribarsky W (2001) Simulator sickness and presence in a high FOV virtual environment. In: Proceedings of the virtual reality 2001 conference, pp 299–300

  43. Stott R (2002) Interaction between the senses: vision and the vestibular system. In: Roberts D (ed) Signals and perception. Palgrave Macmillan, New York

    Google Scholar 

  44. McGurk H, MacDonald T (1976) Hearing lips and seeing voices. Nature 264:746–748

    Article  PubMed  Google Scholar 

  45. Kohlers P, von Grunau M (1976) Shape and color in apparent motion. Vis Res 16:329–335

    Article  PubMed  Google Scholar 

  46. Soto-Faraco S, Spence C, Kingstone A (2004) Cross-modal dynamic capture: congruency effects in the perception of motion across sensory modalities. J Exp Psychol Hum Percept Perform 30(2):330–345

    Article  Google Scholar 

  47. Shiffman HR (1996) Sensation and perception, 4th edn. Wiley, New York

    Google Scholar 

  48. Heller MA (1983) Haptic dominance in form perception with blurred vision. Perception 122:607–613

    Article  Google Scholar 

  49. Soto-Faraco S, Kingstone A (2004) Multisensory integration of dynamic information. In: Calvert G, Spence C, Stein B (eds) Handbook of multisensory processes. MIT Press Cambridge

    Google Scholar 

  50. Caclin A, Soto-Faraco S, Kingstone A, Spence C (2002) Tactile “capture” of audition. Percept Psychophys 18:55–60

    Google Scholar 

  51. Richardson BL, Symmons MA, Accardi R (2000) The TDS: A new device for comparing active and passive-guided touch. IEEE Trans Rehabilit Eng 8:414–417

    Article  Google Scholar 

  52. Symmons MA, Richardson BL, Wuillemin DB, VanDoorn GH (2005) Kinaesthetic and cutaneous contributions to raised-line stimulus interpretation. In: World haptics conference, 18–20 March, Pisa. Video clip at http://www-personal.monash.edu.au/~msymmons/images/6_9_qt.mov

  53. Magee LE, Kennedy JM (1980) Exploring pictures tactually. Nature (London) 283:287

    Article  Google Scholar 

  54. Richardson BL, Symmons M, Wuillemin DB (2004) The relative importance of cutaneous and kinesthetic cues in raised line drawing exploration. In: Ballesteros S, Heller MA (eds) Touch, blindness, and neuroscience. Universidad Nacional de Educación a Distancia, Madrid, pp 247–250

    Google Scholar 

  55. Symmons M, Richardson BL, Wuillemin DB (2004) Active versus passive touch: Superiority depends more on the task than the mode. In: Ballesteros S, Heller MA (eds) Touch, blindness, and neuroscience. Universidad Nacional de Educación a Distancia, Madrid, pp 179–185

    Google Scholar 

  56. Wuillemin DB, VanDoorn GH, Richardson BL, Symmons MA (2005) Haptic and visual size judgements in virtual and real environments. In: Proceedings of world haptics conference. IEEE Computer Society, Los Alamitos, pp 86–89

  57. Sarter N (1998) Turning automation into a teamplayer: The development of multisensory and graded feedback for highly automated (flight deck) systems. Willard Airport Aviation Research Lab, http://www.nsfworkshop.engr.ucf.edu/papers/Sarter.asp/

  58. Lee S, Sukhatme GS, Kim GJ, Park C (2005) Haptic teleoperation of a mobile robot: a user study. Presence 14(3):345–365

    Article  Google Scholar 

  59. Bliss JC, Katcher MH, Roger CH, Shepard R (1970) Optical-to-tactile image conversion for the blind. IEEE Trans Man–Mach Syst MMS-11:58–64

    Article  Google Scholar 

  60. Brooks PL, Frost BJ, Mason JL, Gibson DM (1987) Word and feature identification by profoundly deaf teenagers using the Queen’s University Tactile Vocoder. J Speech Hear Res 30:137–141

    PubMed  Google Scholar 

  61. Geldard FA (1966) Cutaneous coding of optical signals: the Optohapt. Percept Psychophys 1:377–381

    Google Scholar 

  62. Richardson BL, Frost BJ (1977) Sensory substitution and the design of an artificial ear. J Psychol 96:259–285

    PubMed  Google Scholar 

  63. Gallace A, Tan HZ, Spence C (2005) Tactile change detection. In: Proceedings of world haptics conference. IEEE Computer Society, Los Alamitos, pp 12–16

  64. van Erp JBF (2005) Vibrotactile spatial acuity on the torso: effects of location and timing parameters. In: Proceedings of world haptics conference. IEEE Computer Society, Los Alamitos, pp 80–85

Download references

Acknowledgements

The research conducted at the Bionics and Cognitive Science Centre of Monash University was supported by a grant from the Australian Commonwealth Government’s Sustainable Regions Programme.

Author information

Authors and Affiliations

  1. Bionics and Cognitive Science Centre, Monash University, Gippsland Campus, 3842, Churchill, Australia

    Barry Richardson, Mark Symmons & Dianne Wuillemin

Authors
  1. Barry Richardson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mark Symmons
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dianne Wuillemin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Barry Richardson.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Richardson, B., Symmons, M. & Wuillemin, D. The contribution of virtual reality to research on sensory feedback in remote control. Virtual Reality 9, 234–242 (2006). https://doi.org/10.1007/s10055-006-0020-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10055-006-0020-z

Keywords

Search

Navigation