Skip to main content

Advertisement

SpringerLink
Log in

Effects of sound on visual realism perception and task performance

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Before the application of virtual simulations and serious games for surgical education and training becomes more widespread, there are a number of open questions and issues that must be addressed including the relationship between realism, multi-modal cue interaction, immersion, and knowledge transfer and retention. Using the serious game surgical cognitive education and training framework developed specifically for cognitive surgical skills training, here we examine the effect of sound on visual realism perception and task completion time while performing a task within a virtual environment. Our preliminary experimental results indicate that the appropriate use of sound can lead to performance improvements when performing a task within a virtual environment without a corresponding decrease in the perception of visual realism.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Halsted, W.S.: The Training of the Surgeon, vol. xv, pp. 267–275. Bull Johns Hopkins Hosp, USA (1904)

    Google Scholar 

  2. Reznick, R.K.: Teaching and testing technical skills. Am. J. Surg. 165(3), 358–361 (1993)

    Article  Google Scholar 

  3. Gorman, P.J., Meier, A.H., Rawn, C., Krummel, T.M.: The future of medical education is no longer blood and guts, it is bits and bytes. Am. J. Surg. 180, 353–356 (2000)

    Article  Google Scholar 

  4. Kohls-Gatzoulis, J.A., Regehr, G., Hutchison, C.: Teaching cognitive skills improves learning in surgical skills courses: a blinded, prospective, randomized study. Can. J. Surg. 47(4), 277–283 (2004)

    Google Scholar 

  5. Corti, K.: Game-Based Learning: A Serious Business Application. PIXE-Learning, Coventry (2006)

    Google Scholar 

  6. Shute, V., Ventura, M., Bauer, M., Zapata-Rivera, D.: Melding the power of serious games and embedded assessment to monitor and foster learning. In: Ritterfeld, U., Cody, M.J., Vorderer, P. (eds.) Serious Games. Mechanisms and Effects, pp. 295–321 (2009)

  7. Tashiro, J., Dunlap, D.: The impact of realism on learning engagement in educational games. In: Proceedings of the ACM Future Play 2007 Conference, Toronto, pp. 113–120 (2007)

  8. Farmer, E., Rooij, J., Riemersma, J., Joma, P., Morall, J.: Handbook of Simulator Based Training. Ashgate Publishing, Surrey (1999)

    Google Scholar 

  9. Cook, D.A., Hamstra, S.J., Brydges, R., Zendejas, B., Szostek, J.H., Wang, A.T., Erwin, P.J., Hatala, R.: Comparative effectiveness of instructional design features in simulation-based education: systematic review and meta-analysis. Med. Teach. 35(1), e867–e898 (2013)

    Article  Google Scholar 

  10. Muchinsky, P.M.: Psychology Applied to Work, 9th edn. Hypergraphic Press, Summerfield (1999)

    Google Scholar 

  11. Godden, D.R., Baddeley, A.D.: Context dependent memory in two natural environments: on land and underwater. Brit. J. Psychol. 66(3), 325–331 (1975)

    Article  Google Scholar 

  12. Norman, G., Dore, K., Grierson, L.: The minimal relationship between simulation fidelity and transfer of learning. Med. Educ. 46(7), 636–647 (2012)

    Article  Google Scholar 

  13. Hulusic, V., Aranha, M., Chalmers, A.: The influence of cross-modal interaction on perceived rendering quality thresholds. In: Proceedings of Computer Graphics, Visualization and Vision 2008, Plzen - Bory, Czech Republic, pp. 41–48 (2008)

  14. Blascovich, J., Bailenson, J.: Infinite Reality. Harper Collins, New York (2011)

    Google Scholar 

  15. Howard, I.P., Templeton, W.B.: Human Spatial Orientation. John Wiley and Sons, Ney York (1966)

    Google Scholar 

  16. McGurk, H., Macdonald, J.: Hearing lips and seeing voices. Nature 264, 746–748 (1976)

    Article  Google Scholar 

  17. Rojas, D., Kapralos, B., Crsitancho, S., Collins, K., Conati, C., Dubrowski, A.: The effect of background sound on visual fidelity perception. In: Proceedings of the ACM Audio Mostly Conference 2011, Coimbra (2011)

  18. Rojas, D., Kapralos, B., Cristancho, S., Collins, K., Hogue, A., Conati, C., Dubrowski, A.: Developing effective serious games: the effect of background sound on visual fidelity perception with varying texture resolution. Stud. Health Technol. Inform. 173, 386–392 (2012)

    Google Scholar 

  19. Rojas, D., Kapralos, B., Hogue, A., Collins, K., Nacke, L., Crsitancho, S., Conati, C., Dubrowski, A.: The effect of ambient auditory conditions on visual fidelity perception in stereoscopic 3D. IEEE Trans. Syst. Man. Cy. B 43(6), 1572–1583 (2013)

    Google Scholar 

  20. Rojas, D., Kapralos, B., Collins, K., Dubrowski, A.: The effect of contextual sound cues on visual fidelity perception. Stud. Health Technol. Inform. 196, 346–352 (2014)

    Google Scholar 

  21. Cowan, B., Sabri, H., Kapralos, B., Porte, M., Backstein, D., Cristancho, S., Dubrowski, A.: A serious game for total knee arthroplasty procedure education and training. J. Cyberther. Rehabil. 3(3), 285–298 (2010)

    Google Scholar 

  22. Kapralos, B., Moussa, F., Dubrowski, A.: An overview of virtual simulations and serious games for surgical education and training. In Brooks, A., Braham, S., Jain, L. (eds.) Serious Games, Alternative Realities, and Play Therapy. Springer Series Studies in Computational Intelligence Ch.14, pp. 289–306 (2014)

  23. Mastoropoulou, G., Debattista, K., Chalmers, A., Troscianco, T.: The influence of sound effects on the perceived smoothness of rendered animations. In: Proceedings of the 2nd Symposium on Applied Perception in Graphics and Visualization, La Coruna, pp. 9–15 (2005)

  24. Larsson, P., Västjäll, D., Kleiner, M.: On the quality of experience: a multi-modal approach to perceptual ego-motion and sensed presence in virtual environments. In: Proceedings of the First International Speech Communications Association Tutorial and Research Workshop on Auditory Quality of Systems, Akademie Mont-Cenis (2003)

  25. Bertelson, P., Radeau, M.: Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Percept. Psychophys. 29(6), 578–584 (1981)

    Article  Google Scholar 

  26. Hulusic, V., Debattista, K., Aggarwal, V., Chalmers, A.: Maintaining frame rate perception in interactive environments by exploiting audio-visual cross-modal interaction. Visual Comput. 27(1), 57–66 (2011)

    Article  Google Scholar 

  27. Hulusic, V., Czanner, G., Debattista, K., Sikudova, E., Dubla, P., Chalmers.: Investigation of the beat rate effect on frame rate for animated content. In: Proceedings of the 25th Spring Conference on Computer Graphics, Budmerice (2009)

  28. Bonneel, N., Suied, C., Viaud-Delmon, I., Drettakis, G.: Bimodal perception of audio-visual material properties for virtual environments. ACM Trans. Appl. Percept. 7(1), 1–16 (2010)

    Article  Google Scholar 

  29. Woods, A.T., Poliakoff, E., Lloyd, D.M., Kuenzela, J., Hodsona, J.R., Gondaa, H., Batchelora, J., Dijksterhuis, G.B., Thomas, A.: Effect of background noise on food perception. Food Qual. Prefer. 22(1), 42–47 (2011)

    Article  Google Scholar 

  30. Chang, R.S., Thompson, N.S.: The attention-getting capacity of whines and child-directed speech. Evol. Psychol. 8(2), 260–274 (2010)

    Article  Google Scholar 

  31. Conrad, C., Konuk, Y., Werner, P., Cao, C.G., Warshaw, A., Rattner, D., Jones, D.B., Gee, D.: The effect of define auditory conditions versus mental loading on the laparoscopic motor skill performance of experts. Surg. Endosc. 24(6), 1347–1352 (2010)

    Article  Google Scholar 

  32. Praamsma, M., Carnahan, H., Backstein, D., Veillette, C.J., Gonzalez, D., Dubrowski, A.: Drilling sounds are used by surgeons and intermediate residents, but not novice orthopedic trainees, to guide drilling motions. Can. J. Surg. 51(6), 442–446 (2008)

    Google Scholar 

  33. Hulusic, V., Harvey, C., Debattista, K., Tsingos, N., Walker, S., Howard, D., Chalmers, A.: Acoustic rendering and auditory-visual cross-modal perception and interaction. Comput. Graph. Forum 31(1), 102–131 (2012)

    Article  Google Scholar 

  34. Shams, L., Kim, R.: Crossmodal influences on visual perception. Phys. Life Rev. 7(3), 295–298 (2010)

    Article  Google Scholar 

  35. Morfey, C.: The Dictionary of Acoustics. Academic Press, San Diego (2001)

    Google Scholar 

  36. Suied, C., Bonneel, N., Viaud-Delmon, I.: Integration of auditory and visual information in the recognition of realistic objects. Exp. Brain Res. 194(1), 91–102 (2009)

    Article  Google Scholar 

  37. Treisman, A.M., Riley, J.G.: Is selective attention selective perception or selective response? A further test. J. Exp. Psychol. 79(1), 27–34 (1969)

    Article  Google Scholar 

  38. Paas, F., Renkl, A., Sweller, J.: Cognitive load theory and instructional design: recent developments. Educ. Psychol. 38(1), 1–4 (2003)

    Article  Google Scholar 

  39. Faure, F., Duriez, C., Delingette, H., Allard, J., Gilles, B., Marchesseau, Talbot, H., Courtecuisse, H., Bousquet, G., Peterlik, I., Cotin, S.: SOFA: a multi-model framework for interactive physical simulation. In: Payan, Y. (ed.) Soft Tissue Biomechanical Modeling for Computer Assisted Surgery, vol. 11, pp. 283–321. Springer, Berlin (2012)

    Chapter  Google Scholar 

  40. France, L., Lenoir, J., Angelidis, A., Meseure, P., Cani, M.P., Faure, F., Chaillou, C.: A layered model of a virtual human intestine for surgery simulation. Med Image Anal. 9(2), 123–132 (2005)

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the Natural Sciences and Engineering Research Council of Canada (NSERC), the Social Sciences and Humanities Research Council of Canada (SSHRC), Interactive & Multi-Modal Experience Research Syndicate (IMMERSe) initiative, and the Canadian Network of Centres of Excellence (NCE), Graphics, Animation, and New Media (GRAND) initiative.

Author information

Authors and Affiliations

  1. Faculty of Business and Information Technology, Health Education Technology Research Unit (HETRU), University of Ontario Institute of Technology, 2000 Simcoe St. North, Oshawa, ON , L1H 7K4, Canada

    Brent Cowan, David Rojas & Bill Kapralos

  2. Institute of Medical Science, University of Toronto, Toronto, ON, Canada

    David Rojas

  3. Division of Cardiac and Vascular Surgery, Schulich Heart Centre, Sunnybrook Health Sciences Centre, Toronto, ON, Canada

    Fuad Moussa

  4. Divisions of Emergency Medicine and Pediatrics, Faculty of Medicine, Memorial University, St. John’s, NL, Canada

    Adam Dubrowski

Authors
  1. Brent Cowan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Rojas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bill Kapralos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fuad Moussa
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Adam Dubrowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bill Kapralos.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cowan, B., Rojas, D., Kapralos, B. et al. Effects of sound on visual realism perception and task performance. Vis Comput 31, 1207–1216 (2015). https://doi.org/10.1007/s00371-014-1006-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-014-1006-6

Keywords

Search

Navigation