Audio Drama and the Imagination
The Influence of Sound Effects on Presence in People With and Without Sight
Abstract
Presence describes immersion in a mediated environment such that it seems unmediated. For people with visual impairment, audio description replaces missing visual information with a verbal commentary, transforming an audiovisual medium into audio. Media forms are more or less immersive, with audio-only at the bottom of the scale. Anecdotally, however, pictures are said to be better on radio. Sound effects may contribute by triggering vivid mental images. Yet the role of sound effects on presence has been little explored. The aim of this study was to test the influence of sound effects and visual experience on presence. Participants (N = 73) with full, some, or no sight reported presence levels for a scene from an audio drama presented with or without sound effects. Participants with full vision reported higher levels of ecological and spatial presence for dialogue and sound effects than for dialogue alone. For participants with impaired vision, sound effects made no significant difference to presence levels. This was a small, exploratory study. Sound effects increased two dimensions of presence for those with sight. For blind people, words alone provided a rich imaginative experience. This has positive implications for audio description, which necessitates dipping the soundtrack to insert descriptive commentary. It suggests sound effects have a key role in stimulating presence, but this is dependent on the sensory characteristics of the listener.
References
(1995). Presence and performance within virtual environments. In , Virtual environments and advanced interface design (pp. 473–541). Oxford, UK: Oxford University Press.
(2011). Language processing in the occipital cortex of congenitally blind adults. Proceedings of the National Academy of Science, USA, 108(11), 4429–4434.
(2002). I can hear you but can I see you? The use of visual cognition during exposure to high imagery radio advertisements. Communication Research, 29, 537–563.
(1999). Radio drama theory and practice. London, UK: Routledge.
(2007). Media for all, approaches to translation studies. Amsterdam Rodopi.
(2006). Network 1000: Opinions and circumstances of visually impaired people in Great Britain. University of Birmingham Retrieved from www.rnib.org.uk/Search/Pages/results.aspx?k=network%201000&s=All%20Sites.
(1999). Effects of sensory information and prior experience on direct subjective ratings of presence. Presence: Teleoperators and Virtual Environments, 8, 1–13.
(2010). Audio description as audio drama: A practitioner's point of view. Perspectives: Studies in Translatology, 18, 205–213.
(2012a). Presence in those with and without sight: Implications for virtual reality and audio description. Journal of CyberTherapy & Rehabilitation, 5, 15–23.
(2012b). Cinematic language and the description of film: Keeping AD users in the frame. Perspectives: Studies in Translatology, 1–15. doi: 10.1080/0907676X.2012.693108
(2013). Sounds to spark the imagination: multimodal imagery prompted by words and sound effects in people with impaired vision. Manuscript submitted for publication.
(2011). Imagery and retrieval of auditory and visual information: Neural correlates of successful and unsuccessful performance. Neuropsychologia, 49, 1730–1740.
(2004). Development of multisensory spatial integration. In , Crossmodal space & crossmodal attention (pp. 1–24). Oxford, UK: Oxford University Press.
(2006). Cognitive load of navigating without vision when guided by virtual sound versus spatial language. Journal of Experimental Psychology: Applied, 17(4), 223–232.
(2001). A cross-media presence questionnaire: The ITC-Sense of Presence Inventory. . Presence: Teleoperators and Virtual Environments, 10(3), 282–297.
(1997). At the heart of it all: The concept of presence. Journal of Computer Communication, 3(2), 1–39.
(2009). The design of an audio film: Portraying story, action and interaction through sound. Journal of Music and Meaning, 8, Retrieved from www.musicandmeaning.net/issues/showArticle.php?artID=8.2.
(2008). Rapid and reversible: Recruitment of early visual cortex for touch. PLoS ONE, 3(8), e3046 doi:10.1371/journal.pone.0003046.
(1992). Mental imagery and sound effects in radio commercials. Journal of Advertising, 21(4), 83–93.
(1997). Hamlet on the Holodeck: The future of narrative in cyberspace. Cambridge, MA: MIT Press.
(2001). The metamodal organization of the brain. Progress in Brain Research, 134, 427–445.
(1996). Evaluating visual information provided by audio description. Journal of Visual Impairment and Blindness, 90(5), 378–385.
(2004). The language system of audio description: An investigation as a discursive process. Journal of Visual Impairment and Blindness, 98(8), 453–469.
. (2012). Radio listening habits. Retrieved from www.rab.co.uk/why-use-radio/listener-insight/radio-listening-habits.
(2000). Event-related potentials during auditory language processing in congenitally blind and sighted people. Neuropsychologia, 38(11), 1482–1502.
(2012). See it on a radio story: sound effects and shots to evoked imagery and attention on audio fiction. Communication Research, 39, 458–479.
(1992). Musings on telepresence and virtual presence. Presence: Teleoperators and Virtual Environments, 1, 120–125.
(2010). Image versus sound: A comparison of formal feature effects on presence and video game enjoyment. PsychNology Journal, 8(1), 67–84.
(1993). Representation systems, perceptual positions, and presence in immersive virtual environments. Presence, 2, 221–233.
(2010). Early but not late blindness leads to enhanced auditory perception. Neuropsychologia, 48, 344–348.
(2009). Crossmodal recruitment of primary visual cortex following brief exposure to bimodal audiovisual stimuli. Neuropsychologia, 48, 591–600.
