Skip to main content
SpringerLink
Log in

When You Hear the Chime: Movable Books and the Dramaturgical Functions of Sound in Mixed Reality Interactive Narrative Design

  • Conference paper
  • First Online:
Interactive Storytelling (ICIDS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13762))

Included in the following conference series:

Abstract

In this paper we outline the pre-digital histories of recorded and synthesized sound, exploring their entanglements with both the literal codex and larger literary imaginary. In particular, we focus on intersections of sound and movable books, offering the rich genealogy of the movable book as a fertile addition to the IDN (interactive digital narrative) family tree, as an example of pre-computational interactive narrative with a long history. Drawing on this intermedial history, along with our own experience designing an MR (mixed reality) movable book, we offer a taxonomy of dramaturgical functions of sound in MR IDN. We demonstrate the use of this taxonomy in analysis of our own work, and suggest opportunities for expanding the taxonomy in support of future speculative research and design imaginaries for IDNs.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Koenitz, H., Di Pastena, A., Jansen, D., de Lint, B., Moss, A.: The myth of ‘universal’ narrative models. In: Rouse, Rebecca, Koenitz, Hartmut, Haahr, Mads (eds.) ICIDS 2018. LNCS, vol. 11318, pp. 107–120. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04028-4_8

    Chapter  Google Scholar 

  2. Gampe, J.: Interactive narration within audio augmented realities. In: Iurgel, Ido A., Zagalo, Nelson, Petta, Paolo (eds.) ICIDS 2009. LNCS, vol. 5915, pp. 298–303. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-10643-9_34

    Chapter  Google Scholar 

  3. Rettberg, S.: Electronic Literature. Polity Press, Cambridge (2019)

    Google Scholar 

  4. Ryan, J.: Grimes’ fairy tales: a 1960s story generator. In: Nunes, N. et al. (ed.) Interactive Storytelling. LNCS, vol. 10690, pp. 89–103 (2017). https://doi.org/10.1007/978-3-319-71027-3_8

  5. Reid-Walsh, J.: Interactive Books: Playful Media Before Pop-Ups. Routledge, London and New York (2018)

    Google Scholar 

  6. Field, H.: Playing with the Book: Victorian Movable Picture Books and the Child Reader. University of Minnesota Press, Minneapolis (2019)

    Book  Google Scholar 

  7. Rouse, R., Holloway-Attaway, L.: Playing at the page: designing to support creative readership practices. J. Interact. Books 1(1) (2021). https://doi.org/10.57579/2022JIB013RR

  8. Rouse, R., Engberg, M., Parvin, N., Bolter, J.D.: MRx: an interdisciplinary framework for mixed reality experience design and criticism. Digital Creativity 26(3/4), 175–227, 179 (2015)

    Google Scholar 

  9. Sigarchian, H.G., De Meester, B., Salliau, F., De Neve, W., et al.: Hybrid books for interactive digital storytelling: connecting story entities and emotions to smart environments. In: Schoenau-Fog, H. et al. (eds.) Interactive Storytelling. LNCS, vol. 9445, pp. 105–116. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-319-27036-4_10

  10. Rubery, M. (ed.): Audiobooks, Literature, and Sound Studies. Routledge (2011)

    Google Scholar 

  11. Mattern, S.: Sonic archaeologies. In: Bull, M. (ed.) The Routledge Companion to Sound Studies, pp. 222–230. Routledge, London and New York (2019)

    Google Scholar 

  12. Alderson-Day, B., Bernini, M., Feryhough, C.: Uncharted features and dynamics of reading: voices, characters, and crossing of experiences. Consciousness Cognition 49, 98–109 (2017)

    Google Scholar 

  13. Von Kempelen, W.: Mechanismus der menschlichen Sprache nebst der Beschreibung seiner sprechenden Maschine. J. C. Degen, Vienna (1791)

    Google Scholar 

  14. Brackhane, F., Sproat, R., Trouvain, J. (eds.): Wolfgang von Kempelen: Mechanismus der menschlichen Sprache, p. 513. TUD Press, Dresden (2017)

    Google Scholar 

  15. Brackhane, F., Sproat, R., Trouvain, J. (eds.): Wolfgang von Kempelen: Mechanismus der menschlichen Sprache, p. 388. TUD Press, Dresden (2017)

    Google Scholar 

  16. Nadarajan, G.: Islamic automation: a reading of Al-Jazari’s the book of knowledge of ingenious mechanical devices (1206). In: Grau, O., (ed.) Media Art Histories, pp. 163–178. Cambridge University Press, Cambridge (2007)

    Google Scholar 

  17. Truitt, E.R.: Medieval Robots: Mechanism, Magic, Nature, and Art. University of Pennsylvania Press, Philadelphia (2015)

    Google Scholar 

  18. Peppè, R.: Automata and Mechanical Toys. The Crowood Press, Ramsbury (2002)

    Google Scholar 

  19. Blott, M., Manschot, B. (eds.): Museum Guide to the National Museum from Musical Clock to Street Organ. National Museum van Speelklok tot Pierement, Utrecht (2005)

    Google Scholar 

  20. Connor, S.: Dumbstruck: A Cultural History of Ventriloquism. Oxford University Press, Oxford (2000)

    Book  Google Scholar 

  21. Dudley, H.W.: System for the Artificial Production of Vocal or Other Sounds. US Patent 2,121,142, 21 June 1938

    Google Scholar 

  22. Levin, T.Y.: Tones from out of Nowhere: Rudolf Pfenninger and the Archaeology of Synthetic Sound. In: New Media Old Media: A History and Theory Reader, pp. 45–81. Routledge, New York (2006)

    Google Scholar 

  23. Luck, M., Aylett, R.: Applying artificial intelligence to virtual reality: intelligent virtual environments. Appl. Artif. Intell. 14(1), 3–32 (2000)

    Article  Google Scholar 

  24. Szarkowska, A.: Text-to-speech audio description: towards wider availability of AD. J. Specialized Transl. 15, 142–162 (2011)

    Google Scholar 

  25. Johnson, E.H.: A wonderful invention-speech capable of indefinite repetition from automatic records. Scientific American, p. 304, 17 November 1877

    Google Scholar 

  26. Sterne, J.: The Audible Past: Cultural Origins of Sound Reproduction, p. 50. Duke University Press, Durham and London (2003)

    Book  Google Scholar 

  27. Bell, A.G., Blake, C.J.: Letter from Clarence J. Blake to Alexander Graham Bell, August 5. 5 August 1874 Manuscript, Library of Congress (1874). https://www.loc.gov/item/magbell.29700101/. Accessed 19 July 2021

  28. Blake, C.J.: The use of the membrana tympani as a phonautograph. Boston Med. Surg. J. 92(5), 121–124 (1875)

    Article  Google Scholar 

  29. Sterne: pp. 37–41 (2003)

    Google Scholar 

  30. Wood, G.: Edison’s Eve: A Magical History of the Quest for Mechanical Life. AA Knopf, New York (2002)

    Google Scholar 

  31. Good, K.D.: Listening to pictures: converging media histories and the multimedia newspaper. Journalism Stud 18(6), 691–709 (2017)

    Google Scholar 

  32. Mayhew, R.: Book. U.S. Patent 1,236,333, 7 August 1917

    Google Scholar 

  33. Mayhew, R.: Book. U.S. Patent 1,236,333, 7 August 1917, Line 17–23

    Google Scholar 

  34. Johnson, B.: Cut Out Toy. US Patent 1,544,645, 7 July 1925

    Google Scholar 

  35. Yip, W.: Children’s Sound Book. US Patent 6,641,170, 4 November 2003

    Google Scholar 

  36. Kirwan, D.G.: Interactive Picture Book with Voice Recording Features and Method of Use. US Patent 6,985,693, 10 January 2006

    Google Scholar 

  37. Rouse, R., Engberg, M., Parvin, N., Bolter, J.D.: MRX: an interdisciplinary framework for mixed reality experience design and criticism. Digital Creativity 26(3–4), 175–181 (2015)

    Article  Google Scholar 

  38. Billinghurst, M., Kato, H., Poupyrev, I.: The MagicBook: moving seamlessly between reality and virtuality. IEEE Graph. Appl. 21(3), 6–8 (2001)

    Google Scholar 

  39. Rodenburg, P.: The Actor Speaks: Voice and the Performer. Methuen Drama, London (1997)

    Google Scholar 

  40. Linklater, K.: Freeing the Natural Voice: Imagery and Art in the Practice of Voice and Language (2006)

    Google Scholar 

  41. Berndt, A., Hartmann, K.: The functions of music in interactive media. In: Spierling, Ulrike, Szilas, Nicolas (eds.) ICIDS 2008. LNCS, vol. 5334, pp. 126–131. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-89454-4_19

    Chapter  Google Scholar 

  42. Turner, C., Behrndt, S.: Dramaturgy and Performance, Revised Palgrave MacMillan, London and New York (2016)

    Book  Google Scholar 

  43. Barad, K.: Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning. Duke University Press, Durham and London (2007)

    Book  Google Scholar 

  44. Parikka, J.: 12. Mapping noise: techniques and tactics of irregularities, interception, and disturbance. In: Media Archaeology, pp. 256–277. University of California Press (2011)

    Google Scholar 

  45. Rouse, R.: Woyzeck: augmented reality performance installation Master’s project. Master’s thesis, Communication and Culture. York University, Canada (2007)

    Google Scholar 

  46. Speiginer, G., MacIntyre, B., Bolter, J., Rouzati, H., Lambeth, A., Levy, L., Baird, L., Gandy, M., Sanders, M., Davidson, B., Engberg, M., Clark, R., Mynatt, E.: The evolution of the Argon web framework through its use creating cultural heritage and community–based augmented reality applications. In: Kurosu, Masaaki (ed.) HCI 2015. LNCS, vol. 9171, pp. 112–124. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21006-3_12

    Chapter  Google Scholar 

  47. Holloway-Attaway, L.: Digital bridges and liquid borders: everyday storytelling and/as Baltic identity. In: Ekstrand, T. (ed.) Art Line: A Baltic Collaboration. Blekinge Museum (2012)

    Google Scholar 

  48. Holloway-Attaway, L.: Performing materialities: exploring mixed media reality and Moby-Dick. Convergence Int. J. Res. New Media Technol. 20(1), 55–68 (2014)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Skövde, 54128, Skövde, Sweden

    Lissa Holloway-Attaway & Rebecca Rouse

Authors
  1. Lissa Holloway-Attaway
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rebecca Rouse
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Lissa Holloway-Attaway or Rebecca Rouse .

Editor information

Editors and Affiliations

  1. Amsterdam University of Applied Sciences, Amsterdam, The Netherlands

    Mirjam Vosmeer

  2. University of Skövde, Skövde, Sweden

    Lissa Holloway-Attaway

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Holloway-Attaway, L., Rouse, R. (2022). When You Hear the Chime: Movable Books and the Dramaturgical Functions of Sound in Mixed Reality Interactive Narrative Design. In: Vosmeer, M., Holloway-Attaway, L. (eds) Interactive Storytelling. ICIDS 2022. Lecture Notes in Computer Science, vol 13762. Springer, Cham. https://doi.org/10.1007/978-3-031-22298-6_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-22298-6_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-22297-9

  • Online ISBN: 978-3-031-22298-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation