Skip to main content
SpringerLink
Log in

Deploying Prerecorded Audio Description for Musical Theater Using Live Performance Tracking

  • Conference paper
  • First Online:
Perception, Representations, Image, Sound, Music (CMMR 2019)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12631))

Included in the following conference series:

  • 993 Accesses

Abstract

Audio description, an accessibility service used by blind or visually impaired individuals, provides spoken descriptions of visual content. This alternative format allows those with low or no vision the ability to access information that sighted people obtain visually. In this paper a method for deploying prerecorded audio description in a live musical theater environment is presented. This method uses a reference audio recording and an online time warping algorithm to align tracks of audio description with live performances. A software implementation that is integrated into an existing theatrical workflow is also described. This system is used in two evaluation experiments that show the method successfully aligns multiple recordings of works of musical theater in order to automatically trigger prerecorded, descriptive audio in real time.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Arzt, A.: Flexible and robust music tracking. Ph.D. dissertation, Johannes Kepler University, Linz (2016)

    Google Scholar 

  2. Arzt, A., Widmer, G., Dixon, S.: Automatic page turning for musicians via real-time machine listening. In: Proceedings of the European Conference on Artificial Intelligence, pp. 241–245 (2008)

    Google Scholar 

  3. Branje, C.J., Fels, D.I.: LiveDescribe: can amateur describers create high-quality audio description? J. Vis. Impair. Blind. 106(3), 154–165 (2012)

    Article  Google Scholar 

  4. Campos, V.P., de Araujo, T.M.U., de Souza Filho, G.L., Goncalves, L.M.G.: CineAD: a system for automated audio description script generation for the visually impaired. Universal Access in the Information Society, pp. 1–13 (2018)

    Google Scholar 

  5. Dixon, S.: Live tracking of musical performances using on-line time warping. In: Proceedings of the 8th International Conference on Digital Audio Effects, pp. 92–97 (2005)

    Google Scholar 

  6. Dubagunta, S.P.: A simple MFCC extractor using C++ STL and C++11. Source code at (2016). http://www.github.com/dspavankumar/compute-mfcc

  7. Fryer, L.: An Introduction to Audio Description: A Practical Guide. Routledge, London (2016)

    Book  Google Scholar 

  8. Lertwongkhanakool, N., Kertkeidkachorn, N., Punyabukkana, P., Suchato, A.: An automatic real-time synchronization of live speech with its transcription approach. Eng. J. 19(5), 81–99 (2015)

    Article  Google Scholar 

  9. Litsyn, E., Pipko, H.: System and method for distribution and synchronized presentation of content. U.S. Patent Application 16/092,775, 2 May 2019

    Google Scholar 

  10. Logan, B.: Mel frequency cepstral coefficients for music modeling. ISMIR. 270, 1–11 (2000)

    Google Scholar 

  11. Muda, L., Begam, M., Elamvazuthi, I.: Voice recognition algorithms using mel frequency cepstral coefficient (MFCC) and dynamic time warping (DTW) techniques. J. Comput. 2(3) (2010)

    Google Scholar 

  12. Plaza, M.: Cost-effectiveness of audio description process: a comparative analysis of outsourcing and “in-house" methods. Int. J. Prod. Res. 55, 3480–3496 (2017)

    Article  Google Scholar 

  13. Sakoe, H., Chiba, S.: Dynamic programming algorithm optimisation for spoken word recognition. IEEE Trans. Acoust. Speech Signal Process. 26, 43–49 (1978)

    Article  Google Scholar 

  14. Snyder, J.: The visual made verbal: A comprehensive training manual and guide to the history and applications of audio description. American Council of the Blind (2014)

    Google Scholar 

  15. Szarkowska, A.: Text-to-speech audio description: towards a wider availability of AD. J. Spec. Transl. 15, 142–162 (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. New York University, New York, NY, USA

    Dirk Vander Wilt & Morwaread Mary Farbood

Authors
  1. Dirk Vander Wilt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Morwaread Mary Farbood
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dirk Vander Wilt .

Editor information

Editors and Affiliations

  1. Laboratoire PRISM, CNRS-AMU, Marseille, France

    Richard Kronland-Martinet

  2. Laboratoire PRISM, CNRS-AMU, Marseille, France

    Sølvi Ystad

  3. Laboratoire PRISM, CNRS-AMU, Marseille, France

    Mitsuko Aramaki

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Vander Wilt, D., Farbood, M.M. (2021). Deploying Prerecorded Audio Description for Musical Theater Using Live Performance Tracking. In: Kronland-Martinet, R., Ystad, S., Aramaki, M. (eds) Perception, Representations, Image, Sound, Music. CMMR 2019. Lecture Notes in Computer Science(), vol 12631. Springer, Cham. https://doi.org/10.1007/978-3-030-70210-6_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-70210-6_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-70209-0

  • Online ISBN: 978-3-030-70210-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation