Skip to main content
Springer Nature Link
Log in

Multi-modal musical environments for mixed-reality performance

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

This article describes a series of multi-modal networked musical performance environments designed and implemented for concert presentation at the Torino-Milano (MiTo) Festival (Settembre musica, 2009, http://www.mitosettembremusica.it/en/home.html) between 2009 and 2010. Musical works, controlled by motion and gestures generated by in-engine performer avatars will be discussed with specific consideration given to the multi-modal presentation of mixed-reality works, combining both software-based and real-world traditional musical instruments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Barbosa A (2003) Displaced soundscapes: a survey of network systems for music and sonic art creation. Leonardo Music J 13:53–59

    Article  MathSciNet  Google Scholar 

  2. Bencina R (2006) Oscpacka simple c++ osc packet manipulation library. URL http://code.google.com/p/oscpack/

  3. Bilinghurst M, Kato H (1999) Merging physical world and virtual world. In: Proceedings of the international symposium on mixed reality, Yokohama, Japan. URL ftp://ftp.hitl.washington.edu/pub/publications/r-91-2/r-98-36/r-98-36.html

    Google Scholar 

  4. Cáceres JP, Chafe C (2009) JackTrip: under the hood of an engine for network audio. In: Proceedings of international computer music conference, International Computer Music Association, San Francisco, CA, pp 509–512

    Google Scholar 

  5. Cáceres JP, Chafe C (2010) JackTrip: under the hood of an engine for network audio. J New Music Res 39(3):183–187. doi:10.1080/09298215.2010.481361

    Article  Google Scholar 

  6. Cáceres JP, Hamilton R, Iyer D, Chafe C, Wang G (2008) To the edge with china: explorations in network performance. In: ARTECH 2008: proceedings of the 4th international conference on digital arts, Porto, Portugal, pp 61–66

    Google Scholar 

  7. Chadabe J (1996) Electric sound: the past and promise of electronic music. Prentice Hall, New York

    Google Scholar 

  8. Freeman JEA (2005) The architecture of auracle: a voice-controlled, networked sound instrument. In: Proceedings of international computer music conference, Barcelona

    Google Scholar 

  9. Furukawa K, Fujihata M, Muench W (2000) Small_fish. URL http://hosting.zkm.de/wmuench/small_fish

  10. Gresham-Lancaster S (1998) The aesthetics and history of the Hub: the effect of changing technology on network computer music. Leonardo Music J 8:39–44

    Article  Google Scholar 

  11. Hamilton R (2008) q3osc: or how i learned to stop worrying and love the game. In: Proceedings of international computer music conference, Belfast, Northern Ireland

    Google Scholar 

  12. Hamilton R (2008) Maps and legends: designing fps-based interfaces for multi-user composition, improvisation and interactive performance. In: Computer music modeling and retrieval. Lecture notes in computer science, vol 13, pp 478–486

    Chapter  Google Scholar 

  13. id software: Quake 3 (2000) URL http://www.idsoftware.com/games/quake/quake3-arena/

  14. Malham D (1995) 3-D sound spatialization using ambisonic techniques. Comput Music J 4(19):58–70

    Article  Google Scholar 

  15. Marinetti FT (1909) The futurist manifesto

  16. Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE Trans Inf Syst E77-D(12):58–70. URL http://etclab.mie.utoronto.ca/people/paul_dir/IEICE94/ieice.html

    Google Scholar 

  17. Mito: Settembre musica (2009) URL http://www.mitosettembremusica.it/en/home.html

  18. Oliver J (2003) q3apd: making music with bots. URL http://vimeo.com/1380686

  19. Oliver J, Pickles S (2003) q3apd. URL http://www.selectparks.net/archive/q3apd.htm

  20. Oliveros P, Chafe C, Dresser M (2009) Telematic music: six perspectives. Leonardo Music J 13:95–96

    Google Scholar 

  21. Pickles S (2004) Fijuu 2. URL http://www.fijuu.com

  22. Puckette M (1996) Pure data. In: Proceedings of international computer music conference, San Francisco, pp 269–272

    Google Scholar 

  23. Sirikata (2009) A bsd licensed open source platform for games and virtual worlds. URL http://www.sirikata.com

  24. SoundWIRE Group (2010) SoundWIRE research group at CCRMA, Stanford University. URL http://ccrma.stanford.edu/groups/soundwire/

  25. Supercollider (2006) URL http://supercollider.sourceforge.net

  26. Trueman D, Cook P, Smallwood S, Wang G (2006) Plork: Princeton laptop orchestra, year 1. In: Proceedings of international computer music conference

    Google Scholar 

  27. Unreal development kit (2010) URL http://www.udk.com

  28. Wang G, Cook PR (2003) ChucK: a concurrent, on-the-fly, audio programming language. In: Proceedings of international computer music conference, Singapore

    Google Scholar 

  29. Wang G, Bryan N, Oh J, Hamilton R (2009) Stanford laptop orchestra (slork). In: Proceedings of international computer music conference, Montreal

    Google Scholar 

  30. Wright M (2005) Open sound control: an enabling technology for musical networking. Organ Sound 10:193–200

    Article  Google Scholar 

  31. Xu A, Cooperstock JR (2000) Real-time streaming of multichannel audio data over Internet. In: Proceedings of the 108th convention of the audio engineering society, Paris, pp 627–641

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Computer Research in Music and Acoustics (CCRMA), Stanford University, Stanford, CA, 94305, USA

    Robert Hamilton, Juan-Pablo Caceres, Chryssie Nanou & Chris Platz

Authors
  1. Robert Hamilton
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Juan-Pablo Caceres
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Chryssie Nanou
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Chris Platz
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Robert Hamilton.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hamilton, R., Caceres, JP., Nanou, C. et al. Multi-modal musical environments for mixed-reality performance. J Multimodal User Interfaces 4, 147–156 (2011). https://doi.org/10.1007/s12193-011-0069-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-011-0069-1

Keywords