research-article

Virtual reality musical instruments: Guidelines for multisensory interaction design

Authors:

Niels C. NilssonAuthors Info & Claims

AM '16: Proceedings of the Audio Mostly 2016

Pages 266 - 271

https://doi.org/10.1145/2986416.2986431

Published: 04 October 2016 Publication History

Get Access

Abstract

The rapid development and availability of low cost technologies has created a wide interest in virtual reality (VR), but how to design and evaluate multisensory interactions in VR remains as a challenge. In this paper, we focus on virtual reality musical instruments, present an overview of our design and evaluation guidelines, and examine historical case studies. Our main contribution is to inform the design and evaluation of the future VRMIs and consider the challenges.

References

[1]

Durand Begault. 1994. 3-D sound for virtual reality and multimedia. AP Professional, New York, NY, USA.

Digital Library

Google Scholar

[2]

Chuck Blanchard, Scott Burgess, Young Harvill, Jaron Lanier, Ann Lasko, Mark Oberman, Mike Teitel, Chuck Blanchard, Scott Burgess, Young Harvill, Ann Lasko, Mark Oberman, and Mike Teitel. 1990. Reality built for two: a virtual reality tool. SIGGRAPH Comput. Graph. 24, 2 (Feb. 1990), 35--36.

Digital Library

Google Scholar

[3]

Cumhur Erkut, Antti Jylhä, and Reha Discioglu. 2011. A Structured Design and Evaluation Model with Application to Rhythmic Interaction Displays. In Proceedings of the 2011 conference on New interfaces for musical expression. Oslo, Norway, 477--480.

Google Scholar

[4]

Steven Gelineck, Niels Böttcher, Linda Martinussen, and Stefania Serafin. 2005. Virtual Reality Instruments capable of changing Dimensions in Real-time. Proceedings Enactive (2005).

Google Scholar

[5]

Antti Jylhä, I Ekman, Cumhur Erkut, and Koray Tahiroğlu. 2011. Design and Evaluation of Rhythmic Interaction with an Interactive Tutoring System. Computer Music Journal 35, 2 (July 2011), 36--48.

Digital Library

Google Scholar

[6]

Matti Karjalainen and Teemu Mäki-Patola. 2004. Physics-based modeling of musical instruments for interactive virtual reality. In Multimedia Signal Processing, 2004 IEEE 6th Workshop on. IEEE, 223--226.

Crossref

Google Scholar

[7]

Konstantina Kilteni, Ilias Bergstrom, and Mel Slater. 2013. Drumming in immersive virtual reality: the body shapes the way we play. Visualization and Computer Graphics, IEEE Transactions on 19, 4 (2013), 597--605.

Digital Library

Google Scholar

[8]

Jaron Lanier. 1998. The Sound of One Hand. Whole Earth Review (1998), 1--4. https://www.youtube.com/watch?v=ItaPqJaUypY

Google Scholar

[9]

Teemu Mäki-Patola and Perttu Hämäläinen. 2004. Effect of latency on playing accuracy of two gesture controlled continuous sound instruments without tactile feedback. In Proc. Intl. Conf. Digital Audio Effects (DAFx). Naples, Italy, 11--16.

Google Scholar

[10]

Teemu Mäki-Patola, Juha Laitinen, Aki Kanerva, and Tapio Takala. 2005. Experiments with virtual reality instruments. In Proceedings of the 2005 conference on New interfaces for musical expression. National University of Singapore, 11--16.

Digital Library

Google Scholar

[11]

Michael Meehan. 2001. Physiological reaction as an objective measure of presence in virtual environments. Ph.D. Dissertation. Ph.D. thesis. University of North Carolina at Chapel Hill.

Digital Library

Google Scholar

[12]

Stefania Serafin, Cumhur Erkut, Juraj Kojs, Niels C. Nilsson, and Rolf Nordahl. 2017. Virtual reality musical instruments: state of the art, design principles and future directions. Computer Music Journal (Winter 2017), 1--37. Accepted.

Google Scholar

[13]

Mel Slater. 2009. Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society B: Biological Sciences 364, 1535 (2009), 3549--3557.

Crossref

Google Scholar

[14]

Mel Slater, D-P Pertaub, and Anthony Steed. 1999. Public speaking in virtual reality: Facing an audience of avatars. IEEE Computer Graphics and Applications 19, 2 (1999), 6--9.

Digital Library

Google Scholar

Cited By

View all

Bacca-Acosta JTejada JFabregat RKinshuk Guevara J(2021)Scaffolding in immersive virtual reality environments for learning English: an eye tracking studyEducational technology research and development10.1007/s11423-021-10068-770:1(339-362)Online publication date: 22-Nov-2021
https://doi.org/10.1007/s11423-021-10068-7
Enriquez KPalacios MPallo DGuerrero G(2020)SENSE: Sensory component VR application for hearing impaired people to enhance the music experience2020 15th Iberian Conference on Information Systems and Technologies (CISTI)10.23919/CISTI49556.2020.9140447(1-6)Online publication date: Jun-2020
https://doi.org/10.23919/CISTI49556.2020.9140447
Visi FFaasch F(2018)Motion Controllers, Sound, and Music in Video Games: State of the Art and Research PerspectivesEmotion in Video Game Soundtracking10.1007/978-3-319-72272-6_8(85-103)Online publication date: 10-Feb-2018
https://doi.org/10.1007/978-3-319-72272-6_8
Show More Cited By

Index Terms

Virtual reality musical instruments: Guidelines for multisensory interaction design
1. Applied computing
  1. Arts and humanities
    1. Sound and music computing
2. Information systems
  1. Information retrieval
    1. Specialized information retrieval
      1. Multimedia and multimodal retrieval
        Music retrieval
  2. Information systems applications
    1. Multimedia information systems

Recommendations

Virtual reality musical instruments: State of the art, design principles, and future directions

The rapid development and availability of low-cost technologies have created a wide interest in virtual reality. In the field of computer music, the term "virtual musical instruments" has been used for a long time to describe software simulations, ...
Experiments with virtual reality instruments
NIME '05: Proceedings of the 2005 conference on New interfaces for musical expression

In this paper, we introduce and analyze four gesture-controlled musical instruments. We briefly discuss the test platform designed to allow for rapid experimentation of new interfaces and control mappings. We describe our design experiences and discuss ...
Extending Virtual Reality Display Wall Environments Using Augmented Reality
SUI '19: Symposium on Spatial User Interaction

Two major form factors for virtual reality are head-mounted displays and large display environments such as CAVE®and the LCD-based successor CAVE2®. Each of these has distinct advantages and limitations based on how they’re used. This work explores ...

Comments

Information & Contributors

Information

Published In

AM '16: Proceedings of the Audio Mostly 2016

October 2016

285 pages

ISBN:9781450348225

DOI:10.1145/2986416

Conference Chair:
Johan Fagerlönn
Interactive Institute Swedish ICT

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

AM '16

AM '16: Audio Mostly 2016

October 4 - 6, 2016

Norrköping, Sweden

Acceptance Rates

AM '16 Paper Acceptance Rate 40 of 53 submissions, 75%;

Overall Acceptance Rate 177 of 275 submissions, 64%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
489
Total Downloads

Downloads (Last 12 months)33
Downloads (Last 6 weeks)3

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Bacca-Acosta JTejada JFabregat RKinshuk Guevara J(2021)Scaffolding in immersive virtual reality environments for learning English: an eye tracking studyEducational technology research and development10.1007/s11423-021-10068-770:1(339-362)Online publication date: 22-Nov-2021
https://doi.org/10.1007/s11423-021-10068-7
Enriquez KPalacios MPallo DGuerrero G(2020)SENSE: Sensory component VR application for hearing impaired people to enhance the music experience2020 15th Iberian Conference on Information Systems and Technologies (CISTI)10.23919/CISTI49556.2020.9140447(1-6)Online publication date: Jun-2020
https://doi.org/10.23919/CISTI49556.2020.9140447
Visi FFaasch F(2018)Motion Controllers, Sound, and Music in Video Games: State of the Art and Research PerspectivesEmotion in Video Game Soundtracking10.1007/978-3-319-72272-6_8(85-103)Online publication date: 10-Feb-2018
https://doi.org/10.1007/978-3-319-72272-6_8
Zhang X(2017)Implementation Study of Virtual Reality Technique in Environmental Artistic Design2017 International Conference on Robots & Intelligent System (ICRIS)10.1109/ICRIS.2017.46(157-160)Online publication date: Oct-2017
https://doi.org/10.1109/ICRIS.2017.46

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Virtual reality musical instruments: State of the art, design principles, and future directions

Experiments with virtual reality instruments

Extending Virtual Reality Display Wall Environments Using Augmented Reality

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations