research-article

Leveraging behavioral models of sounding objects forgesture-controlled sound design

Authors:

Kristian Gohlke,

Jörn LoviscachAuthors Info & Claims

TEI '11: Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction

Pages 245 - 248

https://doi.org/10.1145/1935701.1935750

Published: 22 January 2010 Publication History

Abstract

Sound designers and Foley artists have long struggled to create expressive soundscapes using standard editing software, devoting much time for the calibration of multiple sound samples and parameter adjustments. We present an intuitive approach that exploits the capabilities of off-theshelf motion-sensing input devices to enable quick and fluid interaction with sound to trigger and modulate digital sound generators based on adaptable behavioral models of familiar physical sounding objects. Rather than requiring profound technical knowledge of sound design, the system leverages the user's motor memory and motion skills to mimic generic and familiar interactions with everyday sounding objects. This allows the user to fully focus on the expressive act of sound creation while enjoying a fluent workflow and a satisfying user experience.

References

[1]

Ashbrook, D. and Starner, T. 2010. MAGIC: a motion gesture design tool. In Proc. Conference on Human Factors in Computing Systems. CHI (2010). pp. 2159-- 2168.

Digital Library

[2]

Black, D., Heise, S., Loviscach, J. Generic Sound Effects to Aid in Audio Retrieval. In Proc. 126th Convention of the Audio Engineering Society (2009), Paper 7795.

[3]

Black, D., Gohlke, K., Loviscach, J. FoleySonic: Placing Sounds on a Timeline Through Gestures. In Proc. 128th Convention of the Audio Engineering Society (2010), Paper 8004.

[4]

Böttcher, N., Gelineck, S., and Serafin, S. PHYSMISM: a control interface for creative exploration of physical models. In Proc. Conference on New interfaces For Musical Expression. NIME (2007). pp. 31--36.

Digital Library

[5]

Burtner, M. A theory of modulated objects for new shamanic controller design. In Proc. Conference on New interfaces For Musical Expression. NIME (2004). pp. 193--196.

Digital Library

[6]

Cook, P. Principles for designing computer music controllers. In Proc. Conference on New interfaces For Musical Expression, NIME (2001). pp. 1--4.

Digital Library

[7]

Cook, P. Modeling Bill's Gait: Analysis and Parametri. Synthesis of Walking Sounds. In Proc. 22nd I. Conf. on Virtual, Synthetic, and Entertainment Audio of the Audio Engineering Society (2002), Paper 222

[8]

Fiebrink, R., Wang, G., and Cook, P. R. Don't forget the laptop: using native input capabilities for expressive musical control. In Proc. Conference on New interfaces For Musical Expression. NIME (2007). pp. 164--167

Digital Library

[9]

Heise, S., and Loviscach, J., A Versatile Expressive Percussion Instrument with Game Technology, Proceedings of IEEE ICME (2008). pp. 393--396.

[10]

Lehrman, P. D. The Wii Remote as a Musical Instrument: Technology and Case Studies. In Proc. 127th Convention of the Audio Engineering Society (2009), Paper 7888.

[11]

Levitin D. J., McAdams, S., Adams, R.L. Control parameters for musical instruments: a foundation for new mappings of gesture to sound. Organised Sound 7(2). (2002) Cambridge University Press, UK. pp. 171--189.

Digital Library

[12]

Mäki-Patola, T., Laitinen, J., Kanerva, A., and Takala. T. 2005. Experiments with virtual reality instruments. In Proc. Conference on New interfaces For Musical Expression. NIME (2005). pp. 11--16.

Digital Library

[13]

Nintendo Wii Motion Plus http://www.nintendo.com/wii/what/accessories/wiimotionplus (last accessed: July 29, 2010).

[14]

O'Modhrain, S., Essl, G., PebbleBox and CrumbleBag: Tactile Interfaces for Granular Synthesis. In Proc. Conference on New interfaces For Musical Expression. NIME (2004). pp. 74--79.

Digital Library

[15]

Paine, D. G. 2004. Gesture and musical interaction: interactive engagement through dynamic morphology. In Proc. Conference on New interfaces For Musical Expression. NIME (2004). pp. 80--86.

Digital Library

[16]

Paine, G., Stevenson, I., and Pearce, A. 2007. The Thummer Mapping Project (ThuMP). In Proc. Conference on New interfaces For Musical Expression. NIME (2007). pp. 70--77.

Digital Library

[17]

Sony PlayStation Move Controller http://us.playstation.com/ps3/accessories/scph-98060.htm. (last accessed: July 29, 2010).

[18]

Yewdall, D. L. Practical Art of Motion Picture Sounds. (2007). Focal Press, Burlington, MA, USA. pp. 403--439.

Cited By

Johansen Svan Berkel NFritsch J(2022)Characterising Soundscape Research in Human-Computer InteractionProceedings of the 2022 ACM Designing Interactive Systems Conference10.1145/3532106.3533458(1394-1417)Online publication date: 13-Jun-2022
https://dl.acm.org/doi/10.1145/3532106.3533458

Index Terms

Leveraging behavioral models of sounding objects forgesture-controlled sound design

Recommendations

Aurally and visually enhanced audio search with soundtorch
CHI EA '09: CHI '09 Extended Abstracts on Human Factors in Computing Systems

Finding a specific or an artistically appropriate sound in a vast collection comprising thousands of audio files containing recordings of, say, footsteps, gunshots, and thunderclaps easily becomes a chore. To improve on this, we have developed an ...
Sound Design Impacts User Experience and Attention in Serious Game
Serious Games
Abstract
We study the impact of sound design – soundscape, sound effects, and auditory notifications, namely earcons – on the player’s experience of serious games. Three sound design versions for the game Venci’s Adventures have been developed: 1) no sound;...
Soundify: Matching Sound Effects to Video
UIST '23: Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology

In the art of video editing, sound helps add character to an object and immerse the viewer within a space. Through formative interviews with professional editors (N=10), we found that the task of adding sounds to video can be challenging. This paper ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

TEI '11: Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction

January 2011

470 pages

ISBN:9781450304788

DOI:10.1145/1935701

General Chairs:
Mark D. Gross
Carnegie Mellon University / Modular Robotics
,
Nuno Jardim Nunes
University of Madeira
,
Program Chairs:
Ellen Yi-Luen Do
Georgia Tech
,
Stephen Brewster
University of Glasgow
,
Ian Oakley
University of Madeira

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 January 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

TEI'11

Sponsor:

SIGCHI

TEI'11: Fifth International Conference on Tangible, Embedded, and Embodied Interaction

January 22 - 26, 2011

Funchal, Portugal

Acceptance Rates

Overall Acceptance Rate 393 of 1,367 submissions, 29%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
322
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Johansen Svan Berkel NFritsch J(2022)Characterising Soundscape Research in Human-Computer InteractionProceedings of the 2022 ACM Designing Interactive Systems Conference10.1145/3532106.3533458(1394-1417)Online publication date: 13-Jun-2022
https://dl.acm.org/doi/10.1145/3532106.3533458

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten