skip to main content
10.1145/2159616.2159618acmconferencesArticle/Chapter ViewAbstractPublication Pagesi3dConference Proceedingsconference-collections
research-article

Tabletop Ensemble: touch-enabled virtual percussion instruments

Published: 09 March 2012 Publication History

Abstract

We present an interactive virtual percussion instrument system, Tabletop Ensemble, that can be used by a group of collaborative users simultaneously to emulate playing music in real world while providing them with flexibility of virtual simulations. An optical multi-touch tabletop serves as the input device. A novel touch handling algorithm for such devices is presented to translate users' interactions into percussive control signals appropriate for music playing. These signals activate the proposed sound simulation system for generating realistic user-controlled musical sounds. A fast physically-based sound synthesis technique, modal synthesis, is adopted to enable users to directly produce rich, varying musical tones, as they would with the real percussion instruments. In addition, we propose a simple coupling scheme for modulating the synthesized sounds by an accurate numerical acoustic simulator to create believable acoustic effects due to cavity in music instruments. This paradigm allows creating new virtual percussion instruments of various materials, shapes, and sizes with little overhead. We believe such an interactive, multi-modal system would offer capabilities for expressive music playing, rapid prototyping of virtual instruments, and active exploration of sound effects determined by various physical parameters in a classroom, museum, or other educational settings. Virtual xylophones and drums with various physics properties are shown in the presented system.

Supplementary Material

JPG File (p7-ren.jpg)
MP4 File (p7-ren.mp4)

References

[1]
Adrien, J.-M. 1991. Representations of musical signals. MIT Press, Cambridge, MA, USA, ch. The missing link: modal synthesis, 269--298.
[2]
Bilbao, S. 2009. Numerical Sound Synthesis. Wiley Online Library.
[3]
Bonneel, N., Drettakis, G., Tsingos, N., Viaud-Delmon, I., and James, D. 2008. Fast modal sounds with scalable frequency-domain synthesis. In ACM SIGGRAPH 2008 papers, ACM, New York, NY, USA, SIGGRAPH '08, 24:1--24:9.
[4]
Boyd, J. P. 2001. Chebyshev and Fourier Spectral Methods: Second Revised Edition, 2 revised ed. Dover Publications, December.
[5]
Brebbia, C. A. 1991. Boundary Element Methods in Acoustics, 1 ed. Springer, October.
[6]
Bruyns, C. 2006. Modal synthesis for arbitrarily shaped objects. Computer Music Journal 30, 3, 22--37.
[7]
Buxton, W., Hill, R., and Rowley, P. 1985. Issues and techniques in touch-sensitive tablet input. In ACM SIGGRAPH Computer Graphics, vol. 19, ACM, 215--224.
[8]
Chandak, A., Lauterbach, C., Taylor, M., Ren, Z., and Manocha, D. 2008. Ad-frustum: Adaptive frustum tracing for interactive sound propagation. IEEE Transactions on Visualization and Computer Graphics 14, 1707--1722.
[9]
Chuchacz, K., O'Modhrain, S., and Woods, R. 2007. Physical models and musical controllers: designing a novel electronic percussion instrument. In NIME '07: Proceedings of the 7th international conference on New interfaces for musical expression, ACM, New York, NY, USA, 37--40.
[10]
Cook, P. 2002. Real sound synthesis for interactive applications. AK Peters, Ltd.
[11]
Davidson, P., and Han, J. 2006. Synthesis and control on large scale multi-touch sensing displays. In Proceedings of the 2006 conference on New interfaces for musical expression, IRCAM-Centre Pompidou, 216--219.
[12]
Deines, E., Michel, F., Bertram, M., Hagen, H., and Nielson, G. 2006. Visualizing the phonon map. In Eurovis.
[13]
Framework Named pipes, M. I. 2011. Named pipes.
[14]
Funkhouser, T., Tsingos, N., and Jot, J.-M. 2003. Survey of methods for modeling sound propagation in interactive virtual environment systems. Presence and Teleoperation.
[15]
Funkhouser, T., Tsingos, N., Carlbom, I., Elko, G., Sondhi, M., West, J. E., Pingali, G., Min, P., and Ngan, A. 2004. A beam tracing method for interactive architectural acoustics. The Journal of the Acoustical Society of America 115, 2, 739--756.
[16]
Guski, R., and Troje, N. 2003. Audiovisual phenomenal causality. Perception & psychophysics 65, 5, 789.
[17]
Han, J. 2005. Low-cost multi-touch sensing through frustrated total internal reflection. In Proceedings of the 18th annual ACM symposium on User interface software and technology, ACM, 115--118.
[18]
Hochenbaum, J., and Vallis, O. 2009. Bricktable: A musical tangible multi-touch interface. Proceedings of Berlin Open Converence 09.
[19]
Kaltenbrunner, M., Jorda, S., Geiger, G., and Alonso, M. 2006. The reactable*: A collaborative musical instrument.
[20]
Miranda, E., and Wanderley, M. 2006. New digital musical instruments: control and interaction beyond the keyboard. AR Editions, Inc.
[21]
O'Brien, J., Shen, C., and Gatchalian, C. 2002. Synthesizing sounds from rigid-body simulations. In Proc. of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, ACM Press New York, NY, USA, 175--181.
[22]
Raghuvanshi, N., and Lin, M. 2006. Interactive sound synthesis for large scale environments. In Proceedings of the 2006 symposium on Interactive 3D graphics and games, ACM, 101--108.
[23]
Raghuvanshi, N., Narain, R., and Lin, M. C. 2009. Efficient and accurate sound propagation using adaptive rectangular decomposition. IEEE Transactions on Visualization and Computer Graphics 15 (September), 789--801.
[24]
Raghuvanshi, N., Snyder, J., Mehra, R., Lin, M., and Govindaraju, N. 2010. Precomputed wave simulation for real-time sound propagation of dynamic sources in complex scenes. ACM Trans. Graph. 29 (July), 68:1--68:11.
[25]
Ren, Z., Yeh, H., and Lin, M. C. 2011. Example-Guided Physically-Based Sound Synthesis. UNC Techinical Report.
[26]
Rosenberg, I., and Perlin, K. 2009. The unmousepad: an interpolating multi-touch force-sensing input pad. In ACM SIG-GRAPH 2009 papers, ACM, 1--9.
[27]
Sakamoto, S., Ushiyama, A., and Nagatomo, H. 2006. Numerical analysis of sound propagation in rooms using the finite difference time domain method. The Journal of the Acoustical Society of America 120, 5, 3008.
[28]
Schöening, J., Hook, J., Motamedi, N., Olivier, P., Echtler, F., Brandl, P., Muller, L., Daiber, F., Hilliges, O., Loechtefeld, M., Roth, T., Schmidt, D., and von Zadow, U. 2009. Building interactive multi-touch surfaces. Journal of Graphics, GPU, and Game Tools 14, 3, 35--55.
[29]
Shabana, A. 1997. Vibration of discrete and continuous systems. Springer Verlag.
[30]
Si, H. 2011. TetGen: A Quality Tetrahedral Mesh Generator and a 3D Delaunay Triangulator.
[31]
Thompson, L. L. 2006. A review of finite-element methods for time-harmonic acoustics. The Journal of the Acoustical Society of America 119, 3, 1315--1330.
[32]
Van Den Doel, K., and Pai, D. 1998. The sounds of physical shapes. PRESENCE-CAMBRIDGE MASSACHUSETTS-7, 382--395.
[33]
Weinberg, G., and Driscoll, S. 2007. The interactive robotic percussionist: new developments in form, mechanics, perception and interaction design. In Proceedings of the ACM/IEEE international conference on Human-robot interaction, ACM, New York, NY, USA, HRI '07, 97--104.
[34]
XAudio2, M. 2011. XAudio2.

Cited By

View all
  • (2023)Rhythmic Accuracy of 3D Spatial Interaction for Digital MusicProceedings of the 18th International Audio Mostly Conference10.1145/3616195.3616216(169-176)Online publication date: 30-Aug-2023
  • (2017)Six Degree-of-Freedom Haptic Simulation of a Stringed Musical Instrument for Triggering SoundsIEEE Transactions on Haptics10.1109/TOH.2016.262836910:2(265-275)Online publication date: 1-Apr-2017
  • (2017)Investigating pre-touch for sound generation on multi-touch surfaces using blob area detection2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC.2017.8122752(1064-1068)Online publication date: Oct-2017
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
I3D '12: Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games
March 2012
220 pages
ISBN:9781450311946
DOI:10.1145/2159616
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 March 2012

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. multi-modal interaction
  2. multi-touch tabletop
  3. sound simulation

Qualifiers

  • Research-article

Conference

I3D '12
Sponsor:
I3D '12: Symposium on Interactive 3D Graphics and Games
March 9 - 11, 2012
California, Costa Mesa

Acceptance Rates

Overall Acceptance Rate 148 of 485 submissions, 31%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)9
  • Downloads (Last 6 weeks)2
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Rhythmic Accuracy of 3D Spatial Interaction for Digital MusicProceedings of the 18th International Audio Mostly Conference10.1145/3616195.3616216(169-176)Online publication date: 30-Aug-2023
  • (2017)Six Degree-of-Freedom Haptic Simulation of a Stringed Musical Instrument for Triggering SoundsIEEE Transactions on Haptics10.1109/TOH.2016.262836910:2(265-275)Online publication date: 1-Apr-2017
  • (2017)Investigating pre-touch for sound generation on multi-touch surfaces using blob area detection2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC.2017.8122752(1064-1068)Online publication date: Oct-2017
  • (2017)Glass half fullThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-017-1383-833:6-8(1039-1048)Online publication date: 1-Jun-2017
  • (2016)Augmented reality instruction for object assembly based on markerless trackingProceedings of the 20th ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games10.1145/2856400.2856416(95-102)Online publication date: 27-Feb-2016
  • (2016)SynCoPation: Interactive Synthesis-Coupled Sound PropagationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2016.251842122:4(1346-1355)Online publication date: 21-Apr-2016
  • (2015)Caress: An Enactive Electro-acoustic Percussive Instrument for Caressing SoundProceedings of the international conference on New Interfaces for Musical Expression10.5555/2993778.2993841(245-250)Online publication date: 30-May-2015
  • (2015)Interactive virtual percussion instruments on mobile devicesProceedings of the 21st ACM Symposium on Virtual Reality Software and Technology10.1145/2821592.2821616(79-83)Online publication date: 13-Nov-2015
  • (2015)Aerophones in flatlandACM Transactions on Graphics10.1145/276700134:4(1-11)Online publication date: 27-Jul-2015
  • (2013)Dynamics based 3D skeletal hand trackingProceedings of Graphics Interface 201310.5555/2532129.2532141(63-70)Online publication date: 29-May-2013
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media