ABSTRACT
We present an interactive virtual percussion instrument system, Tabletop Ensemble, that can be used by a group of collaborative users simultaneously to emulate playing music in real world while providing them with flexibility of virtual simulations. An optical multi-touch tabletop serves as the input device. A novel touch handling algorithm for such devices is presented to translate users' interactions into percussive control signals appropriate for music playing. These signals activate the proposed sound simulation system for generating realistic user-controlled musical sounds. A fast physically-based sound synthesis technique, modal synthesis, is adopted to enable users to directly produce rich, varying musical tones, as they would with the real percussion instruments. In addition, we propose a simple coupling scheme for modulating the synthesized sounds by an accurate numerical acoustic simulator to create believable acoustic effects due to cavity in music instruments. This paradigm allows creating new virtual percussion instruments of various materials, shapes, and sizes with little overhead. We believe such an interactive, multi-modal system would offer capabilities for expressive music playing, rapid prototyping of virtual instruments, and active exploration of sound effects determined by various physical parameters in a classroom, museum, or other educational settings. Virtual xylophones and drums with various physics properties are shown in the presented system.
Supplemental Material
- Adrien, J.-M. 1991. Representations of musical signals. MIT Press, Cambridge, MA, USA, ch. The missing link: modal synthesis, 269--298. Google ScholarDigital Library
- Bilbao, S. 2009. Numerical Sound Synthesis. Wiley Online Library.Google Scholar
- Bonneel, N., Drettakis, G., Tsingos, N., Viaud-Delmon, I., and James, D. 2008. Fast modal sounds with scalable frequency-domain synthesis. In ACM SIGGRAPH 2008 papers, ACM, New York, NY, USA, SIGGRAPH '08, 24:1--24:9. Google ScholarDigital Library
- Boyd, J. P. 2001. Chebyshev and Fourier Spectral Methods: Second Revised Edition, 2 revised ed. Dover Publications, December.Google Scholar
- Brebbia, C. A. 1991. Boundary Element Methods in Acoustics, 1 ed. Springer, October.Google Scholar
- Bruyns, C. 2006. Modal synthesis for arbitrarily shaped objects. Computer Music Journal 30, 3, 22--37. Google ScholarDigital Library
- Buxton, W., Hill, R., and Rowley, P. 1985. Issues and techniques in touch-sensitive tablet input. In ACM SIGGRAPH Computer Graphics, vol. 19, ACM, 215--224. Google ScholarDigital Library
- Chandak, A., Lauterbach, C., Taylor, M., Ren, Z., and Manocha, D. 2008. Ad-frustum: Adaptive frustum tracing for interactive sound propagation. IEEE Transactions on Visualization and Computer Graphics 14, 1707--1722. Google ScholarDigital Library
- Chuchacz, K., O'Modhrain, S., and Woods, R. 2007. Physical models and musical controllers: designing a novel electronic percussion instrument. In NIME '07: Proceedings of the 7th international conference on New interfaces for musical expression, ACM, New York, NY, USA, 37--40. Google ScholarDigital Library
- Cook, P. 2002. Real sound synthesis for interactive applications. AK Peters, Ltd. Google ScholarDigital Library
- Davidson, P., and Han, J. 2006. Synthesis and control on large scale multi-touch sensing displays. In Proceedings of the 2006 conference on New interfaces for musical expression, IRCAM-Centre Pompidou, 216--219. Google ScholarDigital Library
- Deines, E., Michel, F., Bertram, M., Hagen, H., and Nielson, G. 2006. Visualizing the phonon map. In Eurovis. Google ScholarDigital Library
- Framework Named pipes, M. I. 2011. Named pipes.Google Scholar
- Funkhouser, T., Tsingos, N., and Jot, J.-M. 2003. Survey of methods for modeling sound propagation in interactive virtual environment systems. Presence and Teleoperation.Google Scholar
- Funkhouser, T., Tsingos, N., Carlbom, I., Elko, G., Sondhi, M., West, J. E., Pingali, G., Min, P., and Ngan, A. 2004. A beam tracing method for interactive architectural acoustics. The Journal of the Acoustical Society of America 115, 2, 739--756.Google ScholarCross Ref
- Guski, R., and Troje, N. 2003. Audiovisual phenomenal causality. Perception & psychophysics 65, 5, 789.Google Scholar
- Han, J. 2005. Low-cost multi-touch sensing through frustrated total internal reflection. In Proceedings of the 18th annual ACM symposium on User interface software and technology, ACM, 115--118. Google ScholarDigital Library
- Hochenbaum, J., and Vallis, O. 2009. Bricktable: A musical tangible multi-touch interface. Proceedings of Berlin Open Converence 09.Google Scholar
- Kaltenbrunner, M., Jorda, S., Geiger, G., and Alonso, M. 2006. The reactable*: A collaborative musical instrument.Google Scholar
- Miranda, E., and Wanderley, M. 2006. New digital musical instruments: control and interaction beyond the keyboard. AR Editions, Inc. Google ScholarDigital Library
- O'Brien, J., Shen, C., and Gatchalian, C. 2002. Synthesizing sounds from rigid-body simulations. In Proc. of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, ACM Press New York, NY, USA, 175--181. Google ScholarDigital Library
- Raghuvanshi, N., and Lin, M. 2006. Interactive sound synthesis for large scale environments. In Proceedings of the 2006 symposium on Interactive 3D graphics and games, ACM, 101--108. Google ScholarDigital Library
- Raghuvanshi, N., Narain, R., and Lin, M. C. 2009. Efficient and accurate sound propagation using adaptive rectangular decomposition. IEEE Transactions on Visualization and Computer Graphics 15 (September), 789--801. Google ScholarDigital Library
- Raghuvanshi, N., Snyder, J., Mehra, R., Lin, M., and Govindaraju, N. 2010. Precomputed wave simulation for real-time sound propagation of dynamic sources in complex scenes. ACM Trans. Graph. 29 (July), 68:1--68:11. Google ScholarDigital Library
- Ren, Z., Yeh, H., and Lin, M. C. 2011. Example-Guided Physically-Based Sound Synthesis. UNC Techinical Report.Google Scholar
- Rosenberg, I., and Perlin, K. 2009. The unmousepad: an interpolating multi-touch force-sensing input pad. In ACM SIG-GRAPH 2009 papers, ACM, 1--9. Google ScholarDigital Library
- Sakamoto, S., Ushiyama, A., and Nagatomo, H. 2006. Numerical analysis of sound propagation in rooms using the finite difference time domain method. The Journal of the Acoustical Society of America 120, 5, 3008.Google ScholarCross Ref
- Schöening, J., Hook, J., Motamedi, N., Olivier, P., Echtler, F., Brandl, P., Muller, L., Daiber, F., Hilliges, O., Loechtefeld, M., Roth, T., Schmidt, D., and von Zadow, U. 2009. Building interactive multi-touch surfaces. Journal of Graphics, GPU, and Game Tools 14, 3, 35--55.Google ScholarCross Ref
- Shabana, A. 1997. Vibration of discrete and continuous systems. Springer Verlag.Google Scholar
- Si, H. 2011. TetGen: A Quality Tetrahedral Mesh Generator and a 3D Delaunay Triangulator.Google Scholar
- Thompson, L. L. 2006. A review of finite-element methods for time-harmonic acoustics. The Journal of the Acoustical Society of America 119, 3, 1315--1330.Google ScholarCross Ref
- Van Den Doel, K., and Pai, D. 1998. The sounds of physical shapes. PRESENCE-CAMBRIDGE MASSACHUSETTS-7, 382--395. Google ScholarDigital Library
- Weinberg, G., and Driscoll, S. 2007. The interactive robotic percussionist: new developments in form, mechanics, perception and interaction design. In Proceedings of the ACM/IEEE international conference on Human-robot interaction, ACM, New York, NY, USA, HRI '07, 97--104. Google ScholarDigital Library
- XAudio2, M. 2011. XAudio2.Google Scholar
Index Terms
Tabletop Ensemble: touch-enabled virtual percussion instruments
Recommendations
Designing virtual instruments with touch-enabled interface
CHI EA '12: CHI '12 Extended Abstracts on Human Factors in Computing SystemsWe present and discuss the design of a virtual musical instrument system that can be used by a collaborative group of users to emulate playing percussive music. An optical multi-touch tabletop serves as the input device for multiple users, and an ...
Building a multi-touch tabletop for classrooms
Edutainment'11: Proceedings of the 6th international conference on E-learning and games, edutainment technologiesThis paper aims to build a compact multi-touch tabletop that can be easily integrated with a normal classroom desk to become a student touch desk. Various state-of-the-art multi-touch technologies are investigated. Based on the analysis of their multi-...
Hands-on interactive tabletop LEGO application
ACE '11: Proceedings of the 8th International Conference on Advances in Computer Entertainment TechnologyPresently, multi-touch interactive surfaces have widespread adoption as entertainment devices. Taking advantage of such devices, we present an interactive LEGO application, developed accordingly to an adaptation of building block metaphors and direct ...
Comments