skip to main content
article

Interactive multi-scale visualizations of tonal evolution in MuSA.RT Opus 2

Published: 01 October 2005 Publication History

Abstract

The purpose of this article is to describe MuSA.RT Opus 2, an interactive system for tonal visualization of music at multiple scales, and to present examples of the types of musical features and attributes that can be abstracted and visualized by the system. MuSA.RT aims to create an environment by which musical performances can be mapped in real-time to a concrete and visual metaphor for tonal space, wherein we can see the establishment and evolution of the tonal context. In this environment, expert musicians will be able to see the tonal structures of what they play, initiated listeners will be able to visually follow the structures that they hear, and novices can learn to hear the structures that they see. MuSA.RT is both an interactive art installation that can convert musical performances to mathematically elegant graphics and a scientific tool for visualizing the inner workings of tonal induction and tracking algorithms. In this article we describe the mapping strategies for transforming a MIDI stream into tonal structures in 3D space, and our solution for overcoming the challenge of real-time concurrent processing of data streams; we will also give examples and present case studies of visual mappings of music by Pachelbel, Bach, and Barber.

Supplementary Material

JPG File (bach-bwv1007.jpg)
JPG File (barber-adagio.jpg)
JPG File (pachelbel-canon.jpg)
MP4 File (bach-bwv1007.mp4)
Chew and Francois music file #1
MP4 File (barber-adagio.mp4)
Chew and Francois music file #2
MP4 File (pachelbel-canon.mp4)
Chew and Francois music file #3

References

[1]
Chew, E. 2000. Towards a mathematical model of tonality. Ph.D. dissertation, Massachusetts Institute of Technology, Cambridge, MA.
[2]
Chew, E. 2005/06. Slicing it all ways: Mathematical models for induction, approximation and segmentation using the spiral array. INFORMS J. Computing. To appear.
[3]
Chew, E. 2005. Thinking out of the grid and inside the spiral - Geometric interpretations of and comparisons with the spiral array model. Computing in Musicology 15 (2005). Special issue on harmonic and rhythmic analysis. To appear.
[4]
Chew, E. And Chen, Y.-C. 2005. Real-time pitch spelling using the spiral array. Computer Music J. 29, 2 (Summer 2005). To appear
[5]
Chew, E. And Chen, Y.-C. 2003b. Determining context-defining windows: Pitch spelling using the spiral array. In Proceedings of the 4th International Conference for Music Information Retrieval (ISMIR, Baltimore, MD, Oct. 26-30, 2003).
[6]
Chew, E. And Chen, Y.-C. 2003a. Mapping MIDI to the spiral array: Disambiguating pitch spellings. In Computational Modeling and Problem Solving in the Networked World, Proceedings of the 8th INFORMS Computer Society Conference (ICS2003, Chandler, AZ, Jan 8-10, 2003). Kluwer Academic. 259-275.
[7]
Chew, E. And Francois, A. R. J. 2003. MuSA.RT - Music on the spiral array. Real-time. In Proceedings of the ACM Multimedia Conference (Berkeley, CA, 2003). ACM, New York.
[8]
Francois, A. R. J. 2004. A hybrid architectural style for distributed parallel processing of generic data streams. In Proceedings of the International Conference on Software Engineering (Edinburgh, Scotland), 367-376.
[9]
Francois, A. R. J. 2002. Components for immersion. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME '02, Lausanne, Switzerland, Aug. 2002).
[10]
Francois, A. R. J. 2001. Modular flow scheduling framework (MFSM): mfsm.sourceforge.net.
[11]
Krumhansl, C. L. 1990. Cognitive Foundations of Musical Pitch. Oxford University Press. Oxford, England.
[12]
Sapp, C. S. 2001. Harmonic visualizations of tonal music." In Proceedings of the International Computer Music Conference (2001). 423-430. Tonal Landscape Gallery. ccrma.stanford.edu/~craig/keyscape.
[13]
Toiviainen, P. And Krumhansl, C. L. 2003. Measuring and modeling real-time responses to music: The dynamics of tonality induction. Perception 32, 6 (2003), 741-766.

Cited By

View all
  • (2021)Music, Computing, and Health: A Roadmap for the Current and Future Roles of Music Technology for Health Care and Well-BeingMusic & Science10.1177/20592043219977094Online publication date: 31-May-2021
  • (2021)A Survey of Music Visualization TechniquesACM Computing Surveys10.1145/346183554:7(1-29)Online publication date: 18-Jul-2021
  • (2011)Two-dimensional visual inspection of pitch-space, many time-scales and tonal uncertainty over timeProceedings of the Third international conference on Mathematics and computation in music10.5555/2026243.2026255(140-150)Online publication date: 15-Jun-2011
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Computers in Entertainment
Computers in Entertainment   Volume 3, Issue 4
Theoretical and Practical Computer Applications in Entertainment
October 2005
116 pages
EISSN:1544-3574
DOI:10.1145/1095534
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 October 2005
Published in CIE Volume 3, Issue 4

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. computational music cognition
  2. geometric models
  3. harmonic analysis
  4. music visualization
  5. tonal analysis
  6. tonality models

Qualifiers

  • Article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)10
  • Downloads (Last 6 weeks)1
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2021)Music, Computing, and Health: A Roadmap for the Current and Future Roles of Music Technology for Health Care and Well-BeingMusic & Science10.1177/20592043219977094Online publication date: 31-May-2021
  • (2021)A Survey of Music Visualization TechniquesACM Computing Surveys10.1145/346183554:7(1-29)Online publication date: 18-Jul-2021
  • (2011)Two-dimensional visual inspection of pitch-space, many time-scales and tonal uncertainty over timeProceedings of the Third international conference on Mathematics and computation in music10.5555/2026243.2026255(140-150)Online publication date: 15-Jun-2011
  • (2007)LettersComputer Music Journal10.5555/1326598.132660031:4(5-10)Online publication date: 1-Dec-2007
  • (2007)NewsComputer Music Journal10.1162/comj.2007.31.4.1131:4(11-14)Online publication date: 1-Dec-2007
  • (2007)About this issueComputer Music Journal10.1162/comj.2007.31.4.131:4(1-2)Online publication date: 1-Dec-2007
  • (2007)Audio key findingEURASIP Journal on Advances in Signal Processing10.1155/2007/565612007:1(156-156)Online publication date: 1-Jan-2007
  • (2006)An architectural framework for interactive music systemsProceedings of the 2006 conference on New interfaces for musical expression10.5555/1142215.1142252(150-155)Online publication date: 4-Jun-2006

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media