skip to main content
10.1145/2817721.2820986acmconferencesArticle/Chapter ViewAbstractPublication PagesissConference Proceedingsconference-collections
abstract

Interactive Spaces: Models and Algorithms for Reality-based Music Applications

Published: 15 November 2015 Publication History

Abstract

Reality-based interfaces have the property of linking the user's physical space with the computer digital content, bringing in intuition, plasticity and expressiveness. Moreover, applications designed upon motion and gesture tracking technologies involve a lot of psychological features, like space cognition and implicit knowledge. All these elements are the background of three presented music applications, employing the characteristics of three different interactive spaces: a user centered three dimensional space, a floor bi-dimensional camera space, and a small sensor centered three dimensional space. The basic idea is to deploy the application's spatial properties in order to convey some musical knowledge, allowing the users to act inside the designed space and to learn through it in an enactive way.

References

[1]
1. Brower, C. A cognitive theory of musical meaning. Journal of Music Theory (2000), 323--379.
[2]
2. Bruner, J. S. The act of discovery. Harvard educational review (1961).
[3]
3. Engelbart, D. C. Augmenting human intellect: a conceptual framework (1962). Packer, Randall and Jordan, Ken. Multimedia. From Wagner to Virtual Reality. New York: WW Norton & Company (2001), 64--90.
[4]
4. Harrison, S., and Dourish, P. Re-place-ing space: the roles of place and space in collaborative systems. In Proc. ACM conference on Computer supported cooperative work, ACM (1996), 67--76.
[5]
5. Jetter, H.-C., Reiterer, H., and Geyer, F. Blended interaction: understanding natural human-computer interaction in post-wimp interactive spaces. Personal and Ubiquitous Computing 18, 5 (2014), 1139--1158.
[6]
6. Lakoff, G., and Johnson, M. Metaphors we live by. University of Chicago press, 2008.
[7]
7. Mandanici, M., and Canazza, S. The "hand composer": gesture-driven music composition machines. In Proc. 13th Conf. on Intelligent Autonomous Systems (2014), 553--560.
[8]
8. Mandanici, M., Roda', A., and Canazza, S. The harmonic walk: an interactive educational environment to discover musical chords. In Proc. ICMC-SMC Conference (2014).
[9]
9. Mandanici, M., and Sapir, S. Disembodied voices: a kinect virtual choir conductor. In Proc. 9th Sound and Music Computing Conference (2012), 271--276.
[10]
10. Marrin, T., and Picard, R. The "conductor's jacket": A device for recording expressive musical gestures. In Proc. International Computer Music Conference (1998), 215--219.
[11]
11. Montello, D. R. International Encyclopedia of the Social and Behavioral Sciences. Oxford, Pergamon Press, 2001, ch. Spatial Cognition, 14771--14775.
[12]
12. Murphy, D., Andersen, T. H., and Jensen, K. Conducting audio files via computer vision. In Gesture-based communication in human-computer interaction. Springer, 2004, 529--540.
[13]
13. Povel, D.-J., and Jansen, E. Harmonic factors in the perception of tonal melodies. Music Perception 20, 1 (2002), 51--85.
[14]
14. Reber, A. S. Implicit learning and tacit knowledge. Journal of experimental psychology: General 118, 3 (1989), 219.
[15]
15. Rudolf, M., and Stern, M. The grammar of conducting: A comprehensive guide to baton technique and interpretation. Schirmer Books, New York, 1994.
[16]
16. Tillmann, B., Bharucha, J. J., and Bigand, E. Implicit learning of tonality: a self-organizing approach. Psychological review 107, 4 (2000), 885.
[17]
17. Tversky, B. Functional Significance of Visuospatial Representations. Handbook of higher-level visuospatial thinking Handbook of (2005), 1--34.
[18]
18. Zanolla, S., Canazza, S., Roda', A., Camurri, A., and Volpe, G. Entertaining listening by means of the Stanza Logo-Motoria: an Interactive Multimodal Environment. Entertainment Computing 2013, 4 (2013), 213--220.

Recommendations

Reviews

Pierre Jouvelot

Doctoral sessions are a staple of scientific conferences; they allow early research to be advertised and presented by upcoming PhD students in less intimidating formats than full lectures. Marcella Mandanici presented her recent work at the 2015 International Conference on Interactive Tabletops and Surfaces, describing three experiments that combine music learning, practice, and discovery with 2D and 3D spatial and/or geometrical representation of music concepts. By taking advantage of implicit spatial knowledge to accomplish complex tasks, the idea is to practice "enactivism," or "learning by doing," via what is called blended interaction theory. The three presented musical applications are (1) "Disembodied Voices," where a Kinect sensor is used to capture gestures, to be used as parameters for conducting a prerecorded chorus; (2) "The Harmonic Walk," where a ceiling-based camera is used to track a user's movements and harmonize a melody; and (3) "Hand Composer," where a Leap Motion hand-tracking device drives a musical instrument digital interface (MIDI) synthesizer. Links to YouTube videos are provided for each of these three installations. This short, easy-to-read paper can be useful to computer music researchers interested in the relationship between 2D and 3D space factors and music production; researchers focused on general pedagogy issues could also benefit from it. As mentioned by Mandanici, music knowledge acquisition is just one possible domain in which her approach can be tested; science, technology, engineering, and mathematics (STEM) disciplines could profit from a similar reality- and arts-based learning approach. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ITS '15: Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces
November 2015
522 pages
ISBN:9781450338998
DOI:10.1145/2817721
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 November 2015

Check for updates

Author Tags

  1. blended interaction
  2. learning environments
  3. sound augmented reality

Qualifiers

  • Abstract

Conference

ITS '15
Sponsor:
ITS '15: Interactive Tabletops and Surfaces
November 15 - 18, 2015
Madeira, Portugal

Acceptance Rates

ITS '15 Paper Acceptance Rate 29 of 122 submissions, 24%;
Overall Acceptance Rate 119 of 418 submissions, 28%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 137
    Total Downloads
  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media