Abstract
We explore a new conceptual framework for interaction design in the context of ubiquitous musical activities, banging interaction. Our proposal entails a shift of focus from instruments, tools or device-oriented approaches to a timbre-led design practice that targets the potential relationships between sonic and multimodal qualities and the local available technological resources. One thread of this perspective envisages more flexible usage through the application of adaptive techniques. Another thread engages with recent advances in mid-air interaction, hinting at multimodal sensing. Yet another thread involves the incorporation of an aesthetically pliable approach to sonic materials, materialised through timbre as a target, rather than ‘notes’, ‘instruments’ or ‘orchestras’. We present two examples of deployments: The prototype mixDroid, one of the first systems to employ mobile devices in everyday settings for creative purposes, and the prototype Dynamic Drum Collective, an adaptive implementation for percussive sources based on visual tokens and whole-body interaction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
By “acoustic-instrumental thinking” we describe the social and technological approaches attached to the European instrumental musical tradition of the nineteenth century (i.e. the separation between the audience, the composer and the performer; the adoption of domain-specific knowledge as a requirement for music making; the enforcement of the myth of the genius as the source of creativity embodied by the virtuoso player, conductor or composer).
- 2.
The temporal investment to achieve expertise has been reported at approximately ten thousand hours. The investment required to apply pre-existing knowledge to a different domain is still an open question. In any case, when dealing with casual interaction to require extensive musical training means to exclude a large number of potential participants.
- 3.
There is an ongoing discussion of these issues within the ubimus community, with partial but promising results. See [16] for various examples.
- 4.
“A tendency to perceive an object only in terms of its common use.” https://dictionary.apa.org/functional-fixedness..
- 5.
Our interest for such a composite and poorly described social group is rooted in Milton Babbitt’s (1958) well-known essay ‘Who Cares if You Listen?’ [23], and needs to be understood as a critique of the overt elitism incorporated in some narratives of musical creative practice. We decided to keep using the adjective ‘lay’ with reference to Babbitt’s exhaustive mention of the ‘layman’, despite the fact that we find the term highly charged and stereotyped.
References
Adams, A.T., Costa, J., Jung, M.F., Choudhury, T.: Mindless computing: designing technologies to subtly influence behavior. In: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp 2015, pp. 719–730. Association for Computing Machinery, New York (2015). https://doi.org/10.1145/2750858.2805843
Algethami, N., Redfern, S.: A robust tracking-by-detection algorithm using adaptive accumulated frame differencing and corner features. J. Imaging 6(4), 6040025. https://doi.org/10.3390/jimaging6040025, https://www.mdpi.com/2313-433X/6/4/25
Babbitt, M.: Who cares if you listen? (1958). In: Strunk, O., Treitler, L., Morgan, R.P. (eds.) Source Readings in Music History, vol. 7: The Twentieth Century, pp. 35–41. Norton, New York (1998). http://www.palestrant.com/babbitt.html
Bolt, R.A.: “put-that-there": Voice and gesture at the graphics interface. In: Proceedings of the 7th Annual Conference on Computer Graphics and Interactive Techniques, pp. 262–270. ACM, New York (1980)
Chakraborty, S., Yaseen, A., Timoney, J., Lazzarini, V., Keller, D.: Adaptive touchless whole-body interaction for casual ubiquitous musical activities. In: Proceedings of the International Computer Music Conference (ICMC2022), pp. 132–138. University of Limerick, Limerick (2022)
Chattopadhyay, D.: Understanding interaction mechanics in touchless target selection. PhD Thesis in Computer Science. Indiana University, Bloomington, IN (2016)
Connors, T.: From the edge. In: Proceedings of the 11th Workshop on Ubiquitous Music (UbiMus 2021), pp. 17–19. g-ubimus (2021)
Cope, D.: Virtual Music: Computer Synthesis of Musical Style. MIT Press, Cambridge (2004). http://books.google.com.br/books?id=dJ7Numf8fwMC
Flores, L.V., Pimenta, M.S., Keller, D.: Patterns of musical interaction with computing devices. Cadernos de Informática 8(2), 68–81 (2014). http://compmus.ime.usp.br/ubimus/pt-br/node/33
Graham-Knight, K., Tzanetakis, G.: Adaptive music technology: history and future perspectives. In: Proceedings of Internationl Computer Music Conference (ICMC 2015), pp. 416–419. CEMI, University of North Texas, Denton (2015)
Keller, D., Aliel, L., Filho, M.C., Costalonga, L.: Toward ubimus philosophical frameworks. Open Phil. 4(1), 353–371 (2021). https://doi.org/10.1515/opphil-2020-0188
Keller, D., Lazzarini, V., Pimenta, M.S.: Ubiquitous Music, vol. XXVIII. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-319-11152-0
Keller, D., Otero, N., Lazzarini, V., Pimenta, M.S., Lima, M.H., Johann, M., Costalonga, L.: Relational properties in interaction aesthetics: The ubiquitous music turn. In: Ng, K., Bowen, J.P., McDaid, S. (eds.) Proceedings of the Electronic Visualisation and the Arts Conference (EVA 2014), , London, UK, 8–14 July 2014. BCS, Computer Arts Society Specialist Group, London. https://doi.org/10.1426/ewic/eva2104.25
Keller, D., Simurra, I., Messina, M., Neiva, T.M., Tedesco, S., Mesz, B.: Domestic ubimus. EAI Endorsed Trans. Creative Technol. 9(30), 173493 (2022). https://doi.org/10.4108/eai.22-2-2022.173493
Keller, D., Gomes, C., Aliel, L.: The handy metaphor: bimanual, touchless interaction for the internet of musical things. J. New Music Res. 48(4), 385–396 (2019). https://doi.org/10.1080/09298215.2019.1654525
Kramann, G.: Composing by laypeople: a broader perspective provided by arithmetic operation grammar. Comput. Music J. 44(1), 17–34 (2020)
Lazzarini, V., Keller, D., Otero, N., Turchet, L.: Ubiquitous Music Ecologies. Taylor & Francis (Routledge), London (2020)
Lima, M.H., Keller, D., Pimenta, M.S., Lazzarini, V., Miletto, E.M.: Creativity-centred design for ubiquitous musical activities: two case studies. J. Music Technol. Educ. 5(2), 195–222 (2012). https://doi.org/10.1386/jmte.5.2.195_1
Lugaresi, C., et al.: Mediapipe: a framework for building perception pipelines. arXiv preprint arXiv:1906.08172 (2019)
Mackay, R., Rodríguez, J., Rodríguez, R., Blink, D., López, P.J.: Networking the flight of the monarchs (performance). In: Proceedings of the Ubiquitous Music Symposium (UbiMus 2022), pp. 141–142. g-ubimus, Curitiba, PR (2022)
Messina, M., Aliel, L.: Ubiquitous music, gelassenheit and the metaphysics of presence: Hijacking the live score piece ntrallazzu 4. In: Proceedings of the Workshop on Ubiquitous Music (UbiMus2019), Marseille, France, 14–18 October 2019, pp. 685–695. Proceedings of the 14th International Symposium on CMMR, Marseille (2019)
Messina, M., Célio Filho, M., Mejía, C.M.G., Keller, D., Aliel, L., Simurra, I.: A internet do bagulho musical (internet of musical stuff)-iomust. In: Proceedings of the Ubiquitous Music Symposium (ubimus2022), pp. 85–93. g-ubimus (2022)
Pinheiro da Silva, F., Pimenta, M.S., Lazzarini, V., Keller, D.: Time tagging in its niche: engagement, explorability and creative attention (a marcação temporal no seu nicho: Engajamento, explorabilidade e atenção criativa). Cadernos de Informática 8(2) (2014). http://seer.ufrgs.br/cadernosdeinformatica
Subramanian, S., Seah, S.A., Shinoda, H., Hoggan, E., Corenthy, L.: Mid-air haptics and displays: Systems for un-instrumented mid-air interactions. In: Proceedings of the Conference on Human Factors in Computing Systems (CHI2015). ACM, New York (2015)
Acknowledgement
This research was partially funded by the Brazilian National Council for the Development of Science and Technology, grants [CNPq 202559/2020-3, 308790/2021-9] to DK.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Keller, D. et al. (2023). Banging Interaction in Ubiquitous Music. In: Brooks, A.L. (eds) ArtsIT, Interactivity and Game Creation. ArtsIT 2022. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 479. Springer, Cham. https://doi.org/10.1007/978-3-031-28993-4_34
Download citation
DOI: https://doi.org/10.1007/978-3-031-28993-4_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-28992-7
Online ISBN: 978-3-031-28993-4
eBook Packages: Computer ScienceComputer Science (R0)