research-article

Unlogical instrument: Material-driven gesture-controlled sound installation

Authors:

Zhen Wu,

Ze Gao,

Tristan BraudAuthors Info & Claims

DIS '23 Companion: Companion Publication of the 2023 ACM Designing Interactive Systems Conference

Pages 86 - 89

https://doi.org/10.1145/3563703.3596643

Published: 10 July 2023 Publication History

Get Access

Abstract

This proposal describes the design and demonstration of the Unlogical Instrument. This textile-based gesture-controlled sound interface invites the audience to generate sound by interacting with the textile using different gestures, such as poking, flicking, and patting. Unlogical is a statement about breaking the dogmatic approach of commonly used musical instruments. It brings resilience to the form of musical expression and interaction for the audience. It introduces an interactive mechanism mediated by intuitive human gestures and the curiosity to explore tangible material. Unlogical Instrument locates the focus on the textile. This artwork investigates the relationship between the gestures and the sound perception that originates from the textile through a material-driven approach. Inkjet printing is applied to transfer the original textile into a sensorial interface. The audience will listen to the sound while interacting with the textile surface, exploring and understanding this instrument through free-form playing.

Supplemental Material

MP4 File

Introduction Video

Download
51.64 MB

References

[1]

Ze Gao, Anqi Wang, Pan Hui, and Tristan Braud. 2022. Meditation in Motion: Interactive Media Art Visualization Based on Ancient Tai Chi Chuan. In Proceedings of the 30th ACM International Conference on Multimedia. 7241–7242.

Digital Library

Google Scholar

[2]

Ze Gao, Xiaolin Zhang, Sijin Zhu, and Tristan Braud. 2022. Metanalysis: A connected-wall video installation based on the movement of traditional Chinese ink painting. In 2022 Third International Conference on Digital Creation in Arts, Media and Technology (ARTeFACTo). IEEE, 1–6.

Crossref

Google Scholar

[3]

Berit Greinke, Giorgia Petri, Pauline Vierne, Paul Biessmann, Alexandra Börner, Kaspar Schleiser, Emmanuel Baccelli, Claas Krause, Christopher Verworner, and Felix Biessmann. 2021. An Interactive Garment for Orchestra Conducting: IoT-enabled Textile & Machine Learning to Direct Musical Performance. In Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction. 1–6.

Digital Library

Google Scholar

[4]

Mengqi Jiang, Vijayakumar Nanjappan, Hai-Ning Liang, and Martijn ten Bhömer. 2022. GesFabri: Exploring Affordances and Experience of Textile Interfaces for Gesture-based Interaction. Proceedings of the ACM on Human-Computer Interaction 6, EICS (2022), 1–23.

Digital Library

Google Scholar

[5]

Elvin Karana, Bahareh Barati, Valentina Rognoli, Anouk Zeeuw Van Der Laan, 2015. Material driven design (MDD): A method to design for material experiences. International journal of design 9, 2 (2015), 35–54.

Google Scholar

[6]

Monika loss and Christina Kuehnl. 2019. Feel the Music! Exploring the Cross-modal Correspondence between Music and Haptic Perceptions of Softness. Journal of Retailing 95, 4 (2019), 158–169.

Crossref

Google Scholar

[7]

Erica Robles and Mikael Wiberg. 2010. Texturing the "Material Turn" in Interaction Design. In Proceedings of the Fourth International Conference on Tangible, Embedded, and Embodied Interaction (Cambridge, Massachusetts, USA) (TEI ’10). Association for Computing Machinery, New York, NY, USA, 137–144. https://doi.org/10.1145/1709886.1709911

Digital Library

Google Scholar

[8]

Rebecca Stewart. 2019. Cords and chords: Exploring the role of e-textiles in computational audio. Frontiers in ICT 6 (2019), 2.

Crossref

Google Scholar

[9]

Benjamin Tag, Takuya Goto, Kouta Minamizawa, Ryan Mannschreck, Haruna Fushimi, and Kai Kunze. 2017. atmoSphere: mindfulness over haptic-audio cross modal correspondence. In Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers. 289–292.

Digital Library

Google Scholar

[10]

Gili Weinberg, Maggie Orth, and Peter Russo. 2000. The Embroidered Musical Ball: A Squeezable Instrument for Expressive Performance. In CHI ’00 Extended Abstracts on Human Factors in Computing Systems (The Hague, The Netherlands) (CHI EA ’00). Association for Computing Machinery, New York, NY, USA, 283–284. https://doi.org/10.1145/633292.633457

Digital Library

Google Scholar

[11]

Irmandy Wicaksono, Don Derek Haddad, and Joseph Paradiso. 2022. Tapis Magique: Machine-Knitted Electronic Textile Carpet for Interactive Choreomusical Performance and Immersive Environments. In Creativity and Cognition (Venice, Italy). Association for Computing Machinery, New York, NY, USA, 262–274. https://doi.org/10.1145/3527927.3531451

Digital Library

Google Scholar

[12]

Irmandy Wicaksono and Joseph Paradiso. 2020. KnittedKeyboard: Digital knitting of electronic textile musical controllers. In Proceedings of the International Conference on New Interfaces for Musical Expression. 323–326.

Google Scholar

Cited By

View all

Guo MZhang XNiu ZGao Z(2023)Envisioning Wearable Color-Changing Device to Facilitate Emotion Recognition and Communication in Children with Autism Spectrum DisorderProceedings of the Eleventh International Symposium of Chinese CHI10.1145/3629606.3629642(367-373)Online publication date: 13-Nov-2023
https://dl.acm.org/doi/10.1145/3629606.3629642

Recommendations

Combination of Resnet and Spatial Pyramid Pooling for Musical Instrument Identification
Abstract
Identifying similar objects is one of the most challenging tasks in computer vision image recognition. The following musical instruments will be recognized in this study: French horn, harp, recorder, bassoon, cello, clarinet, erhu, guitar ...
Morphing Musical Instrument Sounds with the Sinusoidal Model in the Sound Morphing Toolbox
Perception, Representations, Image, Sound, Music
Abstract
Sound morphing stands out among the sound transformation techniques in the literature due to its creative and research potential. The aim of sound morphing is to gradually blur the categorical distinction between the source and target sounds by ...
Analyzing several musical instrument tones using the randomly modulated periodicity model

The waveform of a simple sustained tone emitted from musical instruments such as a flute played by even the best musician is never exactly periodic. There is always some variation over time in the waveform of a single pitch that is characteristic of the ...

Comments

Information & Contributors

Information

Published In

DIS '23 Companion: Companion Publication of the 2023 ACM Designing Interactive Systems Conference

July 2023

296 pages

ISBN:9781450398985

DOI:10.1145/3563703

Editors:
Daragh Byrne
Carnegie Mellon University
,
Nikolas Martelaro
Carnegie Mellon University
,
Andy Boucher
Northumbria University
,
David Chatting
Goldsmiths, University of London
,
Sarah Fdili Alaoui
LISN-Université Paris Saclay
,
Sarah Fox
Carnegie Mellon University
,
Iohanna Nicenboim
Delft University of Technology
,
Cayley MacArthur
University of Waterloo

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 July 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

DIS '23

Sponsor:

SIGCHI

DIS '23: Designing Interactive Systems Conference

July 10 - 14, 2023

PA, Pittsburgh, USA

Acceptance Rates

Overall Acceptance Rate 1,158 of 4,684 submissions, 25%

Upcoming Conference

DIS '25

Sponsor:
sigchi

Designing Interactive Systems Conference

July 5 - 9, 2025

Funchal , Portugal

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
154
Total Downloads

Downloads (Last 12 months)67
Downloads (Last 6 weeks)14

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Guo MZhang XNiu ZGao Z(2023)Envisioning Wearable Color-Changing Device to Facilitate Emotion Recognition and Communication in Children with Autism Spectrum DisorderProceedings of the Eleventh International Symposium of Chinese CHI10.1145/3629606.3629642(367-373)Online publication date: 13-Nov-2023
https://dl.acm.org/doi/10.1145/3629606.3629642

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Abstract

Supplemental Material

References

Cited By

Recommendations

Combination of Resnet and Spatial Pyramid Pooling for Musical Instrument Identification

Morphing Musical Instrument Sounds with the Sinusoidal Model in the Sound Morphing Toolbox

Analyzing several musical instrument tones using the randomly modulated periodicity model

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

HTML Format

Share

Share this Publication link

Share on social media

Affiliations