demonstration

WhammyPhone: Exploring Tangible Audio Manipulation Using Bend Input on a Flexible Smartphone

Authors:

Lahiru Priyadarshana,

Juan Pablo Carrascal,

Roel VertegaalAuthors Info & Claims

UIST '16 Adjunct: Adjunct Proceedings of the 29th Annual ACM Symposium on User Interface Software and Technology

Pages 159 - 161

https://doi.org/10.1145/2984751.2985742

Published: 16 October 2016 Publication History

Abstract

We present WhammyPhone, a novel audio interface that supports physical manipulation of digital audio through bend gestures. WhammyPhone combines a high-resolution flexible display, bend sensors, and a set of intuitive interaction techniques that enable novice users to manipulate sound in a tangible fashion. With WhammyPhone, bend gestures can control both discrete (e.g. triggering a note) and continuous parameters (e.g. pitch bend). We showcase application scenarios that leverage the unique input modalities of WhammyPhone and discuss its potential for digital audio manipulation.

Supplementary Material

suppl.mov (uistde0176-file3.mp4)

Supplemental video

Download
38.50 MB

References

[1]

Ableton Live. https://www.ableton.com/en

[2]

Ableton Connection Kit. https://www.ableton.com/en/packs/connection-kit

[3]

Apple GarageBand for iOS. www.apple.com/ios/garageband

[4]

Bert, S., and Jean, V. AudioCubes: a Distributed Cube Tangible Interface based on Interaction Range for Sound Design. In Proc. TEI '08, ACM Press (2008), 3--10.

Digital Library

[5]

Byron, L., Audrey, G., Winslow, B., and Roel, V. PaperPhone: Understanding the Use of Bend Gestures in Mobile Devices with Flexible Electronic Paper Displays. In Proc. CHI '11, ACM Press (2011), 1303--1312.

Digital Library

[6]

Caustic. www.singlecellsoftware.com/caustic

[7]

Doug, W., Tim, G., and Roel, V. BendFlip: Examining Input Techniques for Electronic Book Readers with Flexible Form Factors. In Proc. INTERACT '11, 117--133.

Digital Library

[8]

Engin, B., Amit, D., Nancy, Otero., and Paulo, B. BeatTable: a tangible approach to rhythms and ratios. In Proc. IDC '13, ACM (2013), 589--592.

Digital Library

[9]

Frederic, B., et al. SIG NIME: Music, Technology, and Human-Computer Interaction. In CHI EA'13, ACM (2013) 2529--2532.

Digital Library

[10]

Giovanni, M.T., Esben, W.P. and Kasper, H. Deformable Interfaces for Performing Music. In Proc. CHI '16, ACM PRESS (2016), 377--386.

Digital Library

[11]

Ivan, P., Michael, L., Sidney. F. and Tina, B. New interfaces for musical expression. In CHI EA'01, ACM Press (2001), 491--492.

Digital Library

[12]

Jess, R., and Adrian F. Colocated surface sound interaction. In CHI EA '13, ACM PRESS (2013), 3047.

[13]

Luke, D., Nathan, W., and John, V.S. The WaveSaw: a flexible instrument for direct timbral manipulation. In Proc. NIME '07, ACM (2007), 270--272.

Digital Library

[14]

Open Sound Control. http://opensoundcontrol.org

[15]

Paul, S., et al. ReFlex: A Flexible Smartphone with Active Haptic Feedback for Bend Input. In Proc. TEI '16, ACM PRESS (2016), 185--192.

Digital Library

[16]

Reaktor 6. www.nativeinstruments.com/en/products/komplete/synths/reaktor-6

[17]

Roel., V., and Ivan, P. Organic User Interfaces. Comm ACM, 51(6), (2008), 26--30.

[18]

Sergi, J., Günter G., Marcos, A., and Martin, K. The reacTable: Exploring the Synergy between Live Music Performance and Tabletop Tangible Interfaces. In Proc. TEI '07, ACM Press (2007), 139--146.

Digital Library

[19]

Sidney, F. and Michael L. Creating new interfaces for musical expression: introduction to NIME. Proc. SIGGRAPH, (2009), Article No. 11.

[20]

TouchOSC. http://hexler.net/software/touchosc

Cited By

Eady AGuerrero ACheung VThue DGirouard A(2024)BendAide: A Deformable Interface to Augment Touchscreen Mobile DevicesProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670960(1-10)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670960
Yeh YIrudayaraj AVogel D(2024)Single-handed Folding Interactions with a Modified Clamshell Flip PhoneProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642554(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642554
Yin JBai SHan YZhang XDeng SWang S(2021)The Study of Bending and Twisting Input Modalities in Deformable InterfacesElectronics10.3390/electronics1023299110:23(2991)Online publication date: 1-Dec-2021
https://doi.org/10.3390/electronics10232991
Show More Cited By

Index Terms

WhammyPhone: Exploring Tangible Audio Manipulation Using Bend Input on a Flexible Smartphone
1. Applied computing
  1. Arts and humanities
    1. Sound and music computing
2. Information systems
  1. Information retrieval
    1. Specialized information retrieval
      1. Multimedia and multimodal retrieval
        Music retrieval

Recommendations

ReFlex: A Flexible Smartphone with Active Haptic Feedback for Bend Input
TEI '16: Proceedings of the TEI '16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction

ReFlex is a flexible smartphone with bend input and active haptic feedback. ReFlex's features allow the introduction of sensations such as friction or resistance. We report results from an experiment using ReFlex in a targeting task, as well as initial ...
Cobra: flexible displays for mobilegaming scenarios
CHI EA '10: CHI '10 Extended Abstracts on Human Factors in Computing Systems

We discuss Cobra, a handheld peripheral for computer games that applies flexible display design principles to provide a highly intuitive, mobile gaming experience. Cobra is a flexible plastic board interface that uses bends as input to the gaming ...
FlexView: an evaluation of depth navigation on deformable mobile devices
TEI '13: Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction

We present FlexView, a set of interaction techniques for Z-axis navigation on touch-enabled flexible mobile devices. FlexView augments touch input with bend to navigate through depth-arranged content. To investigate Z-axis navigation with FlexView, we ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '16 Adjunct: Adjunct Proceedings of the 29th Annual ACM Symposium on User Interface Software and Technology

October 2016

244 pages

ISBN:9781450345316

DOI:10.1145/2984751

General Chairs:
Jun Rekimoto
The University of Tokyo
,
Takeo Igarashi
The University of Tokyo
,
Program Chairs:
Jacob O. Wobbrock
University of Washington
,
Daniel Avrahami
FXPAL

Copyright © 2016 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 October 2016

Check for updates

Author Tags

Qualifiers

Demonstration

Conference

UIST '16

Sponsor:

UIST '16: The 29th Annual ACM Symposium on User Interface Software and Technology

October 16 - 19, 2016

Tokyo, Japan

Acceptance Rates

UIST '16 Adjunct Paper Acceptance Rate 79 of 384 submissions, 21%;

Overall Acceptance Rate 355 of 1,733 submissions, 20%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
235
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)2

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Eady AGuerrero ACheung VThue DGirouard A(2024)BendAide: A Deformable Interface to Augment Touchscreen Mobile DevicesProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670960(1-10)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670960
Yeh YIrudayaraj AVogel D(2024)Single-handed Folding Interactions with a Modified Clamshell Flip PhoneProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642554(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642554
Yin JBai SHan YZhang XDeng SWang S(2021)The Study of Bending and Twisting Input Modalities in Deformable InterfacesElectronics10.3390/electronics1023299110:23(2991)Online publication date: 1-Dec-2021
https://doi.org/10.3390/electronics10232991
Ahmad Saffian KMohd Norowi N(2021)Gestural Interaction of Gamelan Mobile Application: A Preliminary StudyAsian CHI Symposium 202110.1145/3429360.3468199(146-148)Online publication date: 7-Sep-2021
https://doi.org/10.1145/3429360.3468199
Boem ATroiano GHarrison SBardzell SNeustaedter CTatar D(2019)Non-Rigid HCIProceedings of the 2019 on Designing Interactive Systems Conference10.1145/3322276.3322347(885-906)Online publication date: 18-Jun-2019
https://dl.acm.org/doi/10.1145/3322276.3322347
Sturdee MEveritt ALindley JCoulton PAlexander J(2019)Visual Methods for the Design of Shape-Changing InterfacesHuman-Computer Interaction – INTERACT 201910.1007/978-3-030-29387-1_19(337-358)Online publication date: 25-Aug-2019
https://doi.org/10.1007/978-3-030-29387-1_19
Cheung VEady AGirouard AFernaeus YMcMillan DJonsson MGirouard ATholander J(2018)Deformable ControllersProceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3173225.3173332(732-735)Online publication date: 18-Mar-2018
https://dl.acm.org/doi/10.1145/3173225.3173332
Sturdee MAlexander J(2018)Analysis and Classification of Shape-Changing Interfaces for Design and Application-based ResearchACM Computing Surveys10.1145/314355951:1(1-32)Online publication date: 4-Jan-2018
https://dl.acm.org/doi/10.1145/3143559

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten