research-article

BendAide: A Deformable Interface to Augment Touchscreen Mobile Devices

Authors:

Alexander Keith Eady,

Alfrancis Guerrero,

Audrey GirouardAuthors Info & Claims

GI '24: Proceedings of the 50th Graphics Interface Conference

Article No.: 2, Pages 1 - 10

https://doi.org/10.1145/3670947.3670960

Published: 21 September 2024 Publication History

Abstract

The uses of handheld mobile devices are diverse, yet interaction is not; touchscreens are the singular primary interface on most mobiles. Touch interaction has usability issues (e.g., the “fat fingers problem”) which impair the fine control of small interface elements, such as when working with text. Beyond text entry, this includes tasks like placing the in-text cursor (caret), text selection, and copy/paste. Current solutions for touch usability issues do not address complex uses like working with text. We propose deformable interaction, specifically bend, added alongside touch to support working with text on mobile. We explore this through a study of BendAide, a novel deformable 3D printed case for mobiles that adds bend interaction to the device. We found that people perceive different advantages between bend and touch and that they will alternate between these inputs based on task demands and their personal abilities. Adding alternate input options to mobile could reduce the complexity of on-display interfaces and interactions and give people more choice in how they use their devices.

References

[1]

Teemu T Ahmaniemi, Johan Kildal, and Merja Haveri. 2014. What is a device bend gesture really good for? In SIGCHI Conference on Human Factors in Computing Systems, ACM, 3503–3512.

Digital Library

[2]

Jennifer M. Allen, Leslie A. McFarlin, and Thomas Green. 2008. An In-Depth Look into the Text Entry User Experience on the iPhone. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 52, 5 (September 2008), 508–512.

[3]

Ahmed Sabbir Arif, Sunjun Kim, Wolfgang Stuerzlinger, Geehyuk Lee, and Ali Mazalek. 2016. Evaluation of a Smart-Restorable Backspace Technique to Facilitate Text Entry Error Correction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI ’16, ACM Press, New York, New York, USA, 5151–5162.

Digital Library

[4]

Ahmed Sabbir Arif, Wolfgang Stuerzlinger, Ali Mazalek, Sunjun Kim, and Geehyuk Lee. 2015. A smart-restorable backspace to facilitate text entry error correction. In CHI 2015 Workshop on Text Entry on the Edge, Seoul, South Korea.

[5]

Oscar Kin-Chung Au, Xiaojun Su, and Rynson W.H. Lau. 2014. LinearDragger: a Linear Selector for One-finger Target Acquisition. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI ’14, ACM Press, New York, New York, USA, 2607–2616.

Digital Library

[6]

Daniel Avrahami. 2015. The Effect of Edge Targets on Touch Performance. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI ’15, ACM Press, New York, New York, USA, 1837–1846.

Digital Library

[7]

Shiri Azenkot and Shumin Zhai. 2012. Touch behavior with different postures on soft smartphone keyboards. In Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services - MobileHCI ’12, ACM Press, New York, New York, USA, 251.

Digital Library

[8]

Joanna Bergstrom-Lehtovirta and Antti Oulasvirta. 2014. Modeling the functional area of the thumb on mobile touchscreen surfaces. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI ’14, ACM Press, New York, New York, USA, 1991–2000.

Digital Library

[9]

Xiaojun Bi, Yang Li, and Shumin Zhai. 2013. FFitts law. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 1363–1372.

Digital Library

[10]

Jesse Burstyn, Amartya Banerjee, and Roel Vertegaal. 2013. FlexView: an evaluation of depth navigation on deformable mobile devices. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction - TEI ’13, ACM Press, New York, New York, USA, 193.

Digital Library

[11]

Luis Cavazos Quero, Garam Lee, Jing Yang, and Jundong Cho. 2017. H-Slate. In Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing - CSCW ’17 Companion, ACM Press, New York, New York, USA, 159–162.

Digital Library

[12]

Tom Djajadiningrat, Kees Overbeeke, and Stephan Wensveen. 2002. But how, Donald, tell us how?: on the creation of meaning in interaction design through feedforward and inherent feedback. In Proceedings of the conference on Designing interactive systems processes, practices, methods, and techniques - DIS ’02, New York, New York, USA, 285.

Digital Library

[13]

Trinh-Minh-Tri Do and Daniel Gatica-Perez. 2010. By their apps you shall understand them. Proceedings of the 9th International Conference on Mobile and Ubiquitous Multimedia - MUM ’10 (2010), 1–10.

Digital Library

[14]

Alexander Keith Eady. 2019. Deformable interactions to improve the usability of handheld mobile devices. Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019 (October 2019).

Digital Library

[15]

Alexander Keith Eady and Audrey Girouard. 2015. Caret Manipulation using Deformable Input in Mobile Devices. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction, ACM, New York, NY, USA, 587–591.

Digital Library

[16]

Rachel Eardley, Anne Roudaut, Steve Gill, and Stephen J. Thompson. 2017. Understanding Grip Shifts: How Form Factors Impact Hand Movements on Mobile Phones. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI ’17, ACM Press, New York, New York, USA, 4680–4691.

Digital Library

[17]

Matthew Ernst and Audrey Girouard. 2016. Bending Blindly: Exploring Bend Gestures for the Blind. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, ACM Press, New York, New York, USA, 2088–2096.

Digital Library

[18]

Matthew Ernst and Audrey Girouard. 2016. Exploring Haptics for Learning Bend Gestures for the Blind. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’16 (2016), 2097–2104.

Digital Library

[19]

Matthew Ernst, Travis Swan, Victor Cheung, and Audrey Girouard. 2017. Typhlex: Exploring Deformable Input for Blind Users Controlling a Mobile Screen Reader. IEEE Pervasive Comput 16, 4 (October 2017), 28–35.

[20]

Elias Fares, Victor Cheung, and Audrey Girouard. 2017. Effects of bend gesture training on learnability and memorability in a mobile game. In Interactive Surfaces and Spaces, ACM, 240–245.

Digital Library

[21]

Vittorio Fuccella and Benoît Martin. 2017. TouchTap. In Proceedings of the 12th Biannual Conference on Italian SIGCHI Chapter - CHItaly ’17, ACM Press, New York, New York, USA, 1–6.

Digital Library

[22]

Audrey Girouard and Alexander Keith Eady. 2018. Deformable User Interfaces: Using Flexible Electronics for Human Computer Interaction. In 2018 International Flexible Electronics Technology Conference (IFETC), IEEE, 1–3.

[23]

Audrey Girouard, Jessica Lo, Md Riyadh, Farshad Daliri, Alexander Keith Eady, and Jerome Pasquero. 2015. One-Handed Bend Interactions with Deformable Smartphones. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI ’15, ACM Press, New York, 1509–1518.

Digital Library

[24]

Audrey Girouard, Jessica Lo, Md Riyadh, Farshad Daliri, Alexander Keith Eady, and Jerome Pasquero. 2015. One-Handed Bend Interactions with Deformable Smartphones. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 1509–1518.

Digital Library

[25]

Antonio Gomes, Lahiru Priyadarshana, Juan Pablo Carrascal, and Roel Vertegaal. 2016. WhammyPhone: Exploring Tangible Audio Manipulation Using Bend Input on a Flexible Smartphone. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology - UIST ’16 Adjunct, ACM, New York, New York, USA, 159–161.

Digital Library

[26]

Daniel Gotsch, Xujing Zhang, Juan Pablo Carrascal, and Roel Vertegaal. 2016. HoloFlex: A Flexible Light-Field Smartphone with a Microlens Array and a P-OLED Touchscreen. Proceedings of the 29th Annual Symposium on User Interface Software and Technology - UIST ’16 (2016), 69–79.

Digital Library

[27]

David Holman, Roel Vertegaal, Mark Altosaar, Nikolaus Troje, and Derek Johns. 2005. PaperWindows: Interaction Techniques for Digital Paper. In Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ’05, Berlin, Germany, 591.

Digital Library

[28]

Steven Hoober. 2013. How Do Users Really Hold Mobile Devices? UXMatters. Retrieved September 20, 2018 from https://www.uxmatters.com/mt/archives/2013/02/how-do-users-really-hold-mobile-devices.php

[29]

Amy K Karlson and Benjamin B Bederson. 2008. One-handed Touchscreen Input for Legacy Applications. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2008), 1399–1408.

Digital Library

[30]

Kleomenis Katevas, Ioannis Arapakis, and Martin Pielot. 2018. Typical Phone Use Habits: Intense Use Does Not Predict Negative Well-Being. (2018), 1–13.

Digital Library

[31]

Andreas Komninos, Mark Dunlop, Kyriakos Katsaris, and John Garofalakis. 2018. A glimpse of mobile text entry errors and corrective behaviour in the wild. In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct - MobileHCI ’18, ACM Press, New York, New York, USA, 221–228.

Digital Library

[32]

Byron Lahey, Audrey Girouard, Winslow Burleson, and Roel Vertegaal. 2011. PaperPhone: Understanding the use of bend gestures in mobile devices with flexible electronic paper displays. In SIGCHI Conference on Human Factors in Computing Systems (CHI ’11), ACM, 1303–1312.

Digital Library

[33]

Jianwei Lai, Navid Rajabi, and Elahe Javadi. 2019. A Shortcut for Caret Positioning on Touch-Screen Phones. (2019). In Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '19). Association for Computing Machinery, New York, NY, USA, Article 35, 1–6.

Digital Library

[34]

Jianwei Lai, Navid Rajabi, and Elahe Javadi. 2019. A Shortcut for Caret Positioning on Touch-Screen Phones. In Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, ACM, New York, NY, USA, 1–6.

Digital Library

[35]

Huy Viet Le, Patrick Bader, Thomas Kosch, and Niels Henze. 2016. Investigating Screen Shifting Techniques to Improve One-Handed Smartphone Usage. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction - NordiCHI ’16, ACM Press, New York, New York, USA, 1–10.

Digital Library

[36]

Huy Viet Le, Sven Mayer, Patrick Bader, and Niels Henze. 2018. Fingers’ Range and Comfortable Area for One-Handed Smartphone Interaction Beyond the Touchscreen. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18 (2018), 1–12.

Digital Library

[37]

Huy Viet Le, Sven Mayer, Katrin Wolf, and Niels Henze. 2016. Finger Placement and Hand Grasp during Smartphone Interaction. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’16, ACM Press, New York, New York, USA, 2576–2584.

Digital Library

[38]

Sang-Su Lee, Sohyun Kim, Bipil Jin, Eunji Choi, Boa Kim, Xu Jia, Daeeop Kim, and Kun-pyo Lee. 2010. How users manipulate deformable displays as input devices. In SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, New York, USA, 1647–1656.

Digital Library

[39]

Wing Ho Andy Li and Hongbo Fu. 2013. BezelCursor: bezel-initiated cursor for one-handed target acquisition on mobile touch screens. In SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications on - SA ’13, ACM Press, New York, New York, USA, 1–1.

Digital Library

[40]

Google LLC. Gboard - the Google Keyboard. Retrieved from https://play.google.com/store/apps/details?id=com.google.android.inputmethod.latin&hl=en_CA

[41]

Jessica Lo and Audrey Girouard. 2014. Fabricating Bendy: Design and Development of Deformable Prototypes. IEEE Pervasive Computing Special Issue on Fabrication and Printing 13, 3 (2014), 40–46.

[42]

Jessica Lo and Audrey Girouard. 2017. Bendy: An exploration into gaming with mobile flexible devices. In International Conference on Tangible, Embedded, and Embodied Interaction, ACM, 163–172.

Digital Library

[43]

Fabrice Matulic and Daniel Vogel. 2018. Multiray: Multi-finger raycasting for large displays. Conference on Human Factors in Computing Systems - Proceedings 2018-April, (April 2018).

Digital Library

[44]

Steven Nagels, Raf Ramakers, Kris Luyten, and Wim Deferme. 2018. Silicone Devices. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18, ACM Press, New York, New York, USA, 1–13.

Digital Library

[45]

Vinh P. Nguyen, Sang Ho Yoon, Ansh Verma, and Karthik Ramani. 2014. BendID: Flexible interface for localized deformation recognition. In International Joint Conference on Pervasive and Ubiquitous Computing, ACM, 553–557.

Digital Library

[46]

Alan Poston. 2000. Human Engineering Design Data Digest. Washington, DC. Retrieved from https://www.acq.osd.mil/rd/hptb/hfetag/products/documents/HE_Design_Data_Digest.pdf

[47]

Christian Rendl, David Kim, Patrick Parzer, Sean Fanello, Martin Zirkl, Gregor Scheipl, Michael Haller, and Shahram Izadi. 2016. FlexCase: Enhancing Mobile Interaction with a Flexible Sensing and Display Cover. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 5138–5150.

Digital Library

[48]

Anne Roudaut, Stéphane Huot, and Eric Lecolinet. 2008. TapTap and MagStick. In Proceedings of the working conference on Advanced visual interfaces - AVI ’08, ACM Press, New York, New York, USA, 146.

Digital Library

[49]

Frederik Rudeck and Patrick Baudisch. 2012. Rock-Paper-Fibers: Bringing physical affordance to mobile touch devices. Conference on Human Factors in Computing Systems - Proceedings (2012), 1929–1932.

Digital Library

[50]

Carsten Schwesig, Ivan Poupyrev, and Eijiro Mori. 2004. Gummi: A Bendable Computer. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 263–270.

Digital Library

[51]

Paden Shorey and Audrey Girouard. 2017. Bendtroller: An exploration of in-game action mappings with a deformable game controller. In SIGCHI Conference on Human Factors in Computing Systems, ACM, 1447–1458.

Digital Library

[52]

Amanda L. Smith and Barbara S. Chaparro. 2015. Smartphone Text Input Method Performance, Usability, and Preference With Younger and Older Adults. Human Factors: The Journal of the Human Factors and Ergonomics Society 57, 6 (September 2015), 1015–1028.

[53]

Kenji Suzuki, Kazumasa Okabe, Ryuuki Sakamoto, and Daisuke Sakamoto. 2016. Fix and slide: Caret Navigation with Movable Background. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services - MobileHCI ’16, ACM Press, New York, New York, USA, 478–482.

Digital Library

[54]

Taichi Tajika, Tomoko Yonezawa, and Noriaki Mitsunaga. 2008. Intuitive page-turning interface of e-books on flexible e-paper based on user studies. In Proceeding of the 16th ACM international conference on Multimedia - MM ’08, New York, New York, USA, 793.

Digital Library

[55]

Marc Teyssier, Gilles Bailly, Catherine Pelachaud, Eric Lecolinet, Andrew Conn, and Anne Roudaut. 2019. Skin-On Interfaces: A Bio-Driven Approach for Artificial Skin Design to Cover Interactive Devices. (2019).

Digital Library

[56]

Giovanni Maria Troiano, Esben Warming Pedersen, and Kasper Hornbæk. 2014. User-defined gestures for elastic, deformable displays. In International Working Conference on Advanced Visual Interfaces, ACM, 1–8.

Digital Library

[57]

Matthieu B. Trudeau, Deanna S. Asakawa, Devin L. Jindrich, and Jack T. Dennerlein. 2016. Two-handed grip on a mobile phone affords greater thumb motor performance, decreased variability, and a more extended thumb posture than a one-handed grip. Appl Ergon 52, (January 2016), 24–28.

[58]

Roel Vertegaal and Ivan Poupyrev. 2008. Organic User Interfaces. Commun ACM 51, 6 (June 2008), 26–30.

Digital Library

[59]

Daniel Vogel and Patrick Baudisch. 2007. Shift: a technique for operating pen-based interfaces using touch. In Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ’07, ACM Press, New York, New York, USA, 657.

Digital Library

[60]

Junjue Wang, Kaichen Zhao, Xinyu Zhang, and Chunyi Peng. 2014. Ubiquitous keyboard for small mobile devices. In Proceedings of the 12th annual international conference on Mobile systems, applications, and services - MobiSys ’14, ACM Press, New York, New York, USA, 14–27.

Digital Library

[61]

Kristen Warren, Jessica Lo, Vaibhav Vadgama, and Audrey Girouard. 2013. Bending the rules: Bend gesture classification for flexible displays. In SIGCHI Conference on Human Factors in Computing Systems, ACM, 607–610.

Digital Library

[62]

Chihiro Watanabe, Alvaro Cassinelli, Yoshihiro Watanabe, and Masatoshi Ishikawa. 2014. Generic method for crafting deformable interfaces to physically augment smartphones. In Proceedings of the extended abstracts of the 32nd annual ACM conference on Human factors in computing systems - CHI EA ’14, ACM Press, New York, New York, USA, 1309–1314.

Digital Library

[63]

Daryl Weir, Henning Pohl, Simon Rogers, Keith Vertanen, and Per Ola Kristensson. 2014. Uncertain text entry on mobile devices. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI ’14, ACM Press, New York, New York, USA, 2307–2316.

Digital Library

[64]

Neng-Hao Yu, Da-Yuan Huang, Jia-Jyun Hsu, and Yi-Ping Hung. 2013. Rapid Selection of Hard-to-Access Targets by Thumb on Mobile Touch-Screens. Proc. MobileHCI (2013), 400.

Digital Library

[65]

Shumin Zhai and Per Ola Kristensson. 2012. The word-gesture keyboard: Reimagining Keyboard Interaction. Commun ACM 55, 9 (2012), 91.

Digital Library

Cited By

Thalhammer P(2025)Generative AI Meets Accessibility: Deformable Interfaces and Multimodal SolutionsProceedings of the Nineteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3689050.3704798(1-5)Online publication date: 4-Mar-2025
https://dl.acm.org/doi/10.1145/3689050.3704798

Index Terms

BendAide: A Deformable Interface to Augment Touchscreen Mobile Devices
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Haptic devices
      2. Touch screens
    2. Interaction paradigms
      1. Graphical user interfaces
  2. Interaction design
    1. Interaction design process and methods
      1. Interface design prototyping

Index terms have been assigned to the content through auto-classification.

Recommendations

Evaluating reading and analysis tasks on mobile devices: a case study of tilt and flick scrolling
OZCHI '09: Proceedings of the 21st Annual Conference of the Australian Computer-Human Interaction Special Interest Group: Design: Open 24/7

Flick scrolling is a natural scrolling method for mobile touch devices such as the iPhone™. It is useful not only for its performance but perhaps even more so for its ease of use and user experience. Tilt scrolling instead uses the device's tilt to ...
Deformable Interactions to Improve the Usability of Handheld Mobile Devices
MobileHCI '19: Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services

Handheld mobile device use is dominated by touch interfaces; keyboards and other physical inputs are disappearing. In parallel, there is a trend towards more sophisticated devices supporting complex use. While touch devices are usable, the user ...
Integrating a multitouch kiosk system with mobile devices and multimodal interaction
ITS '10: ACM International Conference on Interactive Tabletops and Surfaces

We present Calisto, a service-oriented information kiosk system for public places, like museums or hotel lobbies. Calisto supports collaboration between multiple users. They can connect their mobile devices to the large public terminal and share ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

GI '24: Proceedings of the 50th Graphics Interface Conference

June 2024

437 pages

ISBN:9798400718281

DOI:10.1145/3670947

Copyright © 2024 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 September 2024

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Conference

GI '24

GI '24: Graphics Interface

June 3 - 6, 2024

NS, Halifax, Canada

Acceptance Rates

Overall Acceptance Rate 206 of 508 submissions, 41%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
44
Total Downloads

Downloads (Last 12 months)44
Downloads (Last 6 weeks)15

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Thalhammer P(2025)Generative AI Meets Accessibility: Deformable Interfaces and Multimodal SolutionsProceedings of the Nineteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3689050.3704798(1-5)Online publication date: 4-Mar-2025
https://dl.acm.org/doi/10.1145/3689050.3704798

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten