skip to main content
10.1145/3229434.3229444acmconferencesArticle/Chapter ViewAbstractPublication PagesmobilehciConference Proceedingsconference-collections
research-article

Designing finger orientation input for mobile touchscreens

Published: 03 September 2018 Publication History

Abstract

A large number of today's systems use interactive touch surfaces as the main input channel. Current devices reduce the richness of touch input to two-dimensional positions on the screen. A growing body of work develops methods that enrich touch input to provide additional degrees of freedom for touch interaction. In particular, previous work proposed to use the finger's orientation as additional input. To efficiently implement new input techniques which make use of the new input dimensions, we need to understand the limitations of the input. Therefore, we conducted a study to derive the ergonomic constraints for using finger orientation as additional input in a two-handed smartphone scenario. We show that for both hands, the comfort and the non-comfort zone depend on how the user interacts with a touch surface. For two-handed smart-phone scenarios, the range is 33.3% larger than for tabletop scenarios. We further show that the phone orientation correlates with the finger orientation. Finger orientations which are harder to perform result in phone orientations where the screen does not directly face the user.

References

[1]
Michiel Bacchiani, Françoise Beaufays, Alexander Gruenstein, Pedro Moreno, Johan Schalkwyk, Trevor Strohman, and Heiga Zen. 2017. Speech Research at Google to Enable Universal Speech Interfaces. In New Era for Robust Speech Recognition: Exploiting Deep Learning, Shinji Watanabe, Marc Delcroix, Florian Metze, and John R. Hershey (Eds.). Springer International Publishing, Cham, 385--399.
[2]
Sebastian Boring, David Ledo, Xiang 'Anthony' Chen, Nicolai Marquardt, Anthony Tang, and Saul Greenberg. 2012. The Fat Thumb: Using the Thumb's Contact Size for Single-handed Mobile Interaction. In Proceedings of the 14th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI '12). ACM, New York, NY, USA, 39--48.
[3]
Thomas Clausen. 1832. Über die Function sin(φ) + 1/2<sup>2</sup>sin(2φ) + 1/3<sup>2</sup>sin(3φ)+ etc. In Journal für die reine und angewandte Mathematik. Duncker und Humblot, Berlin, Germany, 298--300.
[4]
Ashley Colley and Jonna Häkkilä. 2014. Exploring Finger Specific Touch Screen Interaction for Mobile Phone User Interfaces. In Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: The Future of Design (OzCHI '14). ACM, New York, NY, USA, 539--548.
[5]
Chi Tai Dang and Elisabeth André. 2011. Usage and Recognition of Finger Orientation for Multi-Touch Tabletop Interaction. In Human-Computer Interaction - INTERACT 2011. Springer Berlin Heidelberg, Berlin, Heidelberg, 409--426.
[6]
Alix Goguey, Géry Casiez, Daniel Vogel, and Carl Gutwin. 2018. Characterizing Finger Pitch and Roll Orientation During Atomic Touch Actions. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 589, 12 pages.
[7]
Niels Henze, Enrico Rukzio, and Susanne Boll. 2011. 100,000,000 Taps: Analysis and Improvement of Touch Performance in the Large. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '11). ACM, New York, NY, USA, 133--142.
[8]
Eve Hoggan, John Williamson, Antti Oulasvirta, Miguel Nacenta, Per Ola Kristensson, and Anu Lehtiö. 2013. Multi-touch Rotation Gestures: Performance and Ergonomics. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 3047--3050.
[9]
Christian Holz and Patrick Baudisch. 2010. The Generalized Perceived Input Point Model and How to Double Touch Accuracy by Extracting Fingerprints. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 581--590.
[10]
Christian Holz and Patrick Baudisch. 2011. Understanding Touch. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 2501--2510.
[11]
Jochen Huber, Mohamed Sheik-Nainar, and Nada Matic. 2017. Force-enabled Touch Input on the Steering Wheel: An Elicitation Study. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications Adjunct (AutomotiveUI '17). ACM, New York, NY, USA, 168--172.
[12]
Sven Kratz, Patrick Chiu, and Maribeth Back. 2013. PointPose: Finger Pose Estimation for Touch Input on Mobile Devices Using a Depth Sensor. In Proceedings of the 2013 ACM International Conference on Interactive Tabletops and Surfaces (ITS '13). ACM, New York, NY, USA, 223--230.
[13]
Thomas Kubitza, Norman Pohl, Tilman Dingler, and Albrecht Schmidt. 2013. WebClip: A Connector for Ubiquitous Physical Input and Output for Touch Screen Devices. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '13). ACM, New York, NY, USA, 387--390.
[14]
Huy Viet Le, Sven Mayer, Patrick Bader, and Niels Henze. 2018. Fingers' Range and Comfortable Area for One-Handed Smartphone Interaction Beyond the Touchscreen. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI'18). ACM, New York, NY, USA.
[15]
Huy Viet Le, Sven Mayer, Katrin Wolf, and Niels Henze. 2016. Finger Placement and Hand Grasp During Smartphone Interaction. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16). ACM, New York, NY, USA, 2576--2584.
[16]
Cecil Lozano, Devin Jindrich, and Kanav Kahol. 2011. The Impact on Musculoskeletal System During Multitouch Tablet Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 825--828.
[17]
I. Scott MacKenzie and R. William Soukoreff. 2003. Phrase Sets for Evaluating Text Entry Techniques. In CHI '03 Extended Abstracts on Human Factors in Computing Systems (CHI EA '03). ACM, New York, NY, USA, 754--755.
[18]
Nicolai Marquardt, Johannes Kiemer, David Ledo, Sebastian Boring, and Saul Greenberg. 2011. Designing User-, Hand-, and Handpart-aware Tabletop Interactions with the TouchID Toolkit. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS '11). ACM, New York, NY, USA, 21--30.
[19]
Justin Matejka, Michael Glueck, Tovi Grossman, and George Fitzmaurice. 2016. The Effect of Visual Appearance on the Performance of Continuous Sliders and Visual Analogue Scales. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 5421--5432.
[20]
Sven Mayer, Perihan Gad, Katrin Wolf, Paweł W. Woźniak, and Niels Henze. 2017a. Understanding the Ergonomic Constraints in Designing for Touch Surfaces. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '17). ACM, New York, NY, USA, Article 33, 9 pages.
[21]
Sven Mayer, Huy Viet Le, and Niels Henze. 2017b. Estimating the Finger Orientation on Capacitive Touchscreens Using Convolutional Neural Networks. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces (ISS '17). ACM, Brighton, United Kingdom, 220--229.
[22]
Sven Mayer, Lars Lischke, Adrian Lanksweirt, Huy Viet Le, and Niels Henze. 2018. How to Communicate New Input Techniques. In Proceedings of the 10th Nordic Conference on Human-Computer Interaction (NordiCHI '18). ACM, New York, NY, USA, 13.
[23]
Sven Mayer, Michael Mayer, and Niels Henze. 2017. Feasibility Analysis of Detecting the Finger Orientation with Depth Cameras. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI'17). ACM, New York, NY, USA, 82:1--82:8.
[24]
Kanishka Rao, Haşim Sak, and Rohit Prabhavalkar. 2017. Exploring architectures, data and units for streaming end-to-end speech recognition with RNN-transducer. (Dec 2017).
[25]
Simon Rogers, John Williamson, Craig Stewart, and Roderick Murray-Smith. 2011. AnglePose: Robust, Precise Capacitive Touch Tracking via 3D Orientation Estimation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 2575--2584.
[26]
Katie A Siek, Yvonne Rogers, and Kay H Connelly. 2005. Fat finger worries: how older and younger users physically interact with PDAs. In Human-Computer Interaction-INTERACT 2005, Maria Francesca Costabile and Fabio Paternò (Eds.). Lecture Notes in Computer Science, Vol. 3585. Springer Berlin Heidelberg, Berlin, Heidelberg, 267--280.
[27]
Yoshiki Takeoka, Takashi Miyaki, and Jun Rekimoto. 2010. Z-touch: An Infrastructure for 3D Gesture Interaction in the Proximity of Tabletop Surfaces. In ACM International Conference on Interactive Tabletops and Surfaces (ITS '10). ACM, New York, NY, USA, 91--94.
[28]
Horst Treiblmaier and Peter Filzmoser. 2011. Benefits from Using Continuous Rating Scales in Online Survey Research. In Proceedings of the International Conference on Information Systems (ICIS '11). Association for Information Systems, 1--15. http://aisel.aisnet.org/icis2011/proceedings/researchmethods/1
[29]
Feng Wang, Xiang Cao, Xiangshi Ren, and Pourang Irani. 2009. Detecting and Leveraging Finger Orientation for Interaction with Direct-touch Surfaces. In Proceedings of the 22Nd Annual ACM Symposium on User Interface Software and Technology (UIST '09). ACM, New York, NY, USA, 23--32.
[30]
Feng Wang and Xiangshi Ren. 2009. Empirical Evaluation for Finger Input Properties in Multi-touch Interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, New York, NY, USA, 1063--1072.
[31]
Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only Anova Procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 143--146.
[32]
Katrin Wolf, Marilyn McGee-Lennon, and Stephen Brewster. 2012. A Study of On-device Gestures. In Proceedings of the 14th International Conference on Human-computer Interaction with Mobile Devices and Services Companion (MobileHCI '12). ACM, New York, NY, USA, 11--16.
[33]
Katrin Wolf, Stefan Schneegass, Niels Henze, Dominik Weber, Valentin Schwind, Pascal Knierim, Sven Mayer, Tilman Dingler, Yomna Abdelrahman, Thomas Kubitza, Markus Funk, Anja Mebus, and Albrecht Schmidt. 2015. TUIs in the Large: Using Paper Tangibles with Mobile Devices. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '15). ACM, New York, NY, USA, 1579--1584.
[34]
Robert Xiao, Julia Schwarz, and Chris Harrison. 2015. Estimating 3D Finger Angle on Commodity Touchscreens. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces (ITS '15). ACM, New York, NY, USA, 47--50.
[35]
Vadim Zaliva. 2012. 3D finger posture detection and gesture recognition on touch surfaces. In 12th International Conference on Control Automation Robotics Vision (ICARCV 2012). IEEE, 359--364.

Cited By

View all
  • (2024)So You Want to Do ESM? 10 Essential Topics for Implementing the Experience-Sampling MethodAdvances in Methods and Practices in Psychological Science10.1177/251524592412679127:3Online publication date: 27-Sep-2024
  • (2022)ThumbPitch: Enriching Thumb Interaction on Mobile Touchscreens using Deep LearningProceedings of the 34th Australian Conference on Human-Computer Interaction10.1145/3572921.3572925(58-66)Online publication date: 29-Nov-2022
  • (2022)Estimating 3D Finger Angle via Fingerprint ImageProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35172436:1(1-22)Online publication date: 29-Mar-2022
  • Show More Cited By
  1. Designing finger orientation input for mobile touchscreens

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MobileHCI '18: Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services
    September 2018
    552 pages
    ISBN:9781450358989
    DOI:10.1145/3229434
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 03 September 2018

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. ergonomic zone
    2. ergonomics
    3. finger orientation
    4. mobile
    5. non-comfort zone
    6. pitch
    7. surface
    8. touch
    9. yaw

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    MobileHCI '18
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 202 of 906 submissions, 22%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)31
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)So You Want to Do ESM? 10 Essential Topics for Implementing the Experience-Sampling MethodAdvances in Methods and Practices in Psychological Science10.1177/251524592412679127:3Online publication date: 27-Sep-2024
    • (2022)ThumbPitch: Enriching Thumb Interaction on Mobile Touchscreens using Deep LearningProceedings of the 34th Australian Conference on Human-Computer Interaction10.1145/3572921.3572925(58-66)Online publication date: 29-Nov-2022
    • (2022)Estimating 3D Finger Angle via Fingerprint ImageProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35172436:1(1-22)Online publication date: 29-Mar-2022
    • (2022)Estimating 3D Finger Pose via 2D-3D Fingerprint MatchingProceedings of the 27th International Conference on Intelligent User Interfaces10.1145/3490099.3511123(459-469)Online publication date: 22-Mar-2022
    • (2021)A Design Space for User Interface Elements using Finger Orientation InputProceedings of Mensch und Computer 202110.1145/3473856.3473862(1-10)Online publication date: 5-Sep-2021
    • (2019)Investigating the Effect of Orientation and Visual Style on Touchscreen Slider PerformanceProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300419(1-9)Online publication date: 2-May-2019
    • (2018)Fingers and AnglesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/32870422:4(1-21)Online publication date: 27-Dec-2018
    • (2018)How to communicate new input techniquesProceedings of the 10th Nordic Conference on Human-Computer Interaction10.1145/3240167.3240176(460-472)Online publication date: 29-Sep-2018

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media