skip to main content
10.1145/3411763.3451782acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
poster

FootUI: Assisting People with Upper Body Motor Impairments to Use Smartphones with Foot Gestures on the Bed

Published: 08 May 2021 Publication History

Abstract

Some people with upper body motor impairments but sound lower limbs usually use feet to interact with smartphones. However, touching the touchscreen with big toes is tiring, inefficient and easy to mistouch. In this paper, we propose FootUI, which leverages the phone camera to track users’ feet and translates the foot gestures to smartphone operations. This technique enables users to interact with smartphones while reclining on the bed and improves the comfort of users. We explore the usage scenario and foot gestures, define the mapping from foot gestures to smartphone operations and develop the prototype on smartphones, which includes the gesture tracking and recognition algorithm, and the user interface. Evaluation results show that FootUI is easy, efficient and interesting to use. Our work provides a novel input technique for people with upper body motor impairments but sound lower limbs.

Supplemental Material

MP4 File
Supplemental video
Transcript for: Supplemental video
ZIP File
Supplemental material

References

[1]
Jason Alexander, Teng Han, William Judd, Pourang Irani, and Sriram Subramanian. 2012. Putting Your Best Foot Forward: Investigating Real-World Mappings for Foot-Based Gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12). Association for Computing Machinery, New York, NY, USA, 1229–1238. https://doi.org/10.1145/2207676.2208575
[2]
Lisa Anthony, YooJin Kim, and Leah Findlater. 2013. Analyzing User-Generated Youtube Videos to Understand Touchscreen Use by People with Motor Impairments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY, USA, 1223–1232. https://doi.org/10.1145/2470654.2466158
[3]
Paul M Fitts. 1954. The information capacity of the human motor system in controlling the amplitude of movement.Journal of experimental psychology 47, 6 (1954), 381.
[4]
Jon Froehlich, Jacob O. Wobbrock, and Shaun K. Kane. 2007. Barrier Pointing: Using Physical Edges to Assist Target Acquisition on Mobile Device Touch Screens. In Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility (Tempe, Arizona, USA) (Assets ’07). Association for Computing Machinery, New York, NY, USA, 19–26. https://doi.org/10.1145/1296843.1296849
[5]
Christopher Kwan, Isaac Paquette, John J. Magee, Paul Y. Lee, and Margrit Betke. 2011. Click Control: Improving Mouse Interaction for People with Motor Impairments. In The Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility (Dundee, Scotland, UK) (ASSETS ’11). Association for Computing Machinery, New York, NY, USA, 231–232. https://doi.org/10.1145/2049536.2049582
[6]
Katsumi Minakata, John Paulin Hansen, I. Scott MacKenzie, Per Bækgaard, and Vijay Rajanna. 2019. Pointing by Gaze, Head, and Foot in a Head-Mounted Display. In Proceedings of the 11th ACM Symposium on Eye Tracking Research & amp; Applications (Denver, Colorado) (ETRA ’19). Association for Computing Machinery, New York, NY, USA, Article 69, 9 pages. https://doi.org/10.1145/3317956.3318150
[7]
Kyle Montague, Vicki L. Hanson, and Andy Cobley. 2012. Designing for Individuals: Usable Touch-Screen Interaction through Shared User Models. In Proceedings of the 14th International ACM SIGACCESS Conference on Computers and Accessibility (Boulder, Colorado, USA) (ASSETS ’12). Association for Computing Machinery, New York, NY, USA, 151–158. https://doi.org/10.1145/2384916.2384943
[8]
Kyle Montague, Hugo Nicolau, and Vicki L. Hanson. 2014. Motor-Impaired Touchscreen Interactions in the Wild. In Proceedings of the 16th International ACM SIGACCESS Conference on Computers & amp; Accessibility (Rochester, New York, USA) (ASSETS ’14). Association for Computing Machinery, New York, NY, USA, 123–130. https://doi.org/10.1145/2661334.2661362
[9]
Martez E. Mott, Radu-Daniel Vatavu, Shaun K. Kane, and Jacob O. Wobbrock. 2016. Smart Touch: Improving Touch Accuracy for People with Motor Impairments with Template Matching. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 1934–1946. https://doi.org/10.1145/2858036.2858390
[10]
Martez E. Mott and Jacob O. Wobbrock. 2019. Cluster Touch: Improving Touch Accuracy on Smartphones for People with Motor and Situational Impairments. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3290605.3300257
[11]
Guarionex Salivia and Juan Pablo Hourcade. 2013. PointAssist: Assisting Individuals with Motor Impairments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY, USA, 1213–1222. https://doi.org/10.1145/2470654.2466157
[12]
Jeremy Scott, David Dearman, Koji Yatani, and Khai N. Truong. 2010. Sensing Foot Gestures from the Pocket. In Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology (New York, New York, USA) (UIST ’10). Association for Computing Machinery, New York, NY, USA, 199–208. https://doi.org/10.1145/1866029.1866063
[13]
Shari Trewin, Simeon Keates, and Karyn Moffatt. 2006. Developing Steady Clicks: A Method of Cursor Assistance for People with Motor Impairments. In Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility (Portland, Oregon, USA) (Assets ’06). Association for Computing Machinery, New York, NY, USA, 26–33. https://doi.org/10.1145/1168987.1168993
[14]
Eduardo Velloso, Dominik Schmidt, Jason Alexander, Hans Gellersen, and Andreas Bulling. 2015. The Feet in Human–Computer Interaction: A Survey of Foot-Based Interaction. ACM Comput. Surv. 48, 2, Article 21 (Sept. 2015), 35 pages. https://doi.org/10.1145/2816455
[15]
Jacob Wobbrock. 2003. The Benefits of Physical Edges in Gesture-Making: Empirical Support for an Edge-Based Unistroke Alphabet. In CHI ’03 Extended Abstracts on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA) (CHI EA ’03). Association for Computing Machinery, New York, NY, USA, 942–943. https://doi.org/10.1145/765891.766083
[16]
Jacob O. Wobbrock, Brad A. Myers, and Scott E. Hudson. 2003. Exploring Edge-Based Input Techniques for Handheld Text Entry. In Proceedings of the 23rd International Conference on Distributed Computing Systems(ICDCSW ’03). IEEE Computer Society, USA, 280.
[17]
Jacob O. Wobbrock, Brad A. Myers, and John A. Kembel. 2003. EdgeWrite: A Stylus-Based Text Entry Method Designed for High Accuracy and Stability of Motion. In Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology (Vancouver, Canada) (UIST ’03). Association for Computing Machinery, New York, NY, USA, 61–70. https://doi.org/10.1145/964696.964703

Cited By

View all
  • (2024)“Can It Be Customized According to My Motor Abilities?”: Toward Designing User-Defined Head Gestures for People with DystoniaProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642378(1-11)Online publication date: 11-May-2024
  • (2023)Understanding Wheelchair Users’ Preferences for On-Body, In-Air, and On-Wheelchair GesturesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580929(1-16)Online publication date: 19-Apr-2023
  • (2023)How to Achieve Human–Machine Interaction by Foot Gesture Recognition: A ReviewIEEE Sensors Journal10.1109/JSEN.2023.328521423:15(16515-16528)Online publication date: 1-Aug-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems
May 2021
2965 pages
ISBN:9781450380959
DOI:10.1145/3411763
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 May 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. accessibility
  2. foot-based interaction
  3. smartphone
  4. upper body motor impairments

Qualifiers

  • Poster
  • Research
  • Refereed limited

Funding Sources

  • Guo Qiang, Tsinghua University
  • the National Key Research and Development Plan
  • Key Laboratory of Pervasive Computing, Ministry of Education

Conference

CHI '21
Sponsor:

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)60
  • Downloads (Last 6 weeks)2
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)“Can It Be Customized According to My Motor Abilities?”: Toward Designing User-Defined Head Gestures for People with DystoniaProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642378(1-11)Online publication date: 11-May-2024
  • (2023)Understanding Wheelchair Users’ Preferences for On-Body, In-Air, and On-Wheelchair GesturesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580929(1-16)Online publication date: 19-Apr-2023
  • (2023)How to Achieve Human–Machine Interaction by Foot Gesture Recognition: A ReviewIEEE Sensors Journal10.1109/JSEN.2023.328521423:15(16515-16528)Online publication date: 1-Aug-2023
  • (2023)Gesture-Based InteractionHandbook of Human Computer Interaction10.1007/978-3-319-27648-9_20-1(1-47)Online publication date: 9-Feb-2023
  • (2022)Interactive Public Displays and Wheelchair Users: Between Direct, Personal and Indirect, Assisted InteractionProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545662(1-17)Online publication date: 29-Oct-2022
  • (2022)FootUI: Designing and Detecting Foot Gestures to Assist People with Upper Body Motor Impairments to Use Smartphones on the BedProceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3517428.3563285(1-11)Online publication date: 23-Oct-2022
  • (2022)WearSkillProceedings of the 19th International Web for All Conference10.1145/3493612.3520455(1-5)Online publication date: 25-Apr-2022
  • (2022)Understanding Gesture Input Articulation with Upper-Body Wearables for Users with Upper-Body Motor ImpairmentsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501964(1-16)Online publication date: 29-Apr-2022
  • (2022)“Sometimes I feel that I’m being left behind”: Exploring Computing Device Use by People with Upper Extremity Impairment During the COVID-19 PandemicExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491101.3519647(1-9)Online publication date: 27-Apr-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media