Farhani Momotaz
Syed Masum Billah
ABSTRACT
People with low vision interact with smartphones using assistive technologies like screen magnifiers, which provide built-in touch gestures to pan and zoom onscreen content. These gestures are often cumbersome and require bimanual interaction. Of particular interest is panning gestures, which are issued frequently, which involve 2- or 3-finger dragging. This paper aims to utilize tilt-based interaction as a single-handed alternative to built-in panning gestures. To that end, we first identified our design space from the literature and conducted an exploratory user study with 12 low-vision participants to understand key challenges. Among many findings, the study revealed that built-in panning gestures are error-prone, and most tilt-based interaction techniques are designed for sighted users, which low vision users struggle to use as-is. We addressed these challenges by adapting low-vision users’ interaction behavior and proposed Tilt-Explore, a new screen magnifier mode that enables tilt-to-pan. A second study with 16 low-vision participants revealed that, compared to built-in gestures, the participants were significantly less error-prone; and for lower magnification scale (e.g., <4x), they were significantly more efficient with Tilt-Explore. These findings indicate Tilt-Explore is a promising alternative to built-in panning gestures.
Supplemental Material
- 2020. AccessibilityServiceInfo – Android Developers. Retrieved October 8, 2020 from https://developer.android.com/reference/android/accessibilityservice/AccessibilityServiceInfo#attr_android:canRequestTouchExplorationModeGoogle Scholar
- 2020. Android Magnification. https://support.google.com/accessibility/android/answer/6006949?hl=enGoogle Scholar
- 2020. Magnification – Android Accessibility Help. Retrieved October 8, 2020 from https://support.google.com/accessibility/android/answer/6006949?hl=enGoogle Scholar
- 2020. Speed Reading Facts. https://secure.execuread.com/facts/.Google Scholar
- 2020. Zoom in on the iPhone screen. Retrieved October 8, 2020 from https://support.apple.com/guide/iphone/zoom-iph3e2e367e/iosGoogle Scholar
- AFB. 2020. Glossary of Eye Conditions. http://www.afb.org/info/living-with-vision-loss/eye-conditions/12#LGoogle Scholar
- B. Agarwal and W. Stuerzlinger. 2013. WidgetLens: a system for adaptive content magnification of widgets. In Proceedings of the 27th International BCS Human Computer Interaction Conference. British Computer Society, 2578052, 1–10.Google Scholar
- Android. 2018. Android Sensor Manager. https://developer.android.com/reference/android/hardware/SensorManager. (Accessed on 09/19/2018).Google Scholar
- Android. 2019. Android Sensors. https://developer.android.com/guide/topics/sensors/sensors_overview.htmlGoogle Scholar
- Apple Inc. 2020. Use AssistiveTouch on your iPhone, iPad, or iPod touch. https://support.apple.com/en-us/HT202658[Online; accessed 01-Oct-2020].Google Scholar
- Jeff Avery, Sylvain Malacria, Mathieu Nancel, Géry Casiez, and Edward Lank. 2018. Introducing Transient Gestures to Improve Pan and Zoom on Touch Surfaces. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems(CHI ’18). ACM, New York, NY, USA, Article 25, 8 pages. https://doi.org/10.1145/3173574.3173599Google ScholarDigital Library
- Ravin Balakrishnan and I. Scott MacKenzie. 1997. Performance Differences in the Fingers, Wrist, and Forearm in Computer Input Control. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems(CHI ’97). Association for Computing Machinery, New York, NY, USA, 303–310. https://doi.org/10.1145/258549.258764Google ScholarDigital Library
- Sandra Bardot, Anke Brock, Marcos Serrano, and Christophe Jouffrais. 2014. Quick-glance and In-depth Exploration of a Tabletop Map for Visually Impaired People. In Proceedings of the 26th Conference on L’Interaction Homme-Machine(IHM ’14). ACM, New York, NY, USA, 165–170. https://doi.org/10.1145/2670444.2670465Google ScholarDigital Library
- Joel F. Bartlett. 2000. Rock ’n’ Scroll Is Here to Stay. IEEE Comput. Graph. Appl. 20, 3 (2000), 40–45. https://doi.org/10.1109/38.844371Google ScholarDigital Library
- Patrick Baudisch, Edward Cutrell, Dan Robbins, Mary Czerwinski, Peter Tandler, Benjamin Bederson, and Alex Zierlinger. Year. Drag-and-pop and drag-and-pick: Techniques for accessing remote screen content on touch-and pen-operated systems. In Proceedings of INTERACT, Vol. 3. 57–64.Google Scholar
- Benjamin B. Bederson and James D. Hollan. 1994. Pad++: a zooming graphical interface for exploring alternate interface physics. In Proceedings of the 7th annual ACM symposium on User interface software and technology. ACM, 192435, 17–26. https://doi.org/10.1145/192426.192435Google Scholar
- Jeffrey P. Bigham. 2014. Making the web easier to see with opportunistic accessibility improvement. In Proceedings of the 27th annual ACM symposium on User interface software and technology. ACM, 2647357, 117–122. https://doi.org/10.1145/2642918.2647357Google ScholarDigital Library
- Syed Masum Billah, Vikas Ashok, Donald E. Porter, and I.V. Ramakrishnan. 2017. Speed-Dial: A Surrogate Mouse for Non-Visual Web Browsing. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, 3132531, 110–119. https://doi.org/10.1145/3132525.3132531Google ScholarDigital Library
- Syed Masum Billah, Vikas Ashok, Donald E. Porter, and I.V. Ramakrishnan. 2018. SteeringWheel: A Locality-Preserving Magnification Interface for Low Vision Web Browsing. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 3173594, 1–13. https://doi.org/10.1145/3173574.3173594Google ScholarDigital Library
- Renaud Blanch, Yves Guiard, and Michel Beaudouin-Lafon. 2004. Semantic pointing: improving target acquisition with control-display ratio adaptation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 985758, 519–526. https://doi.org/10.1145/985692.985758Google ScholarDigital Library
- P. Blenkhorn and D. G. Evans. 2006. A Screen Magnifier Using ”High Level” Implementation Techniques. IEEE Transactions on Neural Systems and Rehabilitation Engineering 14, 4(2006), 501–504. https://doi.org/10.1109/TNSRE.2006.886728Google ScholarCross Ref
- P. Blenkhorn, D. G. Evans, and A. Baude. 2002. Full-screen magnification for windows using DirectX overlays. IEEE Transactions on Neural Systems and Rehabilitation Engineering 10, 4(2002), 225–231. https://doi.org/10.1109/TNSRE.2002.806835Google ScholarCross Ref
- John Brooke. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189 (1996), 194.Google Scholar
- Maria Claudia Buzzi, Marina Buzzi, Barbara Leporini, and Amaury Trujillo. 2015. Exploring Visually Impaired People’s Gesture Preferences for Smartphones. In Proceedings of the 11th Biannual Conference on Italian SIGCHI Chapter. ACM, 2808448, 94–101. https://doi.org/10.1145/2808435.2808448Google ScholarDigital Library
- Youli Chang, Sehi L’Yi, Kyle Koh, and Jinwook Seo. 2015. Understanding Users’ Touch Behavior on Large Mobile Touch-Screens and Assisted Targeting by Tilting Gesture. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems(CHI ’15). ACM, New York, NY, USA, 1499–1508. https://doi.org/10.1145/2702123.2702425Google ScholarDigital Library
- Wenzhe Cui, Suwen Zhu, Zhi Li, Zheer Xu, Xing-Dong Yang, Iv Ramakrishnan, and Xiaojun Bi. 2021. BackSwipe: Back-of-device Word-Gesture Interaction on Smartphones. 1–12. https://doi.org/10.1145/3411764.3445081Google Scholar
- Julie Fraser and Carl Gutwin. 2000. A framework of assistive pointers for low vision users. In Proceedings of the fourth international ACM conference on Assistive technologies. ACM, 354329, 9–16. https://doi.org/10.1145/354324.354329Google ScholarDigital Library
- George W. Furnas and Benjamin B. Bederson. 1995. Space-Scale Diagrams: Understanding Multiscale Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’95). ACM Press/Addison-Wesley Publishing Co., USA, 234–241. https://doi.org/10.1145/223904.223934Google Scholar
- E. Grandjean and H. Oldroyd. 1980. Fitting the Task to the Man: An Ergonomic Approach. Taylor & Francis. https://books.google.com/books?id=CFtRAAAAMAAJGoogle Scholar
- Tovi Grossman and Ravin Balakrishnan. 2005. The bubble cursor: enhancing target acquisition by dynamic resizing of the cursor’s activation area. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1055012, 281–290. https://doi.org/10.1145/1054972.1055012Google ScholarDigital Library
- Yves Guiard. 2013. Asymmetric Division of Labor in Human Skilled Bimanual Action. Journal of Motor Behavior 19 (08 2013), 486–517. https://doi.org/10.1080/00222895.1987.10735426Google Scholar
- Yves Guiard, Renaud Blanch, and Michel Beaudouin-Lafon. 2004. Object pointing: a complement to bitmap pointing in GUIs. In Proceedings of Graphics Interface 2004. Canadian Human-Computer Communications Society, 1006060, 9–16.Google Scholar
- Elyse C. Hallett, Wayne Dick, Tom Jewett, and Kim-Phuong L. Vu. 2018. How Screen Magnification with and Without Word-Wrapping Affects the User Experience of Adults with Low Vision. Springer International Publishing, Cham, 665–674. https://doi.org/10.1007/978-3-319-60492-3_63Google Scholar
- Beverly L. Harrison, Kenneth P. Fishkin, Anuj Gujar, Carlos Mochon, and Roy Want. 1998. Squeeze me, hold me, tilt me! An exploration of manipulative user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM Press/Addison-Wesley Publishing Co., 274647, 17–24. https://doi.org/10.1145/274644.274647Google ScholarDigital Library
- Ken Hinckley, Jeff Pierce, Mike Sinclair, and Eric Horvitz. 2000. Sensing Techniques for Mobile Interaction. In Proceedings of the 13th Annual ACM Symposium on User Interface Software and Technology(UIST ’00). ACM, New York, NY, USA, 91–100. https://doi.org/10.1145/354401.354417Google ScholarDigital Library
- Ken Hinckley and Hyunyoung Song. 2011. Sensor Synaesthesia: Touch in Motion, and Motion in Touch. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’11). ACM, New York, NY, USA, 801–810. https://doi.org/10.1145/1978942.1979059Google ScholarDigital Library
- Juan Pablo Hourcade, Christopher M. Nguyen, Keith B. Perry, and Natalie L. Denburg. 2010. Pointassist for older adults: analyzing sub-movement characteristics to aid in pointing tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1753494, 1115–1124. https://doi.org/10.1145/1753326.1753494Google ScholarDigital Library
- https://developer.apple.com/. 2020. Adaptivity and Layout - Visual Design - iOS - Human Interface Guidelines - Apple Developer. https://developer.apple.com/design/human-interface-guidelines/ios/visual-design/adaptivity-and-layout/.Google Scholar
- Apple Inc.2021. Apple Previews Powerful Software Updates Designed for People with Disabilities. https://www.apple.com/newsroom/2021/05/apple-previews-powerful-software-updates-designed-for-people-with-disabilities/Google Scholar
- Paul Kabbash and William A. S. Buxton. 1995. The ”prince” technique: Fitts’ law and selection using area cursors. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM Press/Addison-Wesley Publishing Co., 223939, 273–279. https://doi.org/10.1145/223904.223939Google ScholarDigital Library
- Shaun K. Kane, Jeffrey P. Bigham, and Jacob O. Wobbrock. 2008. Slide rule: making mobile touch screens accessible to blind people using multi-touch interaction techniques. In Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility. ACM, 1414487, 73–80. https://doi.org/10.1145/1414471.1414487Google ScholarDigital Library
- Shaun K. Kane, Jacob O. Wobbrock, and Richard E. Ladner. 2011. Usable gestures for blind people: understanding preference and performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1979001, 413–422. https://doi.org/10.1145/1978942.1979001Google ScholarDigital Library
- Richard L. Kline and Ephraim P. Glinert. 1995. Improving GUI accessibility for people with low vision. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM Press/Addison-Wesley Publishing Co., 223919, 114–121. https://doi.org/10.1145/223904.223919Google Scholar
- I Scott MacKenzie. 1992. Fitts’ law as a research and design tool in human-computer interaction. Human-computer interaction 7, 1 (1992), 91–139.Google ScholarDigital Library
- I. Scott MacKenzie and Robert J. Teather. 2012. FittsTilt: The Application of Fitts’ Law to Tilt-Based Interaction. In Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design(NordiCHI ’12). Association for Computing Machinery, New York, NY, USA, 568–577. https://doi.org/10.1145/2399016.2399103Google Scholar
- Material Design. 2020. Understanding layout - Material Design. https://material.io/design/layout/understanding-layout.html.Google Scholar
- Stephania Medryk and I Scott MacKenzie. 2013. A comparison of accelerometer and touch-based input for mobile gaming. In International Conference on Multimedia and Human-Computer Interaction-MHCI 2013. 117–1.Google Scholar
- Alexander Ng, Stephen A. Brewster, and John H. Williamson. 2014. Investigating the Effects of Encumbrance on One- and Two- Handed Interactions with Mobile Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’14). ACM, New York, NY, USA, 1981–1990. https://doi.org/10.1145/2556288.2557312Google ScholarDigital Library
- I. Oakley and S. O’Modhrain. 2005. Tilt to scroll: evaluating a motion based vibrotactile mobile interface. In First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference. 40–49. https://doi.org/10.1109/WHC.2005.138Google ScholarDigital Library
- Kurt Partridge, Saurav Chatterjee, Vibha Sazawal, Gaetano Borriello, and Roy Want. 2002. TiltType: Accelerometer-Supported Text Entry for Very Small Devices. UIST (User Interface Software and Technology): Proceedings of the ACM Symposium, 201–204. https://doi.org/10.1145/571985.572013Google Scholar
- Martin Pielot, Benjamin Poppinga, Wilko Heuten, and Susanne Boll. 2011. A tactile compass for eyes-free pedestrian navigation. In Proceedings of the 13th IFIP TC 13 international conference on Human-computer interaction - Volume Part II. Springer-Verlag, 2042179, 640–656.Google ScholarDigital Library
- Mahfuz Rahman, Sean Gustafson, Pourang Irani, and Sriram Subramanian. 2009. Tilt Techniques: Investigating the Dexterity of Wrist-based Input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’09). ACM, New York, NY, USA, 1943–1952. https://doi.org/10.1145/1518701.1518997Google ScholarDigital Library
- Jun Rekimoto. 1996. Tilting operations for small screen interfaces. In Proceedings of the 9th annual ACM symposium on User interface software and technology. ACM, 237115, 167–168. https://doi.org/10.1145/237091.237115Google ScholarDigital Library
- RotoView. [n. d.]. Smart Scroll Technology for Smartphones and Other Handheld Devices. http://rotoview.com/index.htmGoogle Scholar
- Jaime Ruiz and Yang Li. 2011. DoubleFlip: a motion gesture delimiter for mobile interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1979341, 2717–2720. https://doi.org/10.1145/1978942.1979341Google ScholarDigital Library
- Jaime Ruiz, Yang Li, and Edward Lank. 2011. User-defined motion gestures for mobile interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1978971, 197–206. https://doi.org/10.1145/1978942.1978971Google ScholarDigital Library
- Sonja Rümelin, Enrico Rukzio, and Robert Hardy. 2011. NaviRadar: a novel tactile information display for pedestrian navigation. In Proceedings of the 24th annual ACM symposium on User interface software and technology. ACM, 2047234, 293–302. https://doi.org/10.1145/2047196.2047234Google ScholarDigital Library
- Samsung. [n. d.]. How do I use Motion and Gestures to control my Samsung Galaxy S4?https://www.samsung.com/us/support/answer/ANS00043991/Google Scholar
- David Small and Hiroshi Ishii. 1997. Design of spatially aware graspable displays. In CHI ’97 Extended Abstracts on Human Factors in Computing Systems. ACM, 1120437, 367–368. https://doi.org/10.1145/1120212.1120437Google Scholar
- Lee Stearns, Victor DeSouza, Jessica Yin, Leah Findlater, and Jon E. Froehlich. 2017. Augmented Reality Magnification for Low Vision Users with the Microsoft Hololens and a Finger-Worn Camera. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility(ASSETS ’17). Association for Computing Machinery, New York, NY, USA, 361–362. https://doi.org/10.1145/3132525.3134812Google ScholarDigital Library
- Sarit Felicia Anais Szpiro, Shafeka Hashash, Yuhang Zhao, and Shiri Azenkot. 2016. How People with Low Vision Access Computing Devices: Understanding Challenges and Opportunities. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, 2982168, 171–180. https://doi.org/10.1145/2982142.2982168Google Scholar
- Robert J. Teather and I. Scott MacKenzie. 2014. Position vs. Velocity Control for Tilt-based Interaction. In Proceedings of Graphics Interface 2014(GI ’14). Canadian Information Processing Society, Toronto, Ont., Canada, Canada, 51–58. http://dl.acm.org/citation.cfm?id=2619648.2619658Google ScholarDigital Library
- Robert J. Teather, Andrew Roth, and I. Scott MacKenzie. 2017. Tilt-Touch synergy: Input control for “dual-analog” style mobile games. Entertainment Computing 21 (2017), 33–43. https://doi.org/10.1016/j.entcom.2017.04.005Google ScholarCross Ref
- Mary Frances Theofanos and Janice Redish. 2005. Helping Low-vision and Other Users with Web Sites That Meet Their Needs: Is One Site for All Feasible?Technical Communication 52, 1 (2005), 9–20. http://www.ingentaconnect.com/content/stc/tc/2005/00000052/00000001/art00002Google Scholar
- Theophanis Tsandilas, Caroline Appert, Anastasia Bezerianos, and David Bonnet. 2014. Coordination of Tilt and Touch in One- and Two-handed Use. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems(CHI ’14). ACM, New York, NY, USA, 2001–2004. https://doi.org/10.1145/2556288.2557088Google ScholarDigital Library
- Gregg C. Vanderheiden. 1996. Use of Audio-Haptic Interface Techniques to Allow Nonvisual Access to Touchscreen Appliances. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 40, 24 (1996), 1266–1266. https://doi.org/10.1177/154193129604002430Google ScholarCross Ref
- Lars Weberg, Torbjörn Brange, and Åsa Wendelbo Hansson. 2001. A Piece of Butter on the PDA Display. In CHI ’01 Extended Abstracts on Human Factors in Computing Systems(CHI EA ’01). Association for Computing Machinery, New York, NY, USA, 435–436. https://doi.org/10.1145/634067.634320Google Scholar
- Daniel Wigdor and Ravin Balakrishnan. 2003. TiltText: Using Tilt for Text Input to Mobile Phones. In Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology(UIST ’03). ACM, New York, NY, USA, 81–90. https://doi.org/10.1145/964696.964705Google ScholarDigital Library
- Won Kim, F. Tendick, S. Ellis, and L. Stark. 1987. A comparison of position and rate control for telemanipulations with consideration of manipulator system dynamics. IEEE Journal on Robotics and Automation 3, 5 (Oct. 1987), 426–436. https://doi.org/10.1109/JRA.1987.1087117Google Scholar
- Aileen Worden, Nef Walker, Krishna Bharat, and Scott Hudson. 1997. Making computers easier for older adults to use: area cursors and sticky icons. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems. ACM, 258724, 266–271. https://doi.org/10.1145/258549.258724Google ScholarDigital Library
- Hui-Shyong Yeo, Xiao-Shen Phang, Steven J. Castellucci, Per Ola Kristensson, and Aaron Quigley. 2017. Investigating Tilt-based Gesture Keyboard Entry for Single-Handed Text Entry on Large Devices. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 3025520, 4194–4202. https://doi.org/10.1145/3025453.3025520Google ScholarDigital Library
- Yuhang Zhao, Sarit Szpiro, Lei Shi, and Shiri Azenkot. 2019. Designing and Evaluating a Customizable Head-Mounted Vision Enhancement System for People with Low Vision. ACM Trans. Access. Comput. 12, 4, Article 15 (Dec. 2019), 46 pages. https://doi.org/10.1145/3361866Google ScholarDigital Library
- Zhengxuan Zhao, Pei-Luen Patrick Rau, Ting Zhang, and Gavriel Salvendy. 2009. Visual search-based design and evaluation of screen magnifiers for older and visually impaired users. Int. J. Hum.-Comput. Stud. 67, 8 (2009), 663–675. https://doi.org/10.1016/j.ijhcs.2009.03.006Google ScholarDigital Library
Recommendations
Usable gestures for blind people: understanding preference and performance
CHI '11: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsDespite growing awareness of the accessibility issues surrounding touch screen use by blind people, designers still face challenges when creating accessible touch screen interfaces. One major stumbling block is a lack of understanding about how blind ...
Tilt displays: designing display surfaces with multi-axis tilting and actuation
MobileHCI '12: Proceedings of the 14th international conference on Human-computer interaction with mobile devices and servicesWe present a new type of actuatable display, called Tilt Displays, that provide visual feedback combined with multi-axis tilting and vertical actuation. Their ability to physically mutate provides users with an additional information channel that ...
Press & tilt: one-handed text selection and command execution on smartphone
OzCHI '18: Proceedings of the 30th Australian Conference on Computer-Human InteractionWe show a text selection and text command execution method for a smartphone by tilting called Press & Tilt. The user can perform caret navigation or text selection by tilting the smartphone while pressing a key of the software keyboard. Then, by ...
Comments