Jingun Jung
Geehyuk Lee
ABSTRACT
Users with visual impairments find it difficult to enjoy real-time 2D interactive applications on the touchscreen. Touchscreen applications such as sports games often require simultaneous recognition of and interaction with multiple moving targets through vision. To mitigate this issue, we propose ThroughHand, a novel tactile interaction that enables users with visual impairments to interact with multiple dynamic objects in real time. We designed the ThroughHand interaction to utilize the potential of the human tactile sense that spatially registers both sides of the hand with respect to each other. ThroughHand allows interaction with multiple objects by enabling users to perceive the objects using the palm while providing a touch input space on the back of the same hand. A user study verified that ThroughHand enables users to locate stimuli on the palm with a margin of error of approximately 13 mm and effectively provides a real-time 2D interaction experience for users with visual impairments.
Supplemental Material
Available for Download
- Dragan Ahmetovic, Cristian Bernareggi, Sergio Mascetti, and Federico Pini. 2020. SoundLines: Exploration of Line Segments through Sonification and Multi-touch Interaction. In The 22nd International ACM SIGACCESS Conference on Computers and Accessibility. ACM, Virtual Event, Greece, 1–3.Google Scholar
Digital Library
- Paul Bach-y Rita and Stephen W Kercel. 2003. Sensory substitution and the human–machine interface. Trends in cognitive sciences 7, 12 (2003), 541–546.Google Scholar
- Jonathan Bell, Stanley Bolanowski, and Mark H Holmes. 1994. The structure and function of Pacinian corpuscles: a review. Progress in neurobiology 42, 1 (1994), 79–128.Google ScholarCross Ref
- Nadia Bianchi-Berthouze, Whan Woong Kim, and Darshak Patel. 2007. Does body movement engage you more in digital game play? and why?. In International conference on affective computing and intelligent interaction. Springer, Springer, Lisbon, Portugal, 102–113.Google ScholarDigital Library
- Elyse DZ Chase, Alexa Fay Siu, Abena Boadi-Agyemang, Gene SH Kim, Eric J Gonzalez, and Sean Follmer. 2020. PantoGuide: A Haptic and Audio Guidance System To Support Tactile Graphics Exploration. In The 22nd International ACM SIGACCESS Conference on Computers and Accessibility. ACM, Virtual Event, Greece, 1–4.Google Scholar
- Maurizio De Pascale, Sara Mulatto, and Domenico Prattichizzo. 2008. Bringing haptics to second life for visually impaired people. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications. Springer, Springer, Madrid, Spain, 896–905.Google ScholarDigital Library
- Paul M Fitts. 1954. The information capacity of the human motor system in controlling the amplitude of movement.Journal of experimental psychology 47, 6 (1954), 381.Google Scholar
- Sean Follmer, Daniel Leithinger, Alex Olwal, Akimitsu Hogge, and Hiroshi Ishii. 2013. inFORM: dynamic physical affordances and constraints through shape and object actuation.. In Uist. ACM, St. Andrews, United Kingdom, 2501988–2502032.Google Scholar
- Clifton Forlines, Daniel Wigdor, Chia Shen, and Ravin Balakrishnan. 2007. Direct-touch vs. mouse input for tabletop displays. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, San Jose, California, USA, 647–656.Google ScholarDigital Library
- G M A Games. 2020. Shades of Doom. G M A Games. http://www.gmagames.com/sod.htmlGoogle Scholar
- Dan Gärdenfors. 2003. Designing sound-based computer games. Digital Creativity 14, 2 (2003), 111–114.Google ScholarCross Ref
- Matti Gröhn, Tapio Lokki, and Tapio Takala. 2005. Comparison of auditory, visual, and audiovisual navigation in a 3D space. ACM Transactions on Applied Perception (TAP) 2, 4 (2005), 564–570.Google ScholarDigital Library
- Darren Guinness, Annika Muehlbradt, Daniel Szafir, and Shaun K Kane. 2019. RoboGraphics: Using Mobile Robots to Create Dynamic Tactile Graphics. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility. ACM, Pittsburgh, PA, USA, 673–675.Google ScholarDigital Library
- HyperBraille. 2007. The graphics-enabled display for blind computer users. HyperBraille. http://www.hyperbraille.de/Google Scholar
- Apple Inc.2020. Apple Accessibility. Apple Inc. https://www.apple.com/accessibility/Google Scholar
- Google Inc.2009. TalkBack: An Open Source Screenreader For Android. Google Inc. https://opensource.googleblog.com/2009/10/talkback-open-source-screenreader-for.htmlGoogle Scholar
- Ali Israr and Ivan Poupyrev. 2011. Tactile brush: drawing on skin with a tactile grid display. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, Vancouver, BC, Canada, 2019–2028.Google ScholarDigital Library
- Roland S Johansson and Åke B Vallbo. 1983. Tactile sensory coding in the glabrous skin of the human hand. Trends in neurosciences 6 (1983), 27–32.Google Scholar
- Lynette A Jones and Susan J Lederman. 2006. Human hand function. Oxford University Press, Oxford, England.Google Scholar
- Hiroyuki Kajimoto, Yonezo Kanno, and Susumu Tachi. 2006. A Vision Substitution System Using Forehead Electrical Stimulation. In ACM SIGGRAPH 2006 Sketches (Boston, Massachusetts) (SIGGRAPH ’06). Association for Computing Machinery, New York, NY, USA, 39–es. https://doi.org/10.1145/1179849.1179898Google ScholarDigital Library
- Hiroyuki Kajimoto, Masaki Suzuki, and Yonezo Kanno. 2014. HamsaTouch: feel the world through your palm. In CHI’14 Extended Abstracts on Human Factors in Computing Systems. ACM, Toronto, ON, Canada, 571–574.Google Scholar
- Tapio Lokki and Matti Grohn. 2005. Navigation with auditory cues in a virtual environment. IEEE MultiMedia 12, 2 (2005), 80–86.Google ScholarDigital Library
- I Scott MacKenzie. 1992. Fitts’ law as a research and design tool in human-computer interaction. Human-computer interaction 7, 1 (1992), 91–139.Google ScholarDigital Library
- Daniel Miller, Aaron Parecki, and Sarah A Douglas. 2007. Finger dance: a sound game for blind people. In Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility. ACM, Tempe, Arizona, USA, 253–254.Google ScholarDigital Library
- Tony Morelli, John Foley, and Eelke Folmer. 2010. Vi-bowling: a tactile spatial exergame for individuals with visual impairments. In Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility. ACM, Orlando, FL, USA, 179–186.Google ScholarDigital Library
- Ken Nakagaki, Daniel Fitzgerald, Zhiyao (John) Ma, Luke Vink, Daniel Levine, and Hiroshi Ishii. 2019. InFORCE: Bi-Directional ‘Force’ Shape Display for Haptic Interaction. In Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction (Tempe, Arizona, USA) (TEI ’19). Association for Computing Machinery, New York, NY, USA, 615–623. https://doi.org/10.1145/3294109.3295621Google ScholarDigital Library
- Uran Oh and Leah Findlater. 2015. A performance comparison of on-hand versus on-phone nonvisual input by blind and sighted users. ACM Transactions on Accessible Computing (TACCESS) 7, 4 (2015), 1–20.Google ScholarDigital Library
- Michael A Oren. 2007. Speed sonic across the span: building a platform audio game. In CHI’07 Extended Abstracts on Human Factors in Computing Systems. ACM, San Jose, California, USA, 2231–2236.Google Scholar
- David Parker and Dianne Pawluk. 2019. Exploration of Multi-fingered Access to 2D Spatial Information. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility. ACM, Pittsburgh, PA, USA, 618–620.Google ScholarDigital Library
- Andrzej Radecki, Michał Bujacz, Piotr Skulimowski, and Paweł Strumiłło. 2020. Interactive sonification of images in serious games as an education aid for visually impaired children. British Journal of Educational Technology 51, 2 (2020), 473–497.Google ScholarCross Ref
- Oliver Schneider, Jotaro Shigeyama, Robert Kovacs, Thijs Jan Roumen, Sebastian Marwecki, Nico Boeckhoff, Daniel Amadeus Gloeckner, Jonas Bounama, and Patrick Baudisch. 2018. DualPanto: A Haptic Device that Enables Blind Users to Continuously Interact with Virtual Worlds. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. ACM, Berlin, Germany, 877–887.Google ScholarDigital Library
- Oliver S Schneider, Ali Israr, and Karon E MacLean. 2015. Tactile animation by direct manipulation of grid displays. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology. ACM, Charlotte, NC, USA, 21–30.Google ScholarDigital Library
- Katherine O Sofia and Lynette Jones. 2013. Mechanical and psychophysical studies of surface wave propagation during vibrotactile stimulation. IEEE transactions on haptics 6, 3 (2013), 320–329.Google ScholarDigital Library
- AB Vallbo and RS Johansson. 1978. The tactile sensory innervation of the glabrous skin of the human hand. Active touch 2954(1978), 29–54.Google Scholar
- Angel Walia, Prakhar Goel, Varnika Kairon, and Mayank Jain. 2020. HapTech: Exploring Haptics in Gaming for the Visually Impaired. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, Honolulu, HI, USA, 1–6.Google Scholar
- Tetsuya Watanabe, Makoto Kobayashi, Shoichiro Ono, and Keiko Yokoyama. 2006. Practical use of interactive tactile graphic display system at a school for the blind. In Proc. Fourth International Conference on Multimedia and Information and Communication Technologies in Education (m-ICTE). Citeseer, FORMATEX, Seville, Spain, 1111–1115.Google Scholar
- Sidney Weinstein. 1968. Intensive and extensive aspects of tactile sensitivity as a function of body part, sex and laterality. The skin senses 1, 1 (1968), 195–218.Google Scholar
- Gareth R White, Geraldine Fitzpatrick, and Graham McAllister. 2008. Toward accessible 3D virtual environments for the blind and visually impaired. In Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts. ACM, Athens, Greece, 134–141.Google ScholarDigital Library
- Koji Yatani, Nikola Banovic, and Khai Truong. 2012. SpaceSense: representing geographical information to visually impaired people using spatial tactile feedback. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, Austin, TX, USA, 415–424.Google ScholarDigital Library
- Bei Yuan and Eelke Folmer. 2008. Blind hero: enabling guitar hero for the visually impaired. In Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility. ACM, Halifax, Nova Scotia, Canada, 169–176.Google ScholarDigital Library
- Bei Yuan, Eelke Folmer, and Frederick C Harris. 2011. Game accessibility: a survey. Universal Access in the information Society 10, 1 (2011), 81–100.Google Scholar
- Juan Jose Zarate, Olexandr Gudozhnik, Anthony Sébastien Ruch, and Herbert Shea. 2017. Keep in touch: portable haptic display with 192 high speed taxels. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. Association for Computing Machinery, Denver, CO, USA, 349–352.Google ScholarDigital Library
Index Terms
- ThroughHand: 2D Tactile Interaction to Simultaneously Recognize and Touch Multiple Objects
Index terms have been assigned to the content through auto-classification.
Recommendations
Self-Identifying Tactile Overlays
ASSETS '18: Proceedings of the 20th International ACM SIGACCESS Conference on Computers and AccessibilityTactile overlays for touch-screen devices are an opportunity to display content for users with visual impairments. However, when users switch tactile overlays, the displayed content on the touch-screen devices still correspond to the previous overlay. ...
Smart Touch: Improving Touch Accuracy for People with Motor Impairments with Template Matching
CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing SystemsWe present two contributions toward improving the accessibility of touch screens for people with motor impairments. First, we provide an exploration of the touch behaviors of 10 people with motor impairments, e.g., we describe how touching with the back ...
Enhancing physicality in touch interaction with programmable friction
CHI '11: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsTouch interactions have refreshed some of the 'glowing enthusiasm' of thirty years ago for direct manipulation interfaces. However, today's touch technologies, whose interactions are supported by graphics, sounds or crude clicks, have a tactile sameness ...
Comments