skip to main content
10.1145/3341163.3347715acmconferencesArticle/Chapter ViewAbstractPublication PagesubicompConference Proceedingsconference-collections
research-article

Boosting word recognition for vibrotactile skin reading

Published: 09 September 2019 Publication History

Abstract

Proficiency in any form of reading requires a considerable amount of practice. With exposure, people get better at recognising words, because they develop strategies that enable them to read faster. This paper describes a study investigating recognition of words encoded with a 6-channel vibrotactile display. We train 22 users to recognise ten letters of the English alphabet. Additionally, we repeatedly expose users to 12 words in the form of training and reinforcement testing. Then, we test participants on exposed and unexposed words to observe the effects of exposure to words. Our study shows that, with exposure to words, participants did significantly improve on recognition of exposed words. The findings suggest that such a word exposure technique could be used during the training of novice users in order to boost the word recognition of a particular dictionary of words.

References

[1]
J. C. Bliss, M. H. Katcher, C. H. Rogers, and R. P. Shepard. 1970. Optical-to-Tactile Image Conversion for the Blind. IEEE Transactions on Man-Machine Systems 11, 1 (March 1970), 58--65.
[2]
Łukasz Bola, Katarzyna Siuda-Krzywicka, Małgorzata Paplińska, Ewa Sumera, Paweł Hańczur, and Marcin Szwed Szwed. 2016. Braille in the sighted: Teaching tactile reading to sighted adults. PLoS ONE (2016).
[3]
Jessica R. Cauchard, Janette L. Cheng, Thomas Pietrzak, and James A. Landay. 2016. ActiVibe: Design and Evaluation of Vibrations for Progress Monitoring. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 3261--3271.
[4]
Jennifer Chen, Robert Turcott, Pablo Castillo, Wahyudinata Setiawan, Frances Lau, and Ali Israr. 2018. Learning to Feel Words: A Comparison of Learning Approaches to Acquire Haptic Words. In Proceedings of the 15th ACM Symposium on Applied Perception (SAP '18). ACM, New York, NY, USA, Article 11, 7 pages.
[5]
Roger W. Cholewiak and Amy A. Collins. {n. d.}. The generation of vibrotactile patterns on a linear array: Influences of body site, time, and presentation mode. Perception & Psychophysics 62, 6 ({n. d.}), 1220--1235.
[6]
Roger W. Cholewiak and Amy A. Collins. {n. d.}. Vibrotactile pattern discrimination and communality at several body sites. Perception & Psychophysics 57, 5 ({n. d.}), 724--737.
[7]
Roger W. Cholewiak and James C. Craig. {n. d.}. Vibrotactile pattern recognition and discrimination at several body sites. Perception & Psychophysics 35, 6 ({n. d.}), 503--514.
[8]
Meredyth Daneman. 1988. How reading braille is both like and unlike reading print. Memory & Cognition 16, 6 (01 Nov 1988), 497--504.
[9]
de Jesus Oliveira and Anderson Maciel. 2014. Introducing the Modifier Tactile Pattern for Vibrotactile Communication. In EUROHAPTICS, 2014.
[10]
Robert O. Duncan and Geoffrey M. Boynton. 2007. Tactile Hyperacuity Thresholds Correlate with Finger Maps in Primary Somatosensory Cortex (S1). Cerebral Cortex 17, 12 (2007), 2878.
[11]
Nathan Dunkelberger, Jenny Sullivan, Joshua Bradley, Nickolas P Walling, Indu Manickam, Gautam Dasarathy, Ali Israr, Frances W. Y. Lau, Keith Klumb, Brian Knott, Freddy Abnousi, Richard Baraniuk, and Marcia K O'Malley. 2018. Conveying Language Through Haptics: A Multi-sensory Approach. In Proceedings of the 2018 ACM International Symposium on Wearable Computers (ISWC '18). ACM, 25--32.
[12]
Dennis F. Fisher. 1975. Reading and visual search. Memory & Cognition 3, 2 (01 Mar 1975), 188--196.
[13]
Robert H Gault. 1924. Progress in experiments on tactual interpretation of oral speech. The Journal of Abnormal Psychology and Social Psychology 19, 2 (1924), 155.
[14]
Frank A. Geldard. 1957. Adventures in tactile literacy. American Psychologist 12, 3 (1957), 115--124.
[15]
Eric Gunther. 2001. Skinscape: A tool for composition in the tactile modality. Ph.D. Dissertation. Massachusetts Institute of Technology.
[16]
Eve E. Hoggan, Sohail Anwar, and Stephen A. Brewster. 2007. Mobile Multi-actuator Tactile Displays. 22--33 pages.
[17]
Yang Jiao, Frederico M. Severgnini, Juan Sebastian Martinez, Jaehong Jung, Hong Z. Tan, Charlotte M. Reed, E. Courtenay Wilson, Frances Lau, Ali Israr, Robert Turcott, Keith Klumb, and Freddy Abnousi. 2018. A Comparative Study of Phoneme- and Word-Based Learning of English Words Presented to the Skin. In Haptics: Science, Technology, and Applications. Springer International Publishing, Cham, 623--635.
[18]
Jacob H. Kirman. 1974. Tactile perception of computer-derived formant patterns from voiced speech. The Journal of the Acoustical Society of America 55, 1 (1974), 163--169.
[19]
Kevin Larson. 2004. The Science of Word Recognition. Advanced Reading Technology, Microsoft Corporation (July 2004). https://www.microsoft.com/typography/ctfonts/WordRecognition.aspx
[20]
Seungyon "Claire" Lee and Thad Starner. 2010. BuzzWear: Alert Perception in Wearable Tactile Displays on the Wrist. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM.
[21]
Yi-Chi Liao, Yi-Ling Chen, Jo-Yu Lo, Rong-Hao Liang, Liwei Chan, and Bing-Yu Chen. 2016. EdgeVib: Effective Alphanumeric Character Output Using a Wrist-Worn Tactile Display. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16).
[22]
Granit Luzhnica, Sebastian Stein, Eduardo Veas, Viktoria Pammer, John Williamson, and Roderick Murray Smith. 2017. Personalising Vibrotactile Displays Through Perceptual Sensitivity Adjustment. In Proceedings of the 2017 ACM International Symposium on Wearable Computers (ISWC '17). ACM, New York, NY, USA, 66--73.
[23]
Granit Luzhnica and Eduardo Veas. 2017. Vibrotactile Patterns using Sensitivity Prioritisation. In Proceedings of the 2017 ACM International Symposium on Wearable Computers (ISWC '17). ACM, New York, NY, USA, 148--155.
[24]
Granit Luzhnica and Eduardo Veas. 2018. Investigating Interactions for Text Recognition Using a Vibrotactile Wearable Display. In 23rd International Conference on Intelligent User Interfaces (IUI '18). ACM, New York, NY, USA, 453--465.
[25]
Granit Luzhnica and Eduardo Veas. 2018. Investigating Interactions for Text Recognition Using a Vibrotactile Wearable Display. In 23rd International Conference on Intelligent User Interfaces (IUI '18). ACM, New York, NY, USA, 453--465.
[26]
Granit Luzhnica and Eduardo Veas. 2018. Skin Reading Meets Speech Recognition and Object Recognition for Sensory Substitution. In Proceedings of the 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2018 ACM International Symposium on Wearable Computers (UbiComp '18). ACM, New York, NY, USA. https://doi.org/
[27]
Granit Luzhnica and Eduardo Veas. 2019. Background Perception and Comprehension of Symbols Conveyed through Vibrotactile Wearable Displays. In 24rd International Conference on Intelligent User Interfaces (IUI '19). ACM, New York, NY, USA.
[28]
Granit Luzhnica and Eduardo Veas. 2019. Optimising the Encoding for Vibrotactile Skin Reading. In ACM CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, New York, NY, USA.
[29]
Granit Luzhnica, Eduardo Veas, and Viktoria Pammer. 2016. Skin Reading: Encoding Text in a 6-channel Haptic Display. In Proceedings of the 2016 ACM International Symposium on Wearable Computers (ISWC '16). ACM, New York, NY, USA, 148--155.
[30]
Granit Luzhnica, Eduardo Veas, and Caitlyn Seim. 2018. Passive Haptic Learning for Vibrotactile Skin Reading. In Proceedings of the 2018 ACM International Symposium on Wearable Computers (ISWC '18). ACM.
[31]
Susanna Millar. 2004. Reading by Touch in Blind Children and Adults. Springer Netherlands, Dordrecht, 437--457.
[32]
Hugo Nicolau, João Guerreiro, Tiago Guerreiro, and Luís Carriço. 2013. UbiBraille: Designing and Evaluating a Vibrotactile Braille-reading Device. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '13). ACM, New York, NY, USA, Article 23, 8 pages.
[33]
Hugo Nicolau, Kyle Montague, Tiago Guerreiro, André Rodrigues, and Vicki L. Hanson. 2015. HoliBraille: Multipoint Vibrotactile Feedback on Mobile Devices. In Proceedings of the 12th Web for All Conference (W4A '15). ACM, New York, NY, USA, Article 30, 4 pages.
[34]
Calvin F Nodine and Francine G Simmons. 1974. Processing distinctive features in the differentiation of letterlike symbols. Journal of experimental psychology 103, 1 (1974), 21.
[35]
Scott David Novich. 2015. Sound-to-Touch Sensory Substitution and Beyond. Ph.D. Dissertation. Rice University.
[36]
Scott D. Novich and David M. Eagleman. 2015. Using space and time to encode vibrotactile information: toward an estimate of the skin's achievable throughput. Experimental Brain Research 10 (2015).
[37]
C. M. Reed, H. Z. Tan, Z. D. Perez, E. C. Wilson, F. M. Severgnini, J. Jung, J. S. Martinez, Y. Jiao, A. Israr, F. Lau, K. Klumb, R. Turcott, and F. Abnousi. 2018. A Phonemic-Based Tactile Display for Speech Communication. IEEE Transactions on Haptics (2018), 1--1.
[38]
Caitlyn E. Seim, David Quigley, and Thad E. Starner. 2014. Passive Haptic Learning of Typing Skills Facilitated by Wearable Computers. In CHI '14 Extended Abstracts on Human Factors in Computing Systems (CHI EA '14). ACM, New York, NY, USA, 2203--2208.
[39]
Frank Smith. 1969. Familiarity of configuration vs discriminability of features in the visual identification of words. Psychonomic Science 14, 6 (01 Jun 1969), 261--262.
[40]
Annette Sterr, Lisa Green, and Thomas Elbert. 2003. Blind Braille readers mislocate tactile stimuli. Biological Psychology 63, 2 (2003), 117 -- 127.
[41]
Ian R Summers, Jon J Whybrow, Denise A Gratton, Peter Milnes, Brian H Brown, and John C Stevens. 2005. Tactile information transfer: A comparison of two stimulation sites. The Journal of the Acoustical Society of America 118, 4 (2005), 2527--2534.
[42]
Hong Z. Tan, Nathaniel I. Durlach, William M. Rabinowitz, Charlotte M. Reed, and Jonathan R. Santos. 1997. Reception of Morse code through motional, vibrotactile, and auditory stimulation. Perception & Psychophysics 59, 7 (01 Jan 1997), 1004--1017.
[43]
David Ternes and K E MacLean. 2008. Designing Large Sets of Haptic Icons with Rhythm. In EuroHaptics, M. Ferre (Ed.). Springer, Madrid, Spain, 199--208.
[44]
Francisco Vega-Bermudez and Kenneth O Johnson. 2001. Differences in spatial acuity between digits. Neurology 56, 10 (2001), 1389--1391.
[45]
Benjamin W. White, Frank A. Saunders, Lawrence Scadden, Paul Bach-Y-Rita, and Carter C. Collins. 1970. Seeing with the skin. Perception & Psychophysics 7, 1 (1970), 23--27.
[46]
RS Woodworth. 1937. Experimental Psychology. New York: Holt, 1938. Department of Psychology Dartmouth College Hanover, New Hampshire (1937).
[47]
Cheng Xu, Ali Israr, Ivan Poupyrev, Olivier Bau, and Chris Harrison. 2011. Tactile Display for the Visually Impaired Using TeslaTouch. In CHI '11 Extended Abstracts on Human Factors in Computing Systems (CHI EA '11). ACM, New York, NY, USA, 317--322.
[48]
Siyan Zhao, Ali Israr, Frances Lau, and Freddy Abnousi. 2018. Coding Tactile Symbols for Phonemic Communication. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 392, 13 pages.

Cited By

View all
  • (2024)Functional Now, Wearable Later: Examining the Design Practices of Wearable TechnologistsProceedings of the 2024 ACM International Symposium on Wearable Computers10.1145/3675095.3676615(71-81)Online publication date: 5-Oct-2024
  • (2022)Exploring Feedback-based Testing Effects for Skin ReadingProceedings of the Augmented Humans International Conference 202210.1145/3519391.3519393(212-217)Online publication date: 13-Mar-2022
  • (2022)A Visualisation Driven Training for Vibrotactile Skin ReadingIEEE Transactions on Haptics10.1109/TOH.2021.313877315:1(103-108)Online publication date: 1-Jan-2022
  • Show More Cited By

Index Terms

  1. Boosting word recognition for vibrotactile skin reading

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ISWC '19: Proceedings of the 2019 ACM International Symposium on Wearable Computers
    September 2019
    355 pages
    ISBN:9781450368704
    DOI:10.1145/3341163
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 09 September 2019

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. HCI
    2. haptic display
    3. haptic feedback
    4. skin reading
    5. tactile feedback
    6. traing
    7. user study
    8. wearable
    9. word recognition

    Qualifiers

    • Research-article

    Conference

    UbiComp '19

    Acceptance Rates

    Overall Acceptance Rate 38 of 196 submissions, 19%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)16
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Functional Now, Wearable Later: Examining the Design Practices of Wearable TechnologistsProceedings of the 2024 ACM International Symposium on Wearable Computers10.1145/3675095.3676615(71-81)Online publication date: 5-Oct-2024
    • (2022)Exploring Feedback-based Testing Effects for Skin ReadingProceedings of the Augmented Humans International Conference 202210.1145/3519391.3519393(212-217)Online publication date: 13-Mar-2022
    • (2022)A Visualisation Driven Training for Vibrotactile Skin ReadingIEEE Transactions on Haptics10.1109/TOH.2021.313877315:1(103-108)Online publication date: 1-Jan-2022
    • (2020)Making Sensors, Making Sense, Making Stimuli: The State of the Art in Wearables Research From ISWC 2019IEEE Pervasive Computing10.1109/MPRV.2020.296408819:1(87-91)Online publication date: Jan-2020

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media