skip to main content
10.1145/3290605.3300465acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Optimising Encoding for Vibrotactile Skin Reading

Published: 02 May 2019 Publication History

Abstract

This paper proposes methods of optimising alphabet encoding for skin reading in order to avoid perception errors. First, a user study with 16 participants using two body locations serves to identify issues in recognition of both individual letters and words. To avoid such issues, a two-step optimisation method of the symbol encoding is proposed and validated in a second user study with eight participants using the optimised encoding with a seven vibromotor wearable layout on the back of the hand. The results show significant improvements in the recognition accuracy of letters (97%) and words (97%) when compared to the non-optimised encoding.

Supplementary Material

ZIP File (pn2458.zip)
This document provides more details to the user study 2 which were omitted due to the page length constrains.

References

[1]
J. C. Bliss, M. H. Katcher, C. H. Rogers, and R. P. Shepard. 1970. Opticalto-Tactile Image Conversion for the Blind. IEEE Transactions on ManMachine Systems 11, 1 (March 1970), 58--65.
[2]
Lukasz Bola, Katarzyna Siuda-Krzywicka, Malgorzata Papli'ska, Ewa Sumera, Pawel Hanczur, and Marcin Szwed Szwed. 2016. Braille in the sighted: Teaching tactile reading to sighted adults. PLoS ONE (2016).
[3]
Jessica R. Cauchard, Janette L. Cheng, Thomas Pietrzak, and James A. Landay. 2016. ActiVibe: Design and Evaluation of Vibrations for Progress Monitoring. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 3261--3271.
[4]
Jennifer Chen, Pablo Castillo, Robert Turcott, Ali Israr, and Frances Lau. 2018. Feeling Speech on the Arm. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA '18). ACM, New York, NY, USA, Article D107, 4 pages.
[5]
Jennifer Chen, Robert Turcott, Pablo Castillo, Wahyudinata Setiawan, Frances Lau, and Ali Israr. 2018. Learning to Feel Words: A Comparison of Learning Approaches to Acquire Haptic Words. In Proceedings of the 15th ACM Symposium on Applied Perception (SAP '18). ACM, New York, NY, USA, Article 11, 7 pages.
[6]
Roger W. Cholewiak and Amy A. Collins. 1995. Vibrotactile pattern discrimination and communality at several body sites. Perception & Psychophysics 57, 5 (1995), 724--737.
[7]
Roger W. Cholewiak and Amy A. Collins. 2000. The generation of vibrotactile patterns on a linear array: Influences of body site, time, and presentation mode. Perception & Psychophysics 62, 6 (2000), 1220--1235.
[8]
Roger W. Cholewiak and James C. Craig. {n. d.}. Vibrotactile pattern recognition and discrimination at several body sites. Perception & Psychophysics 35, 6 ({n. d.}), 503--514.
[9]
P.C. Chu and J.E. Beasley. 1997. A genetic algorithm for the generalised assignment problem. Computers & Operations Research 24, 1 (1997), 17 -- 23.
[10]
P. C. Chu and J. E. Beasley. 1997. A Genetic Algorithm for the Generalised Assignment Problem. Comput. Oper. Res. 24, 1 (Jan. 1997), 17--23.
[11]
James C. Craig and Paul M. Evans. 1987. Vibrotactile masking and the persistence of tactual features. Perception & Psychophysics 42, 4 (01 Jul 1987), 309--317.
[12]
de Jesus Oliveira and Anderson Maciel. 2014. Introducing the Modifier Tactile Pattern for Vibrotactile Communication. In EUROHAPTICS, 2014.
[13]
Hubert R. Dinse, Tobias Kalisch, Patrick Ragert, Burkhard Pleger, Peter Schwenkreis, and Martin Tegenthoff. 2005. Improving Human Haptic Performance in Normal and Impaired Human Populations Through Unattended Activation-based Learning. ACM Trans. Appl. Percept. 2, 2 (April 2005), 71--88.
[14]
Nathan Dunkelberger, Jenny Sullivan, Joshua Bradley, Nickolas P Walling, Indu Manickam, Gautam Dasarathy, Ali Israr, Frances W. Y. Lau, Keith Klumb, Brian Knott, Freddy Abnousi, Richard Baraniuk, and Marcia K O'Malley. 2018. Conveying Language Through Haptics: A Multi-sensory Approach. In Proceedings of the 2018 ACM International Symposium on Wearable Computers (ISWC '18). ACM, New York, NY, USA, 25--32.
[15]
Robert H Gault. 1924. Progress in experiments on tactual interpretation of oral speech. The Journal of Abnormal Psychology and Social Psychology 19, 2 (1924), 155.
[16]
Frank A. Geldard. 1957. Adventures in tactile literacy. American Psychologist 12, 3 (1957), 115--124.
[17]
Ben Godde, Beate Stauffenberg, Friederike Spengler, and Hubert R. Dinse. 2000. Tactile Coactivation-Induced Changes in Spatial Discrimination Performance. Journal of Neuroscience 20, 4 (2000), 1597-- 1604.
[18]
Eric Gunther. 2001. Skinscape: A tool for composition in the tactile modality. Ph.D. Dissertation. Massachusetts Institute of Technology.
[19]
Amra Hodzic, Ralf Veit, Ahmed A. Karim, Michael Erb, and Ben Godde. 2004. Improvement and Decline in Tactile Discrimination Behavior after Cortical Plasticity Induced by Passive Tactile Coactivation. Journal of Neuroscience 24, 2 (2004), 442--446. arXiv:http://www.jneurosci.org/content/24/2/442.full.pdf
[20]
Yang Jiao, Frederico M. Severgnini, Juan Sebastian Martinez, Jaehong Jung, Hong Z. Tan, Charlotte M. Reed, E. Courtenay Wilson, Frances Lau, Ali Israr, Robert Turcott, Keith Klumb, and Freddy Abnousi. 2018. A Comparative Study of Phoneme- and Word-Based Learning of English Words Presented to the Skin. In Haptics: Science, Technology, and Applications, Domenico Prattichizzo, Hiroyuki Shinoda, Hong Z. Tan, Emanuele Ruffaldi, and Antonio Frisoli (Eds.). Springer International Publishing, Cham, 623--635.
[21]
Jacob H. Kirman. 1974. Tactile perception of computer-derived formant patterns from voiced speech. The Journal of the Acoustical Society of America 55, 1 (1974).
[22]
Lori Lamel, Robert H. Kassel, and Stephanie Seneff. 1989. Speech database development: Design and analysis of the acoustic-phonetic corpus. Proceedings of the DARPA Speech Recognition Workshop.
[23]
Seungyon "Claire" Lee and Thad Starner. 2010. BuzzWear: Alert Perception in Wearable Tactile Displays on the Wrist. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 433--442.
[24]
Yi-Chi Liao, Yi-Ling Chen, Jo-Yu Lo, Rong-Hao Liang, Liwei Chan, and Bing-Yu Chen. 2016. EdgeVib: Effective Alphanumeric Character Output Using a Wrist-Worn Tactile Display. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, New York, NY, USA, 7.
[25]
Granit Luzhnica, Sebastian Stein, Eduardo Veas, Viktoria Pammer, John Williamson, and Roderick Murray Smith. 2017. Personalising Vibrotactile Displays Through Perceptual Sensitivity Adjustment. In Proceedings of the 2017 ACM International Symposium on Wearable Computers (ISWC '17). ACM, New York, NY, USA, 66--73.
[26]
Granit Luzhnica and Eduardo Veas. 2017. Vibrotactile Patterns using Sensitivity Prioritisation. In Proceedings of the 2017 ACM International Symposium on Wearable Computers (ISWC '17). ACM, New York, NY, USA, 148--155.
[27]
Granit Luzhnica and Eduardo Veas. 2018. Investigating Interactions for Text Recognition Using a Vibrotactile Wearable Display. In 23rd International Conference on Intelligent User Interfaces (IUI '18). ACM, New York, NY, USA, 453--465.
[28]
Granit Luzhnica and Eduardo Veas. 2018. Skin Reading Meets Speech Recognition and Object Recognition for Sensory Substitution. In Proceedings of the 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2018 ACM International Symposium on Wearable Computers (UbiComp '18). ACM, New York, NY, USA.
[29]
Granit Luzhnica and Eduardo Veas. 2019. Background Perception and Comprehension of Symbols Conveyed through Vibrotactile Wearable Displays. In 24rd International Conference on Intelligent User Interfaces (IUI '19). ACM, New York, NY, USA.
[30]
Granit Luzhnica, Eduardo Veas, and Viktoria Pammer. 2016. Skin Reading: Encoding Text in a 6-channel Haptic Display. In Proceedings of the 2016 ACM International Symposium on Wearable Computers (ISWC '16). ACM, New York, NY, USA, 148--155.
[31]
Granit Luzhnica, Eduardo Veas, and Caitlyn Seim. 2018. Passive Haptic Learning for Vibrotactile Skin Reading. In Proceedings of the 2018 ACM International Symposium on Wearable Computers (ISWC '18). ACM.
[32]
Jean-Baptiste Michel, Yuan Kui Shen, Aviva Presser Aiden, Adrian Veres, Matthew K. Gray, Joseph P. Pickett, Dale Hoiberg, Dan Clancy, Peter Norvig, Jon Orwant, Steven Pinker, Martin A. Nowak, and Erez Lieberman Aiden. 2011. Quantitative Analysis of Culture Using Millions of Digitized Books. Science 331, 6014 (2011), 176--182. arXiv:http://science.sciencemag.org/content/331/6014/176.full.pdf
[33]
Hugo Nicolau, João Guerreiro, Tiago Guerreiro, and Luís Carriço. 2013. UbiBraille: Designing and Evaluating a Vibrotactile Braille-reading Device. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '13). ACM, New York, NY, USA, Article 23, 8 pages.
[34]
Hugo Nicolau, Kyle Montague, Tiago Guerreiro, André Rodrigues, and Vicki L. Hanson. 2015. HoliBraille: Multipoint Vibrotactile Feedback on Mobile Devices. In Proceedings of the 12th Web for All Conference (W4A '15). ACM, New York, NY, USA, Article 30, 4 pages.
[35]
Scott David Novich. 2015. Sound-to-Touch Sensory Substitution and Beyond. Ph.D. Dissertation. Rice University.
[36]
Scott D. Novich and David M. Eagleman. 2015. Using space and time to encode vibrotactile information: toward an estimate of the skin's achievable throughput. Experimental Brain Research 10 (2015).
[37]
IH Osman and CN Potts. 1989. Simulated annealing for permutation flow-shop scheduling. Omega 17, 6 (1989), 551 -- 557.
[38]
A. J. Phillips, A. R. D. Thornton, S. Worsfold, A. Downie, and J. Milligan. 1994. Experience of using vibrotactile aids with the profoundly deafened. International Journal of Language and Communication Disorders 29, 1 (1994), 17--26.
[39]
Burkhard Pleger, Hubert R. Dinse, Patrick Ragert, Peter Schwenkreis, Jean Pierre Malin, and Martin Tegenthoff. 2001. Shifts in cortical representations predict human discrimination improvement. Proceedings of the National Academy of Sciences 98, 21 (2001), 12255--12260. arXiv:http://www.pnas.org/content/98/21/12255.full.pdf
[40]
Burkhard Pleger, Ann-Freya Foerster, Patrick Ragert, Hubert R Dinse, Peter Schwenkreis, Jean-Pierre Malin, Volkmar Nicolas, and Martin Tegenthoff. 2003. Functional imaging of perceptual learning in human primary and secondary somatosensory cortex. Neuron 40, 3 (2003), 643--653.
[41]
Charlotte M Reed and Lorraine A Delhorne. 2003. The reception of environmental sounds through wearable tactual aids. Ear and hearing 24, 6 (2003), 528--538.
[42]
C. M. Reed, H. Z. Tan, Z. D. Perez, E. C. Wilson, F. M. Severgnini, J. Jung, J. S. Martinez, Y. Jiao, A. Israr, F. Lau, K. Klumb, R. Turcott, and F. Abnousi. 2018. A Phonemic-Based Tactile Display for Speech Communication. IEEE Transactions on Haptics (2018), 1--1.
[43]
Colin R. Reeves. 1995. A genetic algorithm for flowshop sequencing. Computers & Operations Research 22, 1 (1995), 5 -- 13. Genetic Algorithms.
[44]
Eva-Maria Reuter, Claudia Voelcker-Rehage, Solveig Vieluf, and Ben Godde. 2014. Effects of age and expertise on tactile learning in humans. European Journal of Neuroscience 40, 3 (2014), 2589--2599.
[45]
Jerker Rönnberg, Ulf Andersson, Björn Lyxell, and Karl-Erik Spens. 1998. Vibrotactile Speech Tracking Support: Cognitive Prerequisites. The Journal of Deaf Studies and Deaf Education 3, 2 (1998).
[46]
B. L. Scott and C. L. De Filippo. 1977. Progress in the development of a tactile aid for the deaf. The Journal of the Acoustical Society of America 62, S1 (1977), S76--S76. arXiv:http://dx.doi.org/10.1121/1.2016369
[47]
Caitlyn E. Seim, David Quigley, and Thad E. Starner. 2014. Passive Haptic Learning of Typing Skills Facilitated by Wearable Computers. In CHI '14 Extended Abstracts on Human Factors in Computing Systems (CHI EA '14). ACM, New York, NY, USA, 2203--2208.
[48]
Ian R Summers, Jon J Whybrow, Denise A Gratton, Peter Milnes, Brian H Brown, and John C Stevens. 2005. Tactile information transfer: A comparison of two stimulation sites. The Journal of the Acoustical Society of America 118, 4 (2005), 2527--2534.
[49]
David Ternes and K E MacLean. 2008. Designing Large Sets of Haptic Icons with Rhythm. In EuroHaptics, M. Ferre (Ed.). Springer, Madrid, Spain, 199--208. http://www.cs.ubc.ca/labs/spin/publications/spin/ ternes2008eh.pdf
[50]
Benjamin W. White, Frank A. Saunders, Lawrence Scadden, Paul BachY-Rita, and Carter C. Collins. 1970. Seeing with the skin. Perception & Psychophysics 7, 1 (1970), 23--27.
[51]
Cheng Xu, Ali Israr, Ivan Poupyrev, Olivier Bau, and Chris Harrison. 2011. Tactile Display for the Visually Impaired Using TeslaTouch. In CHI '11 Extended Abstracts on Human Factors in Computing Systems (CHI EA '11). ACM, New York, NY, USA, 317--322.
[52]
Hanfeng Yuan, Charlotte M. Reed, and Nathaniel I. Durlach. 2005. Tactual display of consonant voicing as a supplement to lipreading. The Journal of the Acoustical Society of America 118, 2 (2005), 1003--1015. arXiv:http://dx.doi.org/10.1121/1.1945787
[53]
Siyan Zhao, Ali Israr, Frances Lau, and Freddy Abnousi. 2018. Coding Tactile Symbols for Phonemic Communication. In CHI Conference on Human Factors in Computing Systems.

Cited By

View all
  • (2025)Enhancing Pitch Perception and Melodic Reproduction of Hard-of-Hearing Individuals Using a Vibrotactile GloveInternational Journal of Human–Computer Interaction10.1080/10447318.2025.2452192(1-18)Online publication date: 5-Feb-2025
  • (2024)Passive Haptic Rehearsal for Augmented Piano Learning in the WildProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997488:4(1-26)Online publication date: 21-Nov-2024
  • (2024)Encoding Data Through Tactile Vibrations2024 28th International Conference Information Visualisation (IV)10.1109/IV64223.2024.00011(1-6)Online publication date: 22-Jul-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
May 2019
9077 pages
ISBN:9781450359702
DOI:10.1145/3290605
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. encoding
  2. haptic feedback
  3. hci
  4. optimisation
  5. skin reading
  6. stimulation
  7. tactile feedback
  8. user study
  9. wearable

Qualifiers

  • Research-article

Funding Sources

Conference

CHI '19
Sponsor:

Acceptance Rates

CHI '19 Paper Acceptance Rate 703 of 2,958 submissions, 24%;
Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)59
  • Downloads (Last 6 weeks)2
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Enhancing Pitch Perception and Melodic Reproduction of Hard-of-Hearing Individuals Using a Vibrotactile GloveInternational Journal of Human–Computer Interaction10.1080/10447318.2025.2452192(1-18)Online publication date: 5-Feb-2025
  • (2024)Passive Haptic Rehearsal for Augmented Piano Learning in the WildProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997488:4(1-26)Online publication date: 21-Nov-2024
  • (2024)Encoding Data Through Tactile Vibrations2024 28th International Conference Information Visualisation (IV)10.1109/IV64223.2024.00011(1-6)Online publication date: 22-Jul-2024
  • (2024)CoTacs: A Haptic Toolkit to Explore Effective On-Body Haptic Feedback by Ideating, Designing, Evaluating and Refining Haptic Designs Using Group CollaborationInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2358460(1-21)Online publication date: 7-Jun-2024
  • (2024)Wearable haptics for virtual reality and beyondNature Reviews Electrical Engineering10.1038/s44287-024-00089-81:10(666-679)Online publication date: 25-Sep-2024
  • (2024)Memorable Vibration Pattern Design Based on Writing PatternHaptics: Understanding Touch; Technology and Systems; Applications and Interaction10.1007/978-3-031-70058-3_37(449-463)Online publication date: 30-Jun-2024
  • (2023)A Wearable Haptic Interface for Assisting Blind and Visually Impaired Students in Learning Algebraic EquationsExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585815(1-7)Online publication date: 19-Apr-2023
  • (2023)Relative Design Acquisition: A Computational Approach for Creating Visual Interfaces to Steer User ChoicesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581028(1-17)Online publication date: 19-Apr-2023
  • (2023)OPTIMISM: Enabling Collaborative Implementation of Domain Specific Metaheuristic OptimizationProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580904(1-19)Online publication date: 19-Apr-2023
  • (2023)Training to Understand Complex Haptic Phrases: A Longitudinal Investigation2023 IEEE World Haptics Conference (WHC)10.1109/WHC56415.2023.10224492(141-147)Online publication date: 10-Jul-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media