skip to main content
10.1145/3267305.3267604acmconferencesArticle/Chapter ViewAbstractPublication PagesubicompConference Proceedingsconference-collections
poster

Skin Reading Meets Speech Recognition and Object Recognition for Sensory Substitution

Published: 08 October 2018 Publication History

Abstract

Sensory substitution has been a research subject for decades, and yet its applicability outside of the research is very limited. Thus creating scepticism among researchers that a full sensory substitution is not even possible [8]. In this paper, we do not substitute the entire perceptual channel. Instead, we follow a different approach which reduces the captured information drastically. We present concepts and implementation of two mobile applications which capture the user's environment, describe it in the form of text and then convey its textual description to the user through a vibrotactile wearable display. The applications target users with hearing and vision impairments.

References

[1]
Robert H Gault. 1924. Progress in experiments on tactual interpretation of oral speech. The Journal of Abnormal Psychology and Social Psychology (1924).
[2]
Granit Luzhnica, Sebastian Stein, Eduardo Veas, Viktoria Pammer, John Williamson, and Roderick Murray Smith. 2017. Personalising Vibrotactile Displays Through Perceptual Sensitivity Adjustment. In Proceedings of the 2017 ACM International Symposium on Wearable Computers (ISWC '17). ACM, New York, NY, USA, 66--73.
[3]
Granit Luzhnica and Eduardo Veas. 2017. Vibrotactile Patterns Using Sensitivity Prioritisation. In Proceedings of the 2017 ACM International Symposium on Wearable Computers (ISWC '17). 74--81.
[4]
Granit Luzhnica and Eduardo Veas. 2018. Investigating Interactions for Text Recognition Using a Vibrotactile Wearable Display. In 23rd International Conference on Intelligent User Interfaces (IUI '18). 453--465.
[5]
Granit Luzhnica, Eduardo Veas, and Viktoria Pammer. 2016. Skin Reading: Encoding Text in a 6-channel Haptic Display. In Proceedings of the 2016 ACM International Symposium on Wearable Computers (ISWC '16). ACM, New York, NY, USA.
[6]
Granit Luzhnica, Eduardo Veas, and Caitlyn Seim. 2018. Passive Haptic Learning for Vibrotactile Skin Reading. In Proceedings of the 2018 ACM International Symposium on Wearable Computers (ISWC '18). ACM.
[7]
Scott David Novich. 2015. Sound-to-touch sensory substitution and beyond. Ph.D. Dissertation. Rice University.
[8]
Charles Spence. 2014. The Skin as a Medium for Sensory Substitution. Multisensory Research 27, 5-6 (2014), 293--312.
[9]
Benjamin W. White, Frank A. Saunders, Lawrence Scadden, Paul Bach-Y-Rita, and Carter C. Collins. 1970. Seeing with the skin. Perception & Psychophysics 7, 1 (1970), 23--27.

Cited By

View all
  • (2022)Exploring Feedback-based Testing Effects for Skin ReadingProceedings of the Augmented Humans International Conference 202210.1145/3519391.3519393(212-217)Online publication date: 13-Mar-2022
  • (2022)A Visualisation Driven Training for Vibrotactile Skin ReadingIEEE Transactions on Haptics10.1109/TOH.2021.313877315:1(103-108)Online publication date: 1-Jan-2022
  • (2019)Boosting word recognition for vibrotactile skin readingProceedings of the 2019 ACM International Symposium on Wearable Computers10.1145/3341163.3347715(135-144)Online publication date: 9-Sep-2019
  • Show More Cited By

Index Terms

  1. Skin Reading Meets Speech Recognition and Object Recognition for Sensory Substitution

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    UbiComp '18: Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers
    October 2018
    1881 pages
    ISBN:9781450359665
    DOI:10.1145/3267305
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Sponsors

    In-Cooperation

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 08 October 2018

    Check for updates

    Author Tags

    1. HCI
    2. assistive technologies
    3. haptic display
    4. image recognition
    5. sensory substitution
    6. skin reading
    7. speech recognition
    8. stimulation
    9. tactile feedback
    10. wearable

    Qualifiers

    • Poster
    • Research
    • Refereed limited

    Funding Sources

    Conference

    UbiComp '18
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 764 of 2,912 submissions, 26%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)5
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)Exploring Feedback-based Testing Effects for Skin ReadingProceedings of the Augmented Humans International Conference 202210.1145/3519391.3519393(212-217)Online publication date: 13-Mar-2022
    • (2022)A Visualisation Driven Training for Vibrotactile Skin ReadingIEEE Transactions on Haptics10.1109/TOH.2021.313877315:1(103-108)Online publication date: 1-Jan-2022
    • (2019)Boosting word recognition for vibrotactile skin readingProceedings of the 2019 ACM International Symposium on Wearable Computers10.1145/3341163.3347715(135-144)Online publication date: 9-Sep-2019
    • (2019)Background perception and comprehension of symbols conveyed through vibrotactile wearable displaysProceedings of the 24th International Conference on Intelligent User Interfaces10.1145/3301275.3302282(57-64)Online publication date: 17-Mar-2019
    • (2019)Optimising Encoding for Vibrotactile Skin ReadingProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300465(1-14)Online publication date: 2-May-2019

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media