note

Tongue-in-Cheek: Using Wireless Signals to Enable Non-Intrusive and Flexible Facial Gestures Detection

Authors:

Mayank Goel,

Chen Zhao,

Ruth Vinisha,

Shwetak N. PatelAuthors Info & Claims

CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

Pages 255 - 258

https://doi.org/10.1145/2702123.2702591

Published: 18 April 2015 Publication History

Get Access

Abstract

Serious brain injuries, spinal injuries, and motor neuron diseases often lead to severe paralysis. Individuals with such disabilities can benefit from interaction techniques that enable them to interact with the devices and thereby the world around them. While a number of systems have proposed tongue-based gesture detection systems, most of these systems require intrusive instrumentation of the user's body (e.g., tongue piercing, dental retainers, multiple electrodes on chin). In this paper, we propose a wireless, non-intrusive and non-contact facial gesture detection system using X-band Doppler. The system can accurately differentiate between 8 different facial gestures through non-contact sensing, with an average accuracy of 94.3%.

References

[1]

Huo, X., Wang, J., and Ghovanloo, M. A Magneto-Inductive Sensor Based Wireless Tongue-Computer Interface. Neural Systems and Rehabilitation Engineering 16, 5 (2008).

Google Scholar

[2]

Lukaszewicz, K. The Ultrasound Image of the Tongue Surface as Input for Man/Machine Interface. Proc. INTERACT '03.

Google Scholar

[3]

Saponas, T.S., Kelly, D., Parviz, B. a., and Tan, D.S. Optically sensing tongue gestures for computer input. Proc. UIST'09.

Digital Library

Google Scholar

[4]

Sasaki, M., Arakawa, T., Nakayama, A., Obinata, G., and Yamaguchi, M. Estimation of tongue movement based on suprahyoid muscle activity. Proc. IEEE Engineering in Medicine and Biology Society. 2013.

Google Scholar

[5]

Wilson-Pauwels, L., Akesson, E.J., and Stewart, P.A. Cranial Nerves: Anatomy and Clinical Comments. 1988.

Google Scholar

[6]

Wobbrock, J.O., Myers, B.A., and Kembel, J.A. EdgeWrite: A Stylus-Based Text Entry Method Designed for High Accuracy and Stability of Motion. Proc. UIST 2003.

Digital Library

Google Scholar

[7]

Zhang, Q., Gollakota, S., Taskar, B., and Rao, R.P.N. NonIntrusive Tongue Machine Interface. Proc. CHI'14.

Digital Library

Google Scholar

Cited By

View all

Chidziwisano GCummings APotnis DJalakasi M(2024)Understanding Urban Women's Interaction with Domestic Technologies in Malawi: Advancing Feminist Theories in HCIACM Journal on Computing and Sustainable Societies10.1145/36502042:2(1-25)Online publication date: 5-Mar-2024
https://dl.acm.org/doi/10.1145/3650204
Dong XChen YNishiyama YSezaki KWang YChristofferson KMariakakis A(2024)ReHEarSSE: Recognizing Hidden-in-the-Ear Silently Spelled ExpressionsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642095(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642095
Wong DChou P(2023)TongueMendous: IR-Based Tongue-Gesture Interface with Tiny Machine LearningProceedings of the 8th international Workshop on Sensor-Based Activity Recognition and Artificial Intelligence10.1145/3615834.3615843(1-8)Online publication date: 21-Sep-2023
https://dl.acm.org/doi/10.1145/3615834.3615843
Show More Cited By

Index Terms

Tongue-in-Cheek: Using Wireless Signals to Enable Non-Intrusive and Flexible Facial Gestures Detection
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Touch screens

Recommendations

Tongue-n-cheek: non-contact tongue gesture recognition
IPSN '15: Proceedings of the 14th International Conference on Information Processing in Sensor Networks

Tongue gestures are a key modality for augmentative and alternative communication in patients suffering from speech impairments and full-body paralysis. Systems for recognizing tongue gestures, however, are highly intrusive. They either rely on magnetic ...
TongueTap: Multimodal Tongue Gesture Recognition with Head-Worn Devices
ICMI '23: Proceedings of the 25th International Conference on Multimodal Interaction

Mouth-based interfaces are a promising new approach enabling silent, hands-free and eyes-free interaction with wearable devices. However, interfaces sensing mouth movements are traditionally custom-designed and placed near or within the mouth. TongueTap ...
Gaze & Tongue: A Subtle, Hands-Free Interaction for Head-Worn Devices
CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

Gaze tracking allows hands-free and voice-free interaction with computers, and has gained more use recently in virtual and augmented reality headsets. However, it traditionally uses dwell time for selection tasks, which suffers from the Midas Touch ...

Comments

Information & Contributors

Information

Published In

CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

April 2015

4290 pages

ISBN:9781450331456

DOI:10.1145/2702123

General Chairs:
Bo Begole
Huawei, USA
,
Jinwoo Kim
Yonsei University, Korea
,
Program Chairs:
Kori Inkpen
Microsoft Research, USA
,
Woontack Woo
KAIST, Korea

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 April 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Note

Conference

CHI '15

Sponsor:

SIGCHI

CHI '15: CHI Conference on Human Factors in Computing Systems

April 18 - 23, 2015

Seoul, Republic of Korea

Acceptance Rates

CHI '15 Paper Acceptance Rate 486 of 2,120 submissions, 23%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

35
Total Citations
View Citations
1,046
Total Downloads

Downloads (Last 12 months)39
Downloads (Last 6 weeks)2

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Chidziwisano GCummings APotnis DJalakasi M(2024)Understanding Urban Women's Interaction with Domestic Technologies in Malawi: Advancing Feminist Theories in HCIACM Journal on Computing and Sustainable Societies10.1145/36502042:2(1-25)Online publication date: 5-Mar-2024
https://dl.acm.org/doi/10.1145/3650204
Dong XChen YNishiyama YSezaki KWang YChristofferson KMariakakis A(2024)ReHEarSSE: Recognizing Hidden-in-the-Ear Silently Spelled ExpressionsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642095(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642095
Wong DChou P(2023)TongueMendous: IR-Based Tongue-Gesture Interface with Tiny Machine LearningProceedings of the 8th international Workshop on Sensor-Based Activity Recognition and Artificial Intelligence10.1145/3615834.3615843(1-8)Online publication date: 21-Sep-2023
https://dl.acm.org/doi/10.1145/3615834.3615843
Sun RZhou XSteeper BZhang RYin SLi KWu STilsen SGuimbretiere FZhang C(2023)EchoNose: Sensing Mouth, Breathing and Tongue Gestures inside Oral Cavity using a Non-contact Nose InterfaceProceedings of the 2023 ACM International Symposium on Wearable Computers10.1145/3594738.3611358(22-26)Online publication date: 8-Oct-2023
https://dl.acm.org/doi/10.1145/3594738.3611358
Bhatia ASaini AKalra IMukherjee MParnami AWard JMcGill MMarky K(2023)DUMask: A Discrete and Unobtrusive Mask-Based Interface for Facial GesturesProceedings of the Augmented Humans International Conference 202310.1145/3582700.3582726(255-266)Online publication date: 12-Mar-2023
https://dl.acm.org/doi/10.1145/3582700.3582726
Gemicioglu TWinters RWang YGable TTashev I(2023)TongueTap: Multimodal Tongue Gesture Recognition with Head-Worn DevicesProceedings of the 25th International Conference on Multimodal Interaction10.1145/3577190.3614120(564-573)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3577190.3614120
Jingu ATanaka YLopes P(2023)LipIO: Enabling Lips as both Input and Output SurfaceProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580775(1-14)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580775
Shen XYan YYu CShi Y(2022)ClenchClickProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35503276:3(1-26)Online publication date: 7-Sep-2022
https://dl.acm.org/doi/10.1145/3550327
Gemicioglu TWinters RWang YTashev I(2022)Tongue Gestures for Hands-Free Interaction in Head Worn DisplaysAdjunct Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 ACM International Symposium on Wearable Computers10.1145/3544793.3560363(38-40)Online publication date: 11-Sep-2022
https://dl.acm.org/doi/10.1145/3544793.3560363
Vatavu RUngurean O(2022)Understanding Gesture Input Articulation with Upper-Body Wearables for Users with Upper-Body Motor ImpairmentsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501964(1-16)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501964
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Tongue-n-cheek: non-contact tongue gesture recognition

TongueTap: Multimodal Tongue Gesture Recognition with Head-Worn Devices

Gaze & Tongue: A Subtle, Hands-Free Interaction for Head-Worn Devices

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations