skip to main content
10.1145/1088463.1088512acmconferencesArticle/Chapter ViewAbstractPublication Pagesicmi-mlmiConference Proceedingsconference-collections
Article

Gesture-driven American sign language phraselator

Published: 04 October 2005 Publication History

Abstract

This paper describes a portable American Sign Language (ASL)-to-English phraselator. This wearable device is based on an Acceleglove originally developed for recognizing the hand alphabet, and a two-link arm skeleton that detects hand location and movement with respect to the body. Therefore, this phraselator is able to recognize finger-spelled words as well as hand gestures and translate them into spoken voice through a speech synthesizer. To speed-up the recognition process, a simple prediction algorithm has been introduced so the phraselator predicts words based on the current letter being inputted, or complete sentences based on the current sign being translated. The user selects the rest of the sentence (or word) by means of a predefined hand gesture for the phraselator to speak out the sentence in English or Spanish. New words of phrases are automatically added to the lexicon for future predictions.

References

[1]
Bauer, B., Hienz, H., and Kraiss, K., Video-Based Continuous Sign Language Recognition Using Statistical Methods. IEEE 2000, pp 463--466.
[2]
Costelo, E. Random House Webster's Concise American Sign Language Dictionary. Random House Inc. NY. 1999
[3]
Fels, Sidney S., and Hinton, Geoffrey E., Glove Talk -A Neural-Network Interface Between a Data-Glove and a Speech Synthesizer. IEEE Transactions on Neural Networks, vol. 4, No. 1. January 1993.
[4]
Grimes, G.,. US Patent 4,414,537. November 1983.
[5]
Hernandez Jose L., Kyriakopoulos, N., Lindeman, R. The AcceleGlove a Hole-Hand Input Device for Virtual Reality. ACM SIGGRAPH Conference Abstracts and Applications 2002. San Antonio, Tx, USA, pp 259
[6]
Hernandez, Jose L., Kyriakopoulos, N., Lindeman R., A Multi-Class Pattern Recognition of Practical Finger Spelling Translation, IEEE International Conference on Multimodal Interfaces ICMI'02. October 2002, pp 185--190.
[7]
Hernandez Jose L., Kyriakopoulos, N, Lindeman, R. A New Instrumented Approach For Translating American Sign Language Into Sound And Text. IEEE International Conference on Face and Gesture Recognition 2004, Seoul, Korea, May, 2004.
[8]
Hernandez-Rebollar, Jose L., Phonetic Model for Automatic Recognition of Hand Gesture. IV International Conference on Language Resources and Evaluation.LREC200. Representation and Processing of Sign Languages Workshop proceedings. May 30th 2004, Lisbon, Portugal.
[9]
Kramer, J., and Leifer, L., The Talking Glove: An Expressive and Receptive Verbal Communication Aid for the Deaf, Deaf-Blind, and Nonvocal. SIGCAPH 39, pp.12--15 (spring 1988).
[10]
Lamar, Marcus V., Bhuiyan, S., and Iwata, A., Hand Alphabet Recognition Using Morphological PCA and Neural Networks. Proceedings of the International Joint conference on Nural Networks. Vol 4, pp 2839--2844, Washington, USA 1994.
[11]
Liang, R., and Ouhyoung M. A Real-time Continuous Gesture Recognition System for Sign Language. Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, 1998. 558--567
[12]
Lidell S., and Johnson, R., American Sign Language: The phonological base. Sign Language Studies, 64: 195--277, 1989.
[13]
McGuire, R., Hernandez-Rebollar, J., Starner, T., Henderson, V., Brashear, H., Towards a One Way American Sign Language Translator. IEEE International Conference on Face and Gesture Recognition 2004, Seoul, Korea, May, 2004.
[14]
Perlmutter, D., A moisac theory of American Sign Language syllable structure. Paper presented at Second Conference on Theoretical Issues in Sign Language Research. Gallaudet University, Washington, DC 1988.
[15]
Starner, T., Weaver, J., and Pentland, A., A Wearable Computer Based American Sign Language Recognizer, MIT Media Lab. Technical Report 425. 1998
[16]
Stokoe, William C., Armstrong, David F., Wilcox, Sherman E. Gesture and the Nature of Language. Cambridge University Press, 1995
[17]
Uras, C., and Verri, A., On the Recognition of The Alphabet of the Sign Language through Size Functions. In Proceedings of the 12th IAPR Int. Conf. On Pattern Recognition. Dipartimento di Fisica, Universita di Genova Conference B: Computer Vision and Image Processing Vol 2,1994. pp 334--338.
[18]
Vamplew, P., Recognition of Sign Language Using Neural Networks, Ph.D. Thesis, Department of Computer Science, University of Tasmania, 1996.
[19]
Vogler, C. and Metaxas, D., Toward Scalability in ASL Recognition: Breaking Down Signs into Fonemes. Gesture Workshop '99, Gif-sur-Yvette, France, March 17-19, 1999.
[20]
Waleed, M., Grasp: Recognition of Australian sign language using instrumented gloves. Master thesis, University of New South Wales, Oct. 1995
[21]
W. Sandler, The Spreading hand autosegment of American Sign Language. Sign Language Studies 50: 1--28. 1986.
[22]
Zhao, D., Fang, G., Wen G., Large vocabulary sign language recognition based on hierarchical decision trees. In Intl. Conf. on Multimodal Interfaces, Vancouver, BC, November 2003.

Cited By

View all
  • (2019)Tradeoffs in the Efficient Detection of Sign Language Content in Video Sharing SitesACM Transactions on Accessible Computing10.1145/332586312:2(1-16)Online publication date: 11-Jun-2019
  • (2019)Crawling and classification strategies for generating a multi-language corpus of sign language videoProceedings of the 18th Joint Conference on Digital Libraries10.1109/JCDL.2019.00023(97-106)Online publication date: 2-Jun-2019
  • (2018)Comparing Visual, Textual, and Multimodal Features for Detecting Sign Language in Video Sharing Sites2018 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR)10.1109/MIPR.2018.00010(7-12)Online publication date: Apr-2018
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICMI '05: Proceedings of the 7th international conference on Multimodal interfaces
October 2005
344 pages
ISBN:1595930280
DOI:10.1145/1088463
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 October 2005

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. ASL translation
  2. gestural interfaces
  3. gesture recognition

Qualifiers

  • Article

Conference

ICMI05
Sponsor:

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2019)Tradeoffs in the Efficient Detection of Sign Language Content in Video Sharing SitesACM Transactions on Accessible Computing10.1145/332586312:2(1-16)Online publication date: 11-Jun-2019
  • (2019)Crawling and classification strategies for generating a multi-language corpus of sign language videoProceedings of the 18th Joint Conference on Digital Libraries10.1109/JCDL.2019.00023(97-106)Online publication date: 2-Jun-2019
  • (2018)Comparing Visual, Textual, and Multimodal Features for Detecting Sign Language in Video Sharing Sites2018 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR)10.1109/MIPR.2018.00010(7-12)Online publication date: Apr-2018
  • (2014)Identifying Sign Language Videos in Video Sharing SitesACM Transactions on Accessible Computing10.1145/25796985:4(1-14)Online publication date: 1-Mar-2014
  • (2012)Design and evaluation of classifier for identifying sign language videos in video sharing sitesProceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility10.1145/2384916.2384950(191-198)Online publication date: 22-Oct-2012
  • (2012)Network-Level Power-Performance Trade-Off in Wearable Activity RecognitionACM Transactions on Embedded Computing Systems10.1145/2345770.234578111:3(1-30)Online publication date: 1-Sep-2012
  • (2010)Analyzing grasping for inferring cognitive states of users2010 IEEE International Conference on Acoustics, Speech and Signal Processing10.1109/ICASSP.2010.5495795(2230-2233)Online publication date: Mar-2010
  • (2008)Activity recognition from on-body sensorsProceedings of the 5th European conference on Wireless sensor networks10.5555/1786014.1786017(17-33)Online publication date: 30-Jan-2008
  • (2008)Activity Recognition from On-Body Sensors: Accuracy-Power Trade-Off by Dynamic Sensor SelectionWireless Sensor Networks10.1007/978-3-540-77690-1_2(17-33)Online publication date: 2008
  • (2007)Simple glove-based Korean finger spelling recognition systemProceedings of the 2007 international conference on Computational science and its applications - Volume Part I10.5555/1802834.1802937(1063-1073)Online publication date: 26-Aug-2007
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media