Robyn Moncrief
ABSTRACT
A promising method for increasing Deaf accessibility is the technology of sign language synthesis, which can be used to automate the translation of spoken or written language to sign language through the manipulation of an avatar. Efforts to automatically translate spoken language to sign have lagged behind spoken-to-spoken translation. Avatars are used in various capacities for sign language display, including translation and educational tools. Though the ability of avatars to portray acceptable sign language producing believable human-like motion has improved in recent years, many still lack the naturalness and supporting motions of human motion. This paper presents current efforts being made at the DePaul University ASL Lab to improve the methods of sign synthesis using avatar technology and create believable human-like motion.
- W. Stokoe, “Sign language structure,” Studies in Linguistics, Occasional Papers 8, Buffalo, New York, (1960)Google Scholar
- A. J. Thomson and A. V. Martinet, A practical English grammar, Oxford University Press, 1980.Google Scholar
- C. L. Baker-Shenk and D. Cokely, American Sign Language: A teacher's resource text on grammar and culture, Gallaudet University Press, 1991.Google Scholar
- T. N. Kluwin, “A rationale for modifying classroom signing systems,” Sign Language Studies, vol. 31, p. 179-187, Gallaudet University Press, 1981.Google Scholar
- Jerry Schnepp, Rosalee Wolfe, John McDonald and J. A. Toro, “Combining emotion and facial nonmanual signals in synthesized american sign language,” in Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility, 2012.Google ScholarDigital Library
- T. Johnston and L. De Beuzeville, “Auslan corpus annotation guidelines”, Auslan Corpus, 2016.Google Scholar
- C. A. Padden, Interaction of morphology and syntax in American Sign Language, Routledge, 2016.Google ScholarCross Ref
- Ruiliang Su, Chen Xiang, Shuai Cao, & Xu Zhang. (2016). Random Forest-Based Recognition of Isolated Sign Language Subwords Using Data from Accelerometers and Surface Electromyographic Sensors. Sensors. 16. 100. 10.3390/s16010100.Google Scholar
- Oscar Koller, Hermann Ney, & Richard Bowden. (2015). Deep Learning of Mouth Shapes for Sign Language. 10.1109/ICCVW.2015.69.Google Scholar
- Z. Cao, G. Hidalgo, T. Simon, S. -E. Wei and Y. Sheikh, "OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 1, pp. 172-186, 1 Jan. 2021, doi: 10.1109/TPAMI.2019.2929257.Google ScholarDigital Library
- John McDonald, Rosalee Wolfe, Robyn Moncrief, Souad Baowidan, & Jerry Schnepp, Kinematic Model for Constructed Dialog in American Sign Language. 6th Conference of the International Society for Gesture Studies, San Diego, CA, July 8-11, 2014.Google Scholar
- Mohamed-El-Fatah Benchiheub, Bastien Berret, & Annelies Braffort,. (2016). Collecting and Analysing a Motion-Capture Corpus of French Sign Language.Google Scholar
- Zhe Cao, Tomas Simon, Shih-En Wei, Yaser Sheikh; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 7291-7299Google Scholar
- Chawla, Nitesh V., "SMOTE: synthetic minority over-sampling technique." Journal of artificial intelligence research 16 (2002): 321-357.Google ScholarDigital Library
Index Terms
- Efforts to Improve Avatar Technology for Sign Language Synthesis
Recommendations
Collecting and evaluating the CUNY ASL corpus for research on American Sign Language animation
Automated synthesis of American Sign Language (ASL) animations will benefit people who are deaf with low English literacy.Annotated ASL motion-capture corpora enable researchers to produce animations with complex spatial and linguistic phenomena.We ...
A parametric approach to sign language synthesis
Assets '05: Proceedings of the 7th international ACM SIGACCESS conference on Computers and accessibilityIn this paper we discuss the progress made toward accurate synthesis of signs in American Sign Language (ASL) using a finite number of descriptive parameters. A sign editor produces elements of a sign inventory that can be used with a commercially ...
Joint Optimization of Word Alignment and Epenthesis Generation for Chinese to Taiwanese Sign Synthesis
This work proposes a novel approach to translate Chinese to Taiwanese sign language and to synthesize sign videos. An aligned bilingual corpus of Chinese and Taiwanese Sign Language (TSL) with linguistic and signing information is also presented for ...
Comments