Abstract
The general goal of our research is the creation of a natural and intuitive interface for input and recognition of American Sign Language (ASL) math signs. The specific objective of this work is the development of two new interfaces for the Mathsignertm application. Mathsignertm is an interactive, 3D animation-based game designed to increase the mathematical skills of deaf children. The program makes use of standard input devices such as mouse and keyboard. In this paper we show a significant extension of the application by proposing two new user interfaces: (1) a glove-based interface, and (2) an interface based on the use of a specialized keyboard. So far, the interfaces allow for real-time input and recognition of the ASL numbers zero to twenty.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Holt, J.A., Traxler, C., Allen, T.: Interpreting the scores: A user’s guide to the 9th edition stanford achievement test for educators of deaf and hard-of-hearing students. Technical report, Gallaudet Research Institute, Washington, DC (1997)
Caccamise, F., Lang, H.: Signs for Science and Mathematics: A Resource Book for Teachers and Students. Rochester, NY, National Technical Institute for the Deaf, RIT (1996)
Adamo-Villani, N., Doublestein, J., Martin, Z.: The mathsigner: An interactive learning tool for american sign language k-3 mathematics. In: IEEE Proceedings of IV04 - 8th International Conference on Information Visualization, pp. 713–716. IEEE Computer Society Press, Los Alamitos (2004)
Corporation, I.: 3d interaction (2001), http://www.immersion.com/3d/products/cyber_glove.php
Adamo-Villani, N., Beni, G.: Keyboard encoding of hand gestures. In: Proceedings of HCI International - 10th International Conference on Human-Computer Interaction, vol. 2, pp. 571–575 (2003)
Adamo-Villani, N., Beni, G.: Reconfigurable keyboard for gesture control. In: HCI International-11th International Conference on Human-Computer Interaction, on a CD-ROM (2005)
Frishberg, N., Corazzo, S., Day, L., Wilcox, S., Schulmeister, R.: Sign language interfaces. In: Proc. of INTERCHI 1993, Amsterdam, The Netherlands, pp. 194–197 (1993)
Huang, T., Pavlovic, V.: Hand gesture modeling, analysis, and synthesis. In: Proceedings of the International Workshop on Automatic Face and Gesture Recognition, Zurich (1995)
Geer, D.: Will gesture-recognition technology point the way? Computer 37, 20–23 (2004)
Yi, B., Harris Jr., F.C., Wang, L., Yan, Y.: Real-time natural hand gestures. Computing in Science and Engineering 7, 92–96 (2005)
Sturman, D.J.: Whole-Hand Input. PhD thesis, Massachusetts Institute of Technology, Cambridge, MA, USA (1992)
Sturman, D.J., Zeltzer, D.: A survey of glove-based input. IEEE Comput. Graph. Appl. 14, 30–39 (1994)
Culver, V.R.: A hybrid sign language recognition system. In: IEEE Proceedings of the 8th International Symposium on Wearable Computers ISWC 2004, vol. 00, pp. 30–33. IEEE Computer Society, Los Alamitos (2004)
Vamplew, P.: Recognition of sign language gestures using neural networks. In: Proc. of 1st Euro. Conf. Disability, Virtual Reality Assoc. Tech., Maidenhead, UK, pp. 27–33 (1996)
Starner, T., Pentland, A.: Real-time american sign language recognition from video using hidden markov models. Technical Report MIT TR-375, Media Lab, MIT (1996)
Rabiner, L.: A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE 77(1), 257–286 (1989)
Avilés-Arriaga, H., Sucar, L.E.: Dynamic bayesian networks for visual recognition of dynamic gestures. Journal of Intelligent and Fuzzy Systems 12, 243–250 (2002)
Flodin, M.: Signing illustrated: the complete learning guide. The Berkley Publishing Group, New York (1994)
Hernandez-Rebollar, J.L., Lindeman, R.W., Kyriakopoulos, N.: A multi-class pattern recognition system for practical finger spelling translation. In: Proc. of IEEE 4th Int’l Conference on Multimodal Interfaces (ICMI 2002), pp. 185–190 (2002)
Nissen, S.: Fast artificial neural network library (2000), http://leenissen.dk/fann/
Hernandez-Rebollar, J.L., Lindeman, R.W., Kyriakopoulos, N.: The acceleglove: a whole hand input device for virtual reality. In: Proc. of ACM Siggraph 2002 - 29th Int’l Conference on Computer Graphics and Interactive Techniques - Sketches and Applications, p. 259 (2002)
Kuroda, T., Tabata, Y., Goto, A., Ikuta, H., Murakami, M.: Consumer price data-glove for sign language recognition. In: Proc. of 5th Intl Conf. Disability, Virtual Reality Assoc. Tech., Oxford, UK, pp. 253–258 (2004)
Dawnsign: Numbering in American Sign Language: Number Signs for Everyone. DawnSign Press (1998)
Ong, S., Ranganath, S.: Automatic sign language analysis: A survey and the future beyond lexical meaning. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 873–891 (2005)
Su, M.C.: A fuzzy rule-based approach to spatio-temporal hand gesture recognition. IEEE Trans. Systems, Man, and Cybernetics, Part C: Application Rev. 30, 276–281 (2000)
Adamo-Villani, N., Carpenter, E., Arns, L.: An immersive virtual environment for learning sign language mathematics. In: ACM Proceedings of Siggraph 2006 - 33rd International Conference on Computer Graphics and Interactive Techniques - Educators, ACM Siggraph, New York (2006)
Valli, C., Lucas, C.: Linguistics of American Sign Language: a Resource Text for ASL Users. Gallaudet University Press, Washington (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Adamo-Villani, N., Beneš, B., Brisbin, M., Hyland, B. (2006). A Natural Interface for Sign Language Mathematics. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2006. Lecture Notes in Computer Science, vol 4291. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11919476_8
Download citation
DOI: https://doi.org/10.1007/11919476_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-48628-2
Online ISBN: 978-3-540-48631-2
eBook Packages: Computer ScienceComputer Science (R0)