Abstract
Signing avatars are becoming common and being published on the World Wide Web at an incredible rate. They are actually used in education to help deaf children and their parents to learn sign language thanks to many 3D signing avatars systems. In Tunisia, signing avatars have been used since several years as part of the WebSign project. During the last few years, thousands of 3D signing avatars have been recorded using WebSign and few other systems. One of the major challenges that we was facing is how to index and retrieve efficiently this huge quantity of 3D signed scenes. Indexing and cataloging these signed scenes is beyond the capabilities of current text-based search engines. In this paper, we describe a system that collects 3D signed scenes, processes and recognizes the signed words inside them. The processed scenes are then indexed for later retrieval. We use a novel approach for sign language recognition from 3D scenes based on the Longest Common Subsequence algorithm. Our system is able to recognize signs inside 3D scenes at a rate of 96.5 % using a 3D model dictionary. We present also a novel approach for search and results ranking based on the similarity rates between 3D models. Our method is more efficient than Hidden Markov Models in term of recognition time and in the case of co-articulated signs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Stokoe, W.C.: Sign Language Structure: An Outline of the Visual Communication Systems of the American Deaf. Studies in Linguistics: Occasional Papers (8) (1960)
Cuxac, C.: La langue des signes française, les voies de l’iconicité, Orphys (2000)
Klima, E.S., Bellugi, U.: The Signs of Language. Harvard University Press (1988)
Frishberg, N.: Arbitrariness and Iconicity: Historical Change in American Sign Language. Language 51(23) (1975)
Huenerfauth, M., Pengfei, L., Rosenberg, A.: Evaluating American Sign Language generation through the participation of native ASL signers. In: ACM SIGACCESS Conference on Computers and Accessibility, New York (2007)
Jemni, M., Elghoul, O.: A system to make signs using collaborative approach. In: Miesenberger, K., Klaus, J., Zagler, W.L., Karshmer, A.I. (eds.) ICCHP 2008. LNCS, vol. 5105, pp. 670–677. Springer, Heidelberg (2008)
El Ghoul, O., Jemni, M.: Multimedia courses generator for deaf children. The International Arab Journal for Information Technology 6(5) (2009)
Buttussi, F., Chittaro, L., Nadalutti, D.: H-Animator: A Visual Tool for Modeling, Reuse and Sharing of X3D Humanoid Animations. In: International Conference on 3D Web Technology, New York (2006)
Papadogiorgaki, M., Grammalidis, N., Sarris, N., Strintzis, M.G.: Synthesis of Virtual Reality Animations from SWML using MPEG-4 Body Animation Parameters. In: Sign Processing Workshop, Lisbon (2004)
Sangho, P., Jihun, P., Jake, K.A.: Video retrieval of human interactions using model-based motion tracking and multi-layer finite state automata. In: CIVR (2003)
Ben-Arie, J., Wang, Z., Pandit, P., Rajaram, S.: Human activity recognition using multidimensional indexing. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(8), 1091–1104 (2002)
Dejan, V., Saupe, D.: 3D model retrieval. In: Spring Conference on Computer Graphics, pp. 89–93 (May 2000)
Zhang, C., Chen, T.: Indexing and retrieval of 3D models aided by active learning. In: ACM International Conference on Multimedia, Ottawa (2001)
Akanksha, H.Z., Prabhakaran, B., Ruiz, J.: Reusing motions and models in animations. In: EGMM (2001)
Takahashi, T., Kishino, F.: Hand Gesture Coding Based On Experiments Using A Hand Gesture Interface Device. ACM SIGCHI Bulletin 23(2), 67–73 (1991)
Vogler, C., Metaxas, D.: ASL recognition based on a coupling between HMMs and 3D motion analysis. In: Sixth International Conference on Computer Vision (1998)
Dreuw, P., Neidle, C., Athitsos, V., Sclaroff, S., Ney, H.: Benchmark Databases for Video-Based Automatic Sign Language Recognition. In: International Conference on Language Resources and Evaluation, Morocco (2008)
Kelly, D., Delannoy, J.R., McDonald, J., Markham, C.: A Framework for Continuous Multimodal Sign Language Recognition. In: International Conference on Multimodal Interfaces (2009)
Brutzman, D., Daly, L.: X3D: extensible 3D graphics for Web authors. Morgan Kaufmann Publishers (2006)
Daly, L., Brutzman, D.: X3D: Extensible 3D Graphics for Web Authors. Morgan Kaufmann Publishers (2006)
Jaballah, K., Jemni, M.: Toward automatic Sign Language recognition from Web3D based scenes. In: ICCHP, Vienna (2010)
Benjamin, B., Daniel, A., Dietmar, S., Tobias, S., Dejan, V.: Feature-Based Similarity Search in 3D Object Databases. ACM Computing Surveys 27(4), 345–387 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jaballah, K., Jemni, M. (2013). Towards Indexing of Web3D Signing Avatars. In: Tucker, A., Höppner, F., Siebes, A., Swift, S. (eds) Advances in Intelligent Data Analysis XII. IDA 2013. Lecture Notes in Computer Science, vol 8207. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41398-8_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-41398-8_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-41397-1
Online ISBN: 978-3-642-41398-8
eBook Packages: Computer ScienceComputer Science (R0)