Skip to main content
SpringerLink
Log in

A knowledge-based sign synthesis architecture

  • Long Paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

This paper presents the modules that comprise a knowledge-based sign synthesis architecture for Greek sign language (GSL). Such systems combine natural language (NL) knowledge, machine translation (MT) techniques and avatar technology in order to allow for dynamic generation of sign utterances. The NL knowledge of the system consists of a sign lexicon and a set of GSL structure rules, and is exploited in the context of typical natural language processing (NLP) procedures, which involve syntactic parsing of linguistic input as well as structure and lexicon mapping according to standard MT practices. The coding on linguistic strings which are relevant to GSL provide instructions for the motion of a virtual signer that performs the corresponding signing sequences. Dynamic synthesis of GSL linguistic units is achieved by mapping written Greek structures to GSL, based on a computational grammar of GSL and a lexicon that contains lemmas coded as features of GSL phonology. This approach allows for robust conversion of written Greek to GSL, which is an essential prerequisite for access to e-content by the community of native GSL signers. The developed system is sublanguage oriented and performs satisfactorily as regards its linguistic coverage, allowing for easy extensibility to other language domains. However, its overall performance is subject to current well known MT limitations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Blaxxun Contact 5. http://www.blaxxun.com/home/index.php? option=com_content&task=view&id=42&Itemid=85

  2. Boutsis S., Prokopidis P., Giouli V., Piperidis S.: A robust parser for unrestricted Greek text. In: Proceedings of the 2nd International Conference on Language Resources and Evaluation, LREC 2000, Athens, pp. 467–473 (2000)

  3. Carpenter B.: The Logic of Typed Feature Structures. Cambridge University Press, London (1992)

    MATH  Google Scholar 

  4. Chen, Y., Gao, W., Fang, G., Wang, Z., Yang, C., Jiang, D.: Text to Avatar in Multi-modal Human Computer Interface. In: Proceedings of Asia-Pacific CHI (APCHI2002) (2002)

  5. De Rosis, F., Pelachaud, C., Poggi, I., Carofiglio, V., De Carolis, B.: From Greta’s mind to her face: modelling the dynamics of affective states in a conversational embodied agent. Int. J. Hum. Comput. Stud. 59 (1–2), 81–118 (2003)

    Article  Google Scholar 

  6. Efthimiou, E.: Processing cumulative morphology information in GSL: the case of pronominal reference in a three-dimensional morphological system. In honor of G. Babiniotis, National and Kapodestrian University of Athens (2006, in press)

  7. Efthimiou, E., Fotinea, S.-E., Sapountzaki, G.: Processing linguistic data for GSL structure representation. In: Proceedings of the Workshop on the Representation and Processing of Sign Languages: Lexicographic matters and didactic scenarios, Satellite Workshop to LREC-2006, Genoa, pp.49–54 (2006)

  8. Efthimiou E., Sapountzaki G., Karpouzis K., Fotinea S.-E.: Developing an e-learning platform for the Greek sign language. In: Miesenberger K., Klaus J., Zagler W., Burger D. (eds) Lecture Notes in Computer Science, vol. 3118, 1107–1113. Springer, Heidelberg (2004)

    Google Scholar 

  9. Efthimiou, E., Vacalopoulou, A., Fotinea, S.-Ε., Steinhauer, G.: Multipurpose design and creation of GSL dictionaries. In: Proceedings of the Workshop on the Representation and Processing of Sign Languages “From SignWriting to Image Processing. Information techniques and their implications for teaching, documentation and communication”, Satellite Workshop to LREC-2004 Conference, Lisbon, pp. 51–58 (2004)

  10. Efthimiou, E., Katsoyannou, M.: Research issues on GSL: a study of vocabulary and lexicon creation (in Greek). In: Studies in Greek Linguistics, Computational Linguistics, vol. 2, pp. 42–50, Aristotle University of Thessaloniki, Thessaloniki (2001)

  11. Elliott, R., Glauert, J.R.W., Kennaway, J.R.: A framework for non-manual gestures in a synthetic signing system. In: Keates, S., Clarkson, P.J., Langdon, P., Robinson, P. (eds) Designing a More Inclusive World, pp. 127–136, Springer (2004)

  12. Elliott, R., Glauert, J.R.W., Kennaway, J.R., Marshall, I.: Development of language processing support for the ViSiCAST project. In: Proceedings ASSETS 2000, 4th International ACM SIGCAPH Conference on Assistive Technologies, Washington DC (2000)

  13. Fotinea, S.-E., Efthimiou, E., Karpouzis, K., Caridakis, G.: Dynamic GSL synthesis to support access to e-content. In: Proceedings of the 3rd International Conference on Universal Access in Human-Computer Interaction (UAHCI 2005), Las Vegas, Nevada, USA (2005)

  14. Fotinea, S.-E., Efthimiou, E., Kouremenos, D.: Generating linguistic content for Greek to GSL conversion. In: Proceedings of the HERCMA-2005 Conference (The 7th Hellenic European Conference on Computer Mathematics and its Applications), Athens (2005)

  15. Fischer, S.D., Gough, B.: Verbs in American sign language. Sign Lang Stud 18, 17–48 (1978)

    Google Scholar 

  16. Foulds, R.: Biomechanical and perceptual constraints on the bandwidth requirements of sign language. IEEE Trans Neural Syst Rehab Eng 12, 65–72 (2004)

    Article  Google Scholar 

  17. Huang, Z., Eliens, A., Visser, C.: STEP: A scripting language for embodied agents. In: Proceedings of the Workshop on Lifelike Animated Agents (2002)

  18. EuclideanSpace http://www.euclideanspace.com/maths/algebra/vectors/lookat/index.htm

  19. Information technology—Computer graphics and image processing—Humanoid animation (H-Anim) International Standard (ISO/IEC FCD 19774—Humanoid animation (H-Anim)):http://www.h-anim.org/Specifications/H-Anim200x/ISO_IEC_FCD_19774/

  20. Hanke, T.: HamNoSys in a sign language generation context. In: Schulmeister, R., Reinitzer, H. (eds) Progress in sign language research. In honor of Siegmund Prillwitz / Fortschritte in der Gebärdensprachforschung. Festschrift für Siegmund Prillwitz. (International Studies on Sign Language and Communication of the Deaf; 40) Hamburg, 249–264 (2002)

  21. Karpouzis, K., Caridakis, G., Fotinea, S.-E., Efthimiou, E.: Educational resources and implementation of a Greek sign language synthesis architecture. In: Computers & Education, Web3D technologies in learning, education and training, vol. 49(1), pp. 54–74, Elsevier, Amsterdam (2007)

  22. Karpouzis, K., Raouzaiou, A., Drosopoulos, A., Ioannou, S., Balomenos, T., Tsapatsoulis, N., Kollias, S.: Facial expression and gesture analysis for emotionally-rich man-machine interaction. In: Sarris, N., Strintzis, M. (eds) 3D Modeling and Animation: Synthesis and Analysis Techniques, Idea Group, Toronto, pp. 175–200 (2004)

  23. Kennaway, R.: Experience with, and requirements for, a gesture description language for synthetic animation. In: Proceedings of the 5th International Workshop on Gesture and Sign Language based Human-Computer Interaction, Genova (2003)

  24. Kennaway, R.: Synthetic animation of deaf signing gestures. In: Proceedings of the International Gesture Workshop, London (2001)

  25. Koskenniemi, K.: Finite-state parsing and disambiguation. In: Proceedings of the 13th International Conference on Computational Linguistics (ACL), Helsinki, pp. 229–232 (1990)

  26. Lengyel, E.: Mathematics for 3D Game Programming and Computer Graphics, 2nd edn. Charles River Media, Hingham Massachusetts (2003)

  27. Lenseigne, B., Dalle, P.: A computational model for sign language grammar. In: 2nd Language and Technology Conference: Human Language Technologies as a Challenge for Computer Sciences and Linguistics, Poland (2005)

  28. Padden, C.: Interaction of Morphology and Syntax in American Sign Language. Garland, New York (1988)

  29. Preda, M., Prêteux, F.: Critic review on MPEG-4 face and body animation. In: Proceedings IEEE International Conference on Image Processing (ICIP 2002), Rochester, pp. 505–508 (2002)

  30. Prillwitz, S., Leven, R., Zienert, H., Hanke, T., Henning, J.: HamNoSys. Version 2.0. Hamburg Notation System for Sign Language. An Introductory Guide. Broschur (ISBN 3-927731-01-3) Signum Verlag, Seedorf (1989)

  31. Sapountzaki, G., Efthimiou, E., Fotinea, S.-E.: Digital technology in Greek sign language teaching in primary school curriculum (in Greek). In: Proceedings of Hellenic Conference on Digital Educational Material: Issues of creation, exploitation and evaluation, Volos, 6–7 April 2006 (in press)

  32. Sapountzaki, G., Efthimiou, E., Fotinea, S.-E.: e-Education in the Greek Sign Language using an Internet based platform (in Greek). In: Proceedings of Workshop: e-Learning using ICT, Lamia, 26–28 May 2006 (in press)

  33. Shieber, S.M.: Constraint-Based Grammar Formalisms. MIT Press, Cambridge (1992)

    Google Scholar 

  34. Stephanidis, C.: Editorial. Int J Universal Access Inf Soc 1, 1–3 (2001)

    Google Scholar 

  35. Stephanidis, C., Savidis, A.: Universal access in the information society: methods, tools, and interaction technologies. Int J Universal Access Inf Soc 1, 40–55 (2001)

    Google Scholar 

  36. Stokoe, W., Kuschel, R.: A Field Guide for Sign Language Research. Linstock Press, Maryland (1979)

  37. Suszczanska, N., Szmal, P., Francik, J.: Translating Polish texts into sign language in the TGT system. In: Proceedings of 20th IASTED International Multi-Conference Applied Informatics, Innsbruck, pp 282–287 (2002)

  38. Sutton-Spence R., Woll B.: The linguistics of BSL—an introduction. Cambridge University Press, London (1999)

    Google Scholar 

Download references

Acknowledgments

The authors wish to acknowledge the assistance of all groups involved in the implementation of the work presented here, especially the GSL research group of ILSP and the IVMS lab. This work was partially funded by the national project grants SYNNENOESE (GSRT: eLearning 44).

Author information

Authors and Affiliations

  1. Institute for Language and Speech Processing, Artemidos 6 & Epidavrou, Athens, Greece

    Stavroula-Evita Fotinea & Eleni Efthimiou

  2. Image, Video and Multimedia Systems Lab, 9, Iroon Polytechniou St, Athens, Greece

    George Caridakis & Kostas Karpouzis

Authors
  1. Stavroula-Evita Fotinea
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eleni Efthimiou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. George Caridakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kostas Karpouzis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stavroula-Evita Fotinea.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fotinea, SE., Efthimiou, E., Caridakis, G. et al. A knowledge-based sign synthesis architecture. Univ Access Inf Soc 6, 405–418 (2008). https://doi.org/10.1007/s10209-007-0094-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-007-0094-8

Keywords

Search

Navigation