Analysis and Synthesis of the System for Processing of Sign Language Gestures and Translatation of Mimic Subcode in Communication with Deaf People

Abstract

In the paper a design and principle of operation of the system facilitating communication between hearing and deaf people is presented. The system has a modular architecture and consists of main application, translation server and two complementary databases. The main application is responsible for interaction with the user and visualization of the sign language gestures. The translation server carries out translation of the text written in the Polish language to the appropriate messages of the sign language. The translation server is composed of facts database and translation rules implemented in the Prolog language. The facts database contains the set of the lexemes and their inflected forms with a description of the semantics of units. The translation rules carry out identification and analysis of basic structures of the Polish language sentence. These basic structures are related to the sentence creation function of the verb predicate. On the basis of this analysis equivalent translation of text into the sign language is realized. Translated text in the form of metadata is passed to the main application, where it is translated into the appropriate gestures of the sign language and face mimicry. The gestures in the form of 3d vectors and face mimicry are stored in the main database as binary objects. The authors intend to apply the translation system in various public institutions like hospitals, clinics, post offices, schools and offices.