Abstract
A desktop PC and wire communications net-based traditional studies on pattern recognition and multimodal interaction have some restrictions (e.g. limitation of motion, conditionality in space and so on) and general problems according to using of the vision technologies for recognition and representation of the haptic-gesture information. In this paper, we propose and implement Multi-Modal Recognition Interface (hereinafter, MMRI) integrating speech using Voice-XML and gesture based on wireless networks, it have purposes that recognizes and represents the Korean Standard Sign Language (hereinafter, KSSL) which is a dialog system and interactive elements in the Korean deaf communities, and the need to dialogue with deaf person in their own language, sign language, is well recognized and is widely accepted as being a positive influence on communication. The advantages of our approach are as follows: 1) it improves efficiency of the MMRI input module according to the technology of wireless communication, 2) it shows higher recognition performance than uni-modal recognition system 3) it recognizes and represents continuous sign language of users with flexibility in real time and offer to user a wider range of personalized and differentiated information using the MMRI more effectively. Experimental results, the MMRI deduces an average recognition rate of 96.23% for significant, dynamic and continuous the KSSL and speech of various users.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Stéphane, H.M., et al.: Multimodal Interaction Requirements. W3C Note (2003), http://www.w3.org
Barnett, J., et al.: Multimodal Interaction Activity-Multimodal Architecture and Interfaces. W3C Working Draft (2005), http://www.w3.org
Jang, H.-Y., Kim, D.-J., Kim, J.-B., Bien, Z.-N.: A Study on Hand-Signal Recognition System in 3-Dimensional Space. Journal of IEEK 2004-41CI-3-11. IEEK (2004)
Use of Signs in Hearing Communities, http://en.wikipedia.org/wiki/Sign_language
Kim, S.-G.: Standardization of Signed Korean. Journal of KSSE 9. KSSE (1992)
Kim, S.-G.: Korean Standard Sign Language Tutor, 1st edn. Osung Publishing Company, Seoul (2000)
5DT Data Glove 5 Manual and FASTRAK® Data Sheet, http://www.5dt.com
Kim, J.-H., Kim, D.-G., Shin, J.-H., Lee, S.-W., Hong, K.-S.: Hand Gesture Recognition System using Fuzzy Algorithm and RDBMS for Post PC. In: Wang, L., Jin, Y. (eds.) FSKD 2005. LNCS (LNAI), vol. 3614, Springer, Heidelberg (2005)
Chen, C.H.: Fuzzy Logic and Neural Network Handbook, 1st edn. McGraw-Hill, New York (1992)
McGlashan, S., et al.: Voice Extensible Markup Language (VoiceXML) Version 2.0. W3C Recommendation (1992), http://www.w3.org
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kim, JH., Hong, KS. (2006). Intelligent Multi-Modal Recognition Interface Using Voice-XML and Embedded KSSL Recognizer. In: Gabrys, B., Howlett, R.J., Jain, L.C. (eds) Knowledge-Based Intelligent Information and Engineering Systems. KES 2006. Lecture Notes in Computer Science(), vol 4251. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11892960_96
Download citation
DOI: https://doi.org/10.1007/11892960_96
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-46535-5
Online ISBN: 978-3-540-46536-2
eBook Packages: Computer ScienceComputer Science (R0)