Abstract
This paper describes a wireless PDA-based Multi-Modal Interaction Architecture (hereinafter, MMIA) fusing 3 sensory channels, which are a detachable stylus that can be used on the touch screen by coupling embedded speech and KSSL recognizer. The MMIA fuses and recognizes 15 word-based instruction models that are represented by stylus (or touching), speech and KSSL (Korean Standard Sign Language), and then translates the recognition result into synthetic speech and visual illustration, for multi-modal PDA handling and interaction. The experimental results, each average recognition rate for the MMIA fusing 3 sensory channels was 95.73% in clean environments, and 93.29% was shown in noisy environments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Myers, B.A.: A brief history of human-computer interaction technology. Interactions 5(2), 44–54 (1998)
Jacko, J.A., et al.: Handbook for Human Computer Interaction. Lawrence Erlbaum & Associates, Mahwah (2003)
Wooldridge, M.J., Jennings, N.R.: Intelligent agents: Theory and practice, Know. Eng. Review 10(2), 115–152 (1995)
Buxton, B.: Multi-Touch Systems that I Have Known and Loved, Microsoft Research(2007)
M. Fuchs, P.: Architecture of Multi-modal Dialogue System. In: Sojka, P., Kopeček, I., Pala, K. (eds.) TSD 2000. LNCS (LNAI), vol. 1902, pp. 433–438. Springer, Heidelberg (2000)
Owen, R., et al.: When it Gets More Difficult, Use BothHhands - Exploring Bimanual Curve Manipulation. Proc. Graphics Interface, GI’05, 17–24 (2005)
Kim, S.-G.: Korean Standard Sign Language Tutor, 1st edn. Osung Publishing Company, Seoul (2000)
Kim, J.-H., et al.: Hand Gesture Recognition System using Fuzzy Algorithm and RDBMS for Post PC. In: Wang, L., Jin, Y. (eds.) FSKD 2005. LNCS (LNAI), vol. 3614, pp. 170–175. Springer, Heidelberg (2005)
Kim, J.-H., et al.: An Implementation of KSSL Recognizer for HCI Based on Post Wearable PC and Wireless Networks. In: Gabrys, B., Howlett, R.J., Jain, L.C. (eds.) KES 2006. LNCS (LNAI), vol. 4251, Springer, Heidelberg (2006)
kandasamy, W.B.V.: Smaranda Fuzzy Algebra. American Research Press, Seattle (2003)
Stein, J(Y): Digital Signal Processing: A Computer Science Perspective. John Wiley & Sons, Chichester (2000)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kim, JH., Youn, JS., Hong, KS. (2007). Toward a Novel Multi-modal HCI: Fusion Architecture Using Confidence Score and Fuzzy Value. In: Apolloni, B., Howlett, R.J., Jain, L. (eds) Knowledge-Based Intelligent Information and Engineering Systems. KES 2007. Lecture Notes in Computer Science(), vol 4692. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74819-9_31
Download citation
DOI: https://doi.org/10.1007/978-3-540-74819-9_31
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74817-5
Online ISBN: 978-3-540-74819-9
eBook Packages: Computer ScienceComputer Science (R0)