Abstract
A complete microcomputer system is described, GesRec3D, which facilitates the data acquisition, segmentation, learning, and recognition of 3-Dimensional arm gestures, with application as a Augmentative and Alternative Communication (AAC) aid for people with motor and speech disability. The gesture data is acquired from a Polhemus electro-magnetic tracker system, with sensors attached to the finger, wrist and elbow of one arm. Coded gestures are linked to user-defined text, to be spoken by a text-to-speech engine that is integrated into the system. A segmentation method and an algorithm for classification are presented that includes acceptance/rejection thresholds based on intra-class and inter-class dissimilarity measures. Results of recognition hits, confusion misses and rejection misses are given for two experiments, involving predefined and arbitrary 3D gestures.
This work was funded by grant A/P/0543 to University of Nottingham, School of Electrical and Electronic Engineering, from the UK medical research charity Action Research for the project “Improvement of assessment and the use of communication aids through the quantitative analysis of body movements of people with motor disabilities”.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Rubine, D.: Specifying Gestures by Example. Computer Graphics 25(4), 329–337 (1991)
Cairns, A.Y.: Towards the Automatic Recognition of Gesture. PhD Thesis, Uni. of Dundee (November 1993)
Progress in Gestural Interaction. In: Harling, P.A., Edwards, A.D.N. (eds.) Proc. GW 1996. Springer, Heidelberg (1997)
Pavlovic, V.I., Sharma, R., Huang, T.S.: Visual Interpretation of Hand Gestures for Human Computer Interaction: A Review. IEEE Trans. Pattern Analysis and Machine Intelligence 19(7), 677–695 (1997)
Nam, Y., Wohn, K.: Recognition of hand gestures with 3D, non-linear arm movement. Pattern Recognition Letters 18(1), 105–113 (1997)
Pausch, R., Williams, R.D.: Giving Candy to children: User-tailored input driving an articulator-based speech synthesizer. In: Edwards, A.D.N. (ed.) Extra-Ordinary Human-Computer Interaction: interfaces for people with disabilities. Cambridge Series on Human-Computer Interaction, vol. 7, pp. 169–182. Cambridge University Press, Cambridge (1995)
Fels, S.S., Hinton, G.E.: Glove-Talk II - A Neural Network Interface which maps Gestures to Parallel Formant Speech Synthesizer Controls. IEEE Trans. Neural Networks 8(5), 977–984 (1997)
Tew, A.I., Gray, C.J.: A real-time gesture recognizer based on dynamic programming. Journal of Biomedical Engineering 15, 181–187 (1993)
Keates, S., Perricos, C.: Gesture as a Means of Computer Access. Communication Matters 10(1), 17–19 (1996)
Craven, M.P., Curtis, K.M., Hayes-Gill, B.R., Thursfield, C.D.: A Hybrid Neural Network/ Rule-Based Technique for On-Line Gesture and Hand-Written Character Recognition. In: Proc. IEEE Fourth Intl. Conf. on Electronics, Circuits and Systems, Cairo, Egypt, Vol. 2, pp. 850–853 (1997)
Hofmann, F.G., Heyer, P., Hommel, G.: Velocity Profile Based Recognition of Dynamic Gestures with Discrete Hidden Markov Models. In: Wachsmuth, I., Fröhlich, M. (eds.) GW 1997. LNCS (LNAI), vol. 1371, pp. 81–95. Springer, Heidelberg (1998)
Howell, A.J., Buxton, H.: Gesture Recognition for Visually Mediated Interaction. In: Braffort, A., Gibet, S., Teil, D., Gherbi, R., Richardson, J. (eds.) GW 1999. LNCS (LNAI), vol. 1739, pp. 141–151. Springer, Heidelberg (2000)
3Space Fastrak User’s Manual. Rev. F, Polhemus Inc., Colchester, Vermont, USA (November 1993)
Gordon, A.D.: Classification. Monographs on Applied Probability and Statistics, ch. 2, p. 21. Chapman and Hall, New York (1981)
Milios, E., Petrakis, E.G.M.: Shape Retrieval Based on Dynamic Programming. IEEE Trans. Image Processing 9(1), 141–146 (2000)
Long Jr., A.C., Landay, J.A., Rowe, L.A., Michiels, J.: Visual Similarity of Pen Gestures. In: Proc. Human Factors in Computing, CHI 2000, pp. 360–367 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Craven, M.P., Curtis, K.M. (2004). GesRec3D: A Real-Time Coded Gesture-to-Speech System with Automatic Segmentation and Recognition Thresholding Using Dissimilarity Measures. In: Camurri, A., Volpe, G. (eds) Gesture-Based Communication in Human-Computer Interaction. GW 2003. Lecture Notes in Computer Science(), vol 2915. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24598-8_21
Download citation
DOI: https://doi.org/10.1007/978-3-540-24598-8_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21072-6
Online ISBN: 978-3-540-24598-8
eBook Packages: Springer Book Archive