Abstract
Communication using sign language (SL) provides alternative means for information transmission among the deaf. Automated gesture recognition involved in SL, however, could further expand this communication channel to the world of hearers. In this study, data from five-channel surface electromyogram and three-dimensional accelerometer from signers’ dominant hand were subjected to a feature extraction process. The latter consisted of sample entropy (SampEn)-based analysis, whereas time-frequency feature (TFF) analysis was also performed as a baseline method for the automated recognition of 60-word lexicon Greek SL (GSL) isolated signs. Experimental results have shown a 66 and 92% mean classification accuracy threshold using TFF and SampEn, respectively. These results justify the superiority of SampEn against conventional methods, such as TFF, to provide with high recognition hit-ratios, combined with feature vector dimension reduction, toward a fast and reliable automated GSL gesture recognition.
Similar content being viewed by others
References
Addison PS (2002) The illustrated wavelet transform handbook: introductory theory and applications in science, engineering, medicine and finance. Institute of Physics (IoP) Publishing, Bristol BS1 6BE, UK
Boostani R, Moradi MH (2003) Evaluation of the forearm EMG signal features for the control of a prosthetic hand. Physiol Meas 24:309–319
Brashear H, Starner T, Lukowicz P, Junker H (2005) Using multiple sensors for mobile sign language recognition. In: Proceedings of IEEE 7th International Symposium on Wearable Computers, pp 45–52
Charayaphan C, Marble A (1992) Image processing system for interpreting motion in American Sign Language. J Biomed Eng 14:419–425
Chen Q, Georganas ND, Petriu EM (2008) Hand gesture recognition using Haar-like features and a stochastic context-free grammar. IEEE Trans Instrum Meas 57(8):1562–1571
Cooper H, Bowden R (2007) Large lexicon detection of sign language. In: Lee et al (eds) Human computer interaction, ICCV2007, LNCS 4796. Springer Verlag, Berlin, Heidelberg, pp 88–97
Eckmann JP, Ruelle D (1985) Ergodic theory of chaos and strange attractors. Rev Modern Phys 57:617–654
Fang G, Gao W, Zhao D (2007) Large-vocabulary continuous sign language recognition based on transition-movement models. IEEE Trans Syst Man Cybern 37(1):1–9
Fels SS (1994) Glove-TalkII: mapping hand gestures to speech using neural networks-an approach to building adaptive interfaces. Ph.D. dissertation, Computer Science Department, University of Toronto, Canada
Fels SS, Hinton G (1993) GloveTalk: a neural network interface between a DataDlove and a speech synthesizer. IEEE Trans Neural Netw 4:2–8
Freeman WT, Roth M (1995) Orientation histograms for hand gesture recognition. In: IEEE proceedings of international workshop on automatic face and gesture recognition, pp 296–301
Graupe J, Cine KK (1975) Functional separation of EMG signals via ARMA identification method for prosthesis control purpose. IEEE Trans Syst Man Cybern SMC-5, pp 252–259
Hernandez-Rebollar J, Kyriakopoulos N, Lindeman R (2004) A new instrumented approach for translating American Sign Language into sound and text. Proceedings of IEEE 6th international conference on automatic face and gesture recognition, pp 547–552
Hong P, Turk M, Huang TS (2000) Gesture modeling and recognition using finite state machines. Proceedings of 4th IEEE international conference on automatic face and gesture recognition, pp 410–415
Hongyu X. (2003) Mahalanobis distance based ARTMAP networks. Master Thesis, Dept. Computer Science, San Diego State Univ., California, USA
Kadous MW (1996) Machine recognition of Auslan signed using PowerGlove: towards large-lexicon recognition of sign language. Proceedings of WIGLS, pp 165–174
Kim JH, Kim DG, Shin JH, Lee SW, Hong KS (2005) Hand gesture recognition system using fuzzy algorithm and RDBMS for Post PC. In: Wang L, Jin Y (eds) FSKD 2005 LNCS (LNAI), vol 3614. Springer, Heidelberg, pp 170–175
Kosmidou VE, Hadjileontiadis LJ, Panas SM (2006) Evaluation of surface EMG features for the recognition of American Sign Language gestures. Proceedings of IEEE 28th annual international conference on EMBS 6197–6200
Kosmidou VE, Hadjileontiadis LJ (2007) ICT-based biofeedback in everyday applications for people with disabilities. Proceedings of the 4th personalized health conference
Lee S, Henderson V, Brashear H, Starner T, Hamilton S (2005) User-centered development of a gesture-based american sign language game. Proceedings of instructional technology and education of the deaf symposium. NTID, Rochester, NY
Liang RH, Ouhyoung M (1996) A sign language recognition system using hidden markov model and context sensitive search. Proceedings of ACM VRST’96, pp 59–66
Liang RH, Ouhyoung M (1998) A real-time continuous gesture recognition system for sign language. Proceedings of the 3rd international conference on automatic face and gesture recognition, pp 558–565
Lichtenauer JF, Hendriks EA, Reinders MJT (2008) Sign language recognition by combining statistical DTW and independent classification. IEEE Trans Pattern Anal Mach Intell 30(11):2040–2046
Ma J, Gao W, Wu J, Wang C (2000) A continuous Chinese sign language recognition system. In: Proceedings of IEEE 4th international conference of automatic face and gesture recognition, pp 428–433
Mitra S, Acharya T (2007) Gesture recognition: a survey. IEEE Trans Syst Man Cybern C 37(3):311–323
Murakami K, Taguchi H (1991) Gesture recognition using recurrent neural networks. In: Proceedings of SIGCHI conference on human factors in computing systems, pp 237–242
Needleman SB, Wunsch CD (1970) A general method applicable to the search for similarities in the amino acid sequence of two proteins. J Mol Biol 48(3):443–453
Ng CW, Ranganath S (2002) Real-time gesture recognition system and application. J Image Vision Comput 20(13–14):993–1007
Paschaloudi VN, Margaritis KG (2003) Towards an assistive tool for Greek sign language communication. In: Proceedings of IEEE 3th international conference of advanced learning technologies, pp 125–129
Pincus SM (1991) Approximate entropy as a measure of system complexity. Proc Natl Acad Sci USA 88:2297–2301
Richman JS, Moorman JR (2000) Physiological time-series analysis using approximate entropy and sample entropy. Am J Physiol Heart Circ Physiol 278:H2039–H2049
Seber GAF (2004) Multivariate observations (Wiley Series in probability and statistics). Wiley, San Francisco, CA
Shaw R (1981) Strange attractors, chaotic behavior, and information flow. Z Naturforsch 36A:80–112
Starner T (1995) Visual recognition of american sign language using hidden Markov models. Master thesis, MIT Media Laboratory, Cambridge, MA, USA
Starner T, Pentland A (1996) Real-time American sign language recognition from video using hidden Markov Model. Perceptual Computing Section, Technical Report No. 375, MIT Media Lab Cambridge MA, USA
Tabachnick B, Fidell L (2001) Using multivariate statistics, 4 th edn. Allyn and Bacon, Boston (chap 11)
Takahashi T, Kishino F (1991) Gesture coding based on experiments using a hand gesture interface device. ACM SIGCHI Bull 23(2):67–73
Vogler C, Metaxas D (1997) Adapting hidden Markov models for ASL recognition by using three-dimensional computer vision methods. In: Proceedings of IEEE international conference on systems, man and cybernetics, pp 156–161
Vogler C, Metaxas D (1998) ASL recognition based on a coupling between HMMs and 3D motion analysis. Proc of IEEE Inter Conf on Computer Vision 363-369
Vogler C, Metaxas D (1999) Toward scalability in ASL recognition: breaking down signs into phonemes. In: Proceedings of gesture workshop. http://gri.gallaudet.edu/~cvogler/research/ data/cvdm-gw99.pdf
Vonesch C, Blu T, Unser M (2007) Generalized Daubechies wavelet families. IEEE Trans Signal Process 55(9):4415–4429
Waldron MB (1995) Isolated ASL sign recognition system for deaf persons. IEEE Trans on Rehabilitation Eng 3(3):261–271
Wang SB, Quattoni A, Morency LP, Demirdjian D (2006) Hidden conditional random fields for gesture recognition. Proc IEEE Comp Soc Conf Comput Vis Pattern Recognit 2:1521–1527
Wexelblat AD (1993) A feature-based approach to continuous gesture analysis. Master Thesis, MIT, Media Laboratory, Cambridge, MA, USA
Wong SF, Cipolla R (2005) Real-time interpretation of hand motions using a sparse Bayesian classifier on motion gradient orientation images. Br Mach Vis Conf 1:379–388
Yang MS, Ahuja N (1998) Recognizing hand gesture using motion trajectories. Proc IEEE CS Conf Comput Vis Pattern Recognit 1:466–472
Acknowledgments
The authors would like to thank the Greek Sign Language teachers, Mr. Grigoris Petropoulos, Mr. Thanasis Germanidis, and Mrs. Maria Christoforidou for performing the GSL gestures during the acquisition process. The authors are grateful to Mrs. Anastasia Gouvatzi, Director of the Special School for the Deaf, Thessaloniki, for her contribution to the construction of the GSL vocabulary and her help in the data acquisition experiments. The authors would like to thank the General Secretariat for Research and Technology (GSRT), Greek Ministry of Development, for its financial support under the Grant No. 44 (22195/16 Dec. 2005) within the 3rd Community Support Programme-Operational Programme “Information Society 2000-2008 A3-M3.3” (Funding: 75% European Regional Development Fund-25% National Fund).
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Kosmidou, V.E., Hadjileontiadis, L.I. Using sample entropy for automated sign language recognition on sEMG and accelerometer data. Med Biol Eng Comput 48, 255–267 (2010). https://doi.org/10.1007/s11517-009-0557-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11517-009-0557-6