Skip to main content
SpringerLink
Log in

An efficient gesture based humanoid learning using wavelet descriptor and MFCC techniques

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Recognizing any gesture, pre-processing and feature extraction are the two major issues which we have solved by proposing a novel concept of Indian Sign Language (ISL) gesture recognition in which a combination of wavelet descriptor (WD) and Mel Sec Frequency Cepstral Coefficients (MFCC) feature extraction technique have been used. This combination is very effective against noise reduction and extraction of invariant features. Here we used WD for reducing dimensionality of the data and moment invariant point extraction of hand gestures. After that MFCC is used for finding the spectral envelope of an image frame. This spectral envelope quality is useful for recognizing hand gestures in complex environment by eliminating darkness present in each gesture. These feature vectors are then used for classifying a probe gestures using support vector machine (SVM) and K nearest neighbour classifiers. Performance of our proposed methodology has been tested on in house ISL datasets as well as on Sheffield Kinect gesture dataset. From experimental results we observed that WD with MFCC method provides high recognition rate as compare to other existing techniques [MFCC, orientation histogram (OH)]. Subsequently, ISL gestures have been transferred to a Humanoid HOAP-2 (humanoid open architecture platform) robot in Webots simulation platform. Then these gestures are imitated by HOAP-2 robot exactly in a same manner.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  1. Mavridis N (2014) A review of verbal and non-verbal human–robot interactive communication. Robot Auton Syst. Available online 13 October 2014, ISSN 0921-8890. doi:10.1016/j.robot.2014.09.031. http://www.sciencedirect.com/science/article/pii/S0921889014002164

  2. Böhme H-J, Wilhelm T, Key J, Schauer C, Schröter C, Groß H-M, Hempel T (2003) An approach to multi-modal human–machine interaction for intelligent service robots. Robot Auton Syst 44(1):83–96, ISSN 0921-8890, doi:10.1016/S0921-8890(03)00012-5

  3. Ltd Fujitsu Automation Co. HOAP-2 instruction manual (2004) http://robita.iiita.ac.in/hoap2instruction03e.pdf

  4. Wagner P, Malisz Z, Kopp S (2014) Gesture and speech in interaction: an overview. Speech Comm 57:209–232, ISSN 0167-6393

  5. Starner T, Pentland A (1995) Real-rime American sign language recognition from video using hidden markov models. In: International symposium on computer vision, pp 265–270

  6. Martin J, Crowley JL (1997) An appearance-based approach to gesture recognition. In: Proceedings of the 9th international conference on image analysis and processing, pp 340–347

  7. Kadous W (1995) Recognition of Australian sign language using instrumented gloves. ‘Bachelor’s thesis’, University of New South Wales

  8. Sturman DJ (1992) Whole-hand input. Ph. D dissertation, Massachusetts Institute of Technology, 1992

  9. Watson R (1993) A survey of gesture recognition techniques. Technical Report TCD-CS-93-11, Department of Computer Science, Trinity College Dublin

  10. Freeman WT, Roth M (1995) Orientation histograms for hand gesture recognition. IEEE international workshop on automatic face and gesture recognition

  11. Nandy A, Prasad JS, Chakraborty P, Nandi GC (2010) Recognizing and interpreting Indian Sign Language gesture for human robot interaction. In: International conference on computer and communication technology (ICCCT 2010), and 17–19 Sept. 2010

  12. Singha J, Das K (2013) Hand gesture recognition based on Karhunen-Loeve transform. In: International conference on mobile and embedded technology, pp 365–371

  13. Gupta S, Jaafar J, Ahmad WFW, Bansal A (2013) Feature extraction using MFCC. Signal Image Process 4(4):101

    Google Scholar 

  14. Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Int J Data Mining Knowl Discov (Springer) 2(2):121–167

  15. Böhme H-J, Wilhelm T, Key J, Schauer C, Schröter C, Groß H-M, Hempel T (2003) An approach to multi-modal human–machine interaction for intelligent service robots. Robot Auton Syst 44(1):83–96, ISSN 0921-8890. doi:10.1016/S0921-8890(03)00012-5

  16. Fahmy MMM (2010) Palmprint recognition based on Mel frequency Cepstral coefficients feature extraction. Ain Shams Eng J 1(1):39–47, ISSN 2090-4479. http://www.sciencedirect.com/science/article/pii/S209044791000067

  17. Tomkins W (1931) Indian Sign language. Vol. 92. Courier Corporation

  18. Lin C, Liu A (2010) A tutorial of wavelet transform. NTUEE, Taiwan

  19. Kristian S (1998) A tutorial on Daubechies wavelet transforms. February 19, 1998

  20. Han W, Chan CF, Choy CS, Pun KP (2006) An efficient MFCC extraction method in speech recognition. In: Proceedings of IEEE international symposium on circuits and systems (ISCAS), September 2006, pp 4

  21. Cunnigham P, Delany SJ (2007) K-nearest neighbour classifiers. Mult Classif Syst 1–17

  22. Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Int J Data Mining Knowl Discov (Springer) 2(2):121–167

  23. http://lshao.staff.shef.ac.uk/data/SheffieldKinectGesture.htm

  24. Liu L, Shao L (2013) Learning discriminative representations from RGB-D video data. In: Proceedings of international joint conference on artificial intelligence (IJCAI), Beijing, China, 2013

  25. Baranwal N, Singh N, Nandi GC (2014) Implementation of MFCC based hand gesture recognition on HOAP-2 using Webots platform. In: 2014 international conference on advances in computing, communications and informatics, ICACCI, pp 1897–1902. IEEE

  26. Nandy A, Mondal S, Prasad JS, Chakraborty P, Nandi GC (2010) Recognizing and interpreting indian sign language gesture for human robot interaction. In: 2010 international conference on computer and communication technology (ICCCT), pp 712–717, IEEE

  27. Bhalke DG, Rama Rao CB, Bormane DS (2015). Automatic musical instrument classification using fractional Fourier Transform based-MFCC features and counter propagation neural network. J Intell Inf Syst 1–22

  28. Bhalke DG, Ram Rao CB, Bormane DS (2014) Stringed instrument recognition using fractional fourier transform and linear discriminant analysis. In: International conference in issues and challenges in intelligent computing techniques, ICICT-2014, 7–8 Feb 2014

Download references

Acknowledgments

We would like to thank our robita lab scholar’s Avinash Kumar Singh and Anup Nandy. We would also thank Ms. Neha Singh M.Tech student and as well as thank all the research scholars of our robita lab of Indian Institute of Information Technology, Allahabad, for their comments and suggestions. We also thank our technical staff of robita lab for their help in data collection.

Author information

Authors and Affiliations

  1. Robotics and AI Lab, Indian Institute of Information Technology, Allahabad, India

    Neha Baranwal & G. C. Nandi

Authors
  1. Neha Baranwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G. C. Nandi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Neha Baranwal.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Baranwal, N., Nandi, G.C. An efficient gesture based humanoid learning using wavelet descriptor and MFCC techniques. Int. J. Mach. Learn. & Cyber. 8, 1369–1388 (2017). https://doi.org/10.1007/s13042-016-0512-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-016-0512-4

Keywords

Search

Navigation