Skip to main content
SpringerLink
Log in

Finger spelling recognition using depth information and support vector machine

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In Sign Language fingerspelling systems, letters of the alphabet are presented by a diverse form or/and movement of the fingers. In this study, the presented work focus on developing a real-time translation framework of static fingerspelling alphabets. At first an adaptive k-means based cluster method for depth segmentation is proposed, where a flexible cluster number n is used instead of the pre-defined definitive one. Based on the segmentation step, a recognition framework using intensity and depth information is proposed and compared with some distinctive works. Discriminative features extracted from Histogram of Oriented Gradients (HOG), Local Binary Patterns (LBP), and Zernike moments are used due to their simplicity and good performance. The experiments are executed on a public fingerspelling dataset, which consisted of 120,000 images representing 24 alphabet letters over five different users. The results show that the presented framework is efficient, easy implementation, and performs better than the compared approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Alassaf N, Alkazemi B, Gutub A (2017) Applicable light-weight cryptography to secure medical data in IoT systems. Journal of Research in Engineering and Applied Sciences (JREAS) 2(2):50–58

    Google Scholar 

  2. Alharthi N, Gutub A (2017) Data visualization to explore improving decision-making within hajj services. Scientific Modelling and Research 2(1):9–18. https://doi.org/10.20448/808.2.1.9.18

    Article  Google Scholar 

  3. Al-Otaibi NA, Gutub AA (2014) 2-Leyer security system for hiding sensitive text data on personal computers. LNIT 2(2):151–157

    Google Scholar 

  4. ASL Finger Spelling Dataset (2017) http://personal.ee.surrey.ac.uk/Personal/N.Pugeault/index.php?section=FingerSpellingDataset. Accessed on 12 Feb. 2017

  5. Badi H (2016) A survey on recent vision-based gesture recognition. Intelligent Industrial Systems 2(2):179–191. https://doi.org/10.1007/s40903-016-0046-9

    Article  Google Scholar 

  6. Chang CC, Lin CJ (2001) LIBSVM: a library for support vector machines. Software available at: http://www.csie.ntu.edu.tw/~cjlin/libsvm. Accessed on 12 Feb 2017

  7. Dahmani D, Larabi S (2014) User-independent system for sign language finger spelling recognition. J Vis Commun Image Represent 25(5):1240–1250. https://doi.org/10.1016/j.jvcir.2013.12.019

    Article  Google Scholar 

  8. Dalal N, Triggs B Histograms of oriented gradients for human detection (2005) International Conference on Computer Vision & Pattern Recognition (CVPR'05), San Diego, CA, USA, 20-26 June 2005; IEEE Computer Society, vol. 2, pp. 886–893

  9. Gutub AA (2010) Pixel indicator technique for RGB image steganography. Journal of Emerging Technologies in Web Intelligence (JETWI) 2(1):56–64

    Google Scholar 

  10. Hrúz M, Trojanová J, Železný M (2012) Local binary pattern based features for sign language recognition. Pattern Recogn Image Anal 22(4):519–526. https://doi.org/10.1134/S1054661812040062

    Article  Google Scholar 

  11. Krishnaveni M, Radha DV (2011) Improved histogram based thresholding segmentation using PSO for sign language recognition. Int J Eng Sci Technol 3(2):1014–1020

    Google Scholar 

  12. Lun R, Zhao W (2015) A survey of applications and human motion recognition with Microsoft Kinect. Int J Pattern Recognit 29(5):1–49. https://doi.org/10.1142/S0218001415550083

    Article  Google Scholar 

  13. Ojala T, Pietikainen M, Maenpaa T (2002) Multi-resolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal 24(7):971–987. https://doi.org/10.1109/TPAMI.2002.1017623

    Article  MATH  Google Scholar 

  14. Otiniano-Rodriguez K, Cayllahua-Cahuina E et al (2015) Finger spelling recognition using kernel descriptors and depth images. 28th SIBGRAPI Conference on Graphics, Patterns and Images, Salvador, Bahia, Brazil, August 26–29, 2015. IEEE Computer Society, pp 72–79

  15. Otiniano-Rodriguez KC, Cáamara-Chávez G, Menotti D Hu and Zernike moments for sign language recognition (2012) The 2012 International conference on image processing, computer vision, and pattern recognition (IPCV’12), Las Vegas, Nevada, USA, July 16–19 2012; IEEE Computer Society, pp. 1–5

  16. Pisharady PK, Saerbeck M (2015) Recent methods and databases in vision-based hand gesture recognition: a review. Comput Vis Image Underst 141:152–165. https://doi.org/10.1016/j.cviu.2015.08.004

    Article  Google Scholar 

  17. Prada F, Cruz L, Velho L Object extraction in RGBD Images (2012) 25th SIBGRAPI Conference on Graphics, Patterns and Images, Ouro Preto, Brazil, Aug. 22–25 2012; IEEE Computer Society

  18. Prada F, Cruz L, Velho L Improving object extraction with depth-based methods (2013) XXXIX Latin American Computing Conference (CLEI), Naiguata, Venezuela, 7–11 October 2013; IEEE, pp. 1–9

  19. Pugeault N, Bowden R Spelling it out: real-time ASL fingerspelling recognition (2011) Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Barcelona, Spain, 6–13 November 2011; IEEE, pp 1114–1119

  20. Rioux-Maldague L, Giguere P Sign language fingerspelling classification from depth and color images using a deep belief network (2014) Canadian Conference on Computer and Robot Vision (CRV), Montreal, QC, Canada, May 6–9, 2014; IEEE Computer Society, pp. 92–97

  21. Thippur A, Ek CH, Kjellstrom H Inferring hand pose: a comparative study of visual shape features (2013) 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Shanghai, China, April 22–26 2013; IEEE Computer Society, pp. 1–8

  22. Uebersax D, Gall J, den Bergh et al Real-time sign language letter and word recognition from depth data (2011) Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Barcelona, Spain, 6–13 November 2011. IEEE, pp 383–390

  23. Von Agris U, Zieren J, Canzler U et al (2008) Recent developments in visual sign language recognition. Univ Access Inf 6(4):323–362. https://doi.org/10.1007/s10209-007-0104-x

    Article  Google Scholar 

  24. Yang HD (2015) Sign language recognition with the kinect sensor based on conditional random fields. Sensors 15(1):135–147. https://doi.org/10.3390/s150100135.

    Article  Google Scholar 

  25. Zhu X, Wong K-YK Single-frame hand gesture recognition using color and depth kernel descriptors (2012) Proceedings of the 21st IEEE International Conference on Pattern Recognition (ICPR), Tsukuba Science City, Japan, November 11–15, 2012; IEEE, pp 2989–2992

Download references

Acknowledgments

This work is supported by Top-notch Academic Programs Project of Jiangsu Higher Education Institutions (TAPP).

Author information

Authors and Affiliations

  1. School of Software Engineering, Jinling Institute of Technology, Nanjing, China

    Yong Hu

Authors
  1. Yong Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yong Hu.

Ethics declarations

Conflicts of interest

As the author of this manuscript, I confirmed that there is no any possible conflict of interests in the manuscript. The mentioned funding in the “Acknowledgments” did not lead to any conflict of interests regarding the publication of this manuscript.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, Y. Finger spelling recognition using depth information and support vector machine. Multimed Tools Appl 77, 29043–29057 (2018). https://doi.org/10.1007/s11042-018-6102-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6102-6

Keywords

Search

Navigation