Skip to main content

Advertisement

Springer Nature Link
Log in

Deep Leaning Based Static Indian-Gujarati Sign Language Gesture Recognition

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

In this research paper, deep learning based static sign language recognition system is developed for Gujarati (an Indian) language. Total population of Gujarat State is more than 6 crores which is more than the population of 193 out of 216 countries of the world (90%). 14% of the people of Gujarat are facing deaf–dumb disability. The aim of the research is to create a vision-based application for speech impaired people to communicate easily and meaningfully. Dataset for training and validation is created from scratch due to unavailability of standard dataset. To improve robustness of dataset, augmentation, and various environment conditions were considered. Dataset was included with and without background images in different lighting conditions. Skin color segmentation is used for improved feature extraction. Proposed CNN architecture is designed such that it has minimum possible parameters. All the parameters of CNN were selected in organized structural manner to reduce computational complexity. The robustness of CNN model was verified using tenfold cross validation. Proposed deep learning based model is compared with other state-of-the-art machine learning algorithm. Feature extraction capabilities of proposed model are compared with other well-known techniques like principal component analysis and auto-encoder. The results of the proposed network validate the robustness of the network. This is the first research work on Gujarati (an Indian) static sign language. We contributed digital dataset for Gujarati language for recognition and our system achieved overall 92.69% accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Data Availability

In this article, the database was created by the researchers and is presently unable to be made freely accessible. However, on request, author might make it accessible.

References

  1. N. R. C. Committee on Developments in the Science of Learning with additional material from the Committee on Learning Research and Educational Practice. How people learn: brain, mind, experience, and school: expanded edition. Washington, D.C.: National Academy Press; 2000.

  2. Pinto RF, Borges CDB, Almeida AMA, Paula IC. Static hand gesture recognition based on convolutional neural networks. J Electr Comput Eng. 2019;2019:4167890.

    Google Scholar 

  3. Pigou L, Dieleman S, Kindermans P-J, Schrauwen B. Sign language recognition using convolutional neural networks. In: European Conference on Computer Vision. 2014. p. 572–78.

  4. Nanivadekar PA, Kulkarni V. Indian sign language recognition: database creation, hand tracking and segmentation. In: 2014 International conference on circuits, systems, communication and information technology applications (CSCITA). 2014. p. 358–63.

  5. Pisharady PK, Saerbeck M. Recent methods and databases in vision-based hand gesture recognition: a review. Comput Vis Image Underst. 2015;141:152–65.

    Article  Google Scholar 

  6. Bank TW. World development indicators database—population 2020 [Online].

  7. Jain S, Raja KS, Mukerjee M-PA. Indian sign language character recognition. Indian Institute of Technology, Kanpur Course Project-CS365A. 2016.

  8. Cohen MW, Zikri NB, Velkovich A. Recognition of continuous sign language alphabet using leap motion controller. In: 2018 11th international conference on human system interaction (HSI). 2018. p. 193–99.

  9. Riofrío S, Pozo D, Rosero J, Vásquez J. Gesture recognition using dynamic time warping and kinect: a practical approach. In: 2017 international conference on information systems and computer science (INCISCOS). 2017. p. 302–08.

  10. Sruthi C, Lijiya A. Signet: a deep learning based Indian sign language recognition system. In: 2019 International conference on communication and signal processing (ICCSP). 2019. p. 0596–600.

  11. Lokhande P, Prajapati R, Pansare S. Data gloves for sign language recognition system. Int J Comput Appl. 2015;975:8887.

    Google Scholar 

  12. Chouhan T, Panse A, Voona AK, Sameer S. Smart glove with gesture recognition ability for the hearing and speech impaired. In: 2014 IEEE global humanitarian technology conference-south Asia satellite (GHTC-SAS). 2014. p. 105–10.

  13. Abhishek KS, Qubeley LCF, Ho D. Glove-based hand gesture recognition sign language translator using capacitive touch sensor. In: 2016 IEEE international conference on electron devices and solid-state circuits (EDSSC). 2016. p. 334–37.

  14. Das A, Yadav L, Singhal M, Sachan R, Goyal H, Taparia K et al. Smart glove for sign language communications. In: 2016 international conference on accessibility to digital world (ICADW). 2016. p. 27–31.

  15. Su R, Chen X, Cao S, Zhang X. Random forest-based recognition of isolated sign language subwords using data from accelerometers and surface electromyographic sensors. Sensors. 2016;16:100.

    Article  Google Scholar 

  16. Yang X, Chen X, Cao X, Wei S, Zhang X. Chinese sign language recognition based on an optimized tree-structure framework. IEEE J Biomed Health Inform. 2016;21:994–1004.

    Article  Google Scholar 

  17. Sajanraj T, Beena M. Indian sign language numeral recognition using region of interest convolutional neural network. In: 2018 second international conference on inventive communication and computational technologies (ICICCT). 2018. p. 636–40.

  18. Raheja J, Mishra A, Chaudhary A. Indian sign language recognition using SVM. Pattern Recognit Image Anal. 2016;26:434–41.

    Article  Google Scholar 

  19. Beena M, Namboodiri MA. ASL numerals recognition from depth maps using artificial neural networks. Middle-East J Sci Res. 2017;25:1407–13.

    Google Scholar 

  20. Tripathi K, Baranwal N, Nandi GC. Continuous dynamic Indian sign language gesture recognition with invariant backgrounds. In: 2015 international conference on advances in computing, communications and informatics (ICACCI). 2015. p. 2211–16.

  21. Nguyen T-N, Huynh H-H, Meunier J. Static hand gesture recognition using principal component analysis combined with artificial neural network. J Autom Control Eng. 2015;3:40–5.

    Article  Google Scholar 

  22. Nagendraswamy H, Kumara BC, Chinmayi RL. Indian sign language recognition: an approach based on fuzzy-symbolic data. In: 2016 international conference on advances in computing, communications and informatics (ICACCI). 2016. p. 1006–13.

  23. Adithya V, Vinod P, Gopalakrishnan U. Artificial neural network based method for Indian sign language recognition. In: 2013 IEEE conference on information & communication technologies. 2013. p. 1080–85.

  24. Huang D-Y, Hu W-C, Chang S-H. Gabor filter-based hand-pose angle estimation for hand gesture recognition under varying illumination. Expert Syst Appl. 2011;38:6031–42.

    Article  Google Scholar 

  25. Oyedotun OK, Khashman A. Deep learning in vision-based static hand gesture recognition. Neural Comput Appl. 2017;28:3941–51.

    Article  Google Scholar 

  26. Lilha H, Shivmurthy D. Evaluation of features for automated transcription of dual-handed sign language alphabets. In: 2011 international conference on image information processing. 2011. p. 1–5.

  27. Otiniano-Rodrıguez K, Cámara-Chávez G, Menotti D. Hu and Zernike moments for sign language recognition. In: Proceedings of international conference on image processing, computer vision, and pattern recognition. 2012. p. 1–5.

  28. Geetha M, Manjusha U. A vision based recognition of Indian sign language alphabets and numerals using b-spline approximation. Int J Comput Sci Eng. 2012;4:406.

    Google Scholar 

  29. Singha J, Das K. Indian sign language recognition using eigen value weighted Euclidean distance based classification technique. 2013. http://arxiv.org/abs/1303.0634.

  30. Hussain I, Talukdar AK, Sarma KK. Hand gesture recognition system with real-time palm tracking. In: 2014 Annual IEEE India conference (INDICON). 2014. p. 1–6.

  31. Sharma M, Pal R, Sahoo AK. Indian sign language recognition using neural networks and KNN classifiers. ARPN J Eng Appl Sci. 2014;9:1255–9.

    Google Scholar 

  32. Sharma K, Joshi G, Dutta M. Analysis of shape and orientation recognition capability of complex Zernike moments for signed gestures. In: 2015 2nd international conference on signal processing and integrated networks (SPIN). 2015. p. 730–35.

  33. Kumar A, Thankachan K, Dominic MM. Sign language recognition. In: 2016 3rd international conference on recent advances in information technology (RAIT). 2016. p. 422–28.

  34. Patil SB, Sinha G. Distinctive feature extraction for Indian sign language (ISL) gesture using scale invariant feature transform (SIFT). J Inst Eng (India) Ser B. 2017;98:19–26.

    Article  Google Scholar 

  35. Loke P, Paranjpe J, Bhabal S, Kanere K. Indian sign language converter system using an android app. In: 2017 international conference of electronics, communication and aerospace technology (ICECA). 2017. p. 436–39.

  36. Kumar DA, Sastry A, Kishore P, Kumar EK. Indian sign language recognition using graph matching on 3D motion captured signs. Multimed Tools Appl. 2018;77:32063–91.

    Article  Google Scholar 

  37. Kishore P, Kumar DA, Sastry ACS, Kumar EK. Motionlets matching with adaptive kernels for 3-d Indian sign language recognition. IEEE Sens J. 2018;18:3327–37.

    Article  Google Scholar 

  38. Pariwat T, Seresangtakul P. Thai finger-spelling sign language recognition employing PHOG and local features with KNN. Int J Adv Soft Comput Appl. 2019;11:94–107.

    Google Scholar 

  39. Pariwat T, Seresangtakul P. Thai finger-spelling sign language recognition using global and local features with SVM. In: 2017 9th international conference on knowledge and smart technology (KST). 2017. p. 116–20.

  40. Bheda V, Radpour D. Using deep convolutional networks for gesture recognition in American sign language. 2017. http://arxiv.org/abs/1710.06836.

  41. Wadhawan A, Kumar P. Deep learning-based sign language recognition system for static signs. Neural Comput Appl. 2020;32:7957–68.

    Article  Google Scholar 

  42. Rao GA, Syamala K, Kishore P, Sastry A. Deep convolutional neural networks for sign language recognition. In: 2018 conference on signal processing and communication engineering systems (SPACES). 2018. p. 194–97.

  43. Huang J, Zhou W, Li H, Li W. Sign language recognition using 3d convolutional neural networks. In: 2015 IEEE international conference on multimedia and expo (ICME). 2015. p. 1–6.

  44. Huang J, Zhou W, Li H, Li W. Sign language recognition using real-sense. In: 2015 IEEE China summit and international conference on signal and information processing (ChinaSIP). 2015. p. 166–70.

  45. Nagi J, Ducatelle F, Di Caro GA, Cireşan D, Meier U, Giusti A et al. Max-pooling convolutional neural networks for vision-based hand gesture recognition. In: 2011 IEEE international conference on signal and image processing applications (ICSIPA). 2011. p. 342–47.

  46. Rioux-Maldague L, Giguere P. Sign language fingerspelling classification from depth and color images using a deep belief network. In: 2014 Canadian conference on computer and robot vision. 2014. p. 92–7.

  47. Molchanov P, Gupta S, Kim K, Pulli K. Multi-sensor system for driver's hand-gesture recognition. In: 2015 11th IEEE international conference and workshops on automatic face and gesture recognition (FG). 2015. p. 1–8.

  48. Tushar AK, Ashiquzzaman A, Islam MR. Faster convergence and reduction of overfitting in numerical hand sign recognition using DCNN. In: 2017 IEEE region 10 humanitarian technology conference (R10-HTC). 2017. p. 638–41.

  49. Tang A, Lu K, Wang Y, Huang J, Li H. A real-time hand posture recognition system using deep neural networks. ACM Trans Intell Syst Technol (TIST). 2015;6:1–23.

    Article  Google Scholar 

  50. Yang S, Zhu Q. Video-based Chinese sign language recognition using convolutional neural network. In: 2017 IEEE 9th international conference on communication software and networks (ICCSN). 2017. p. 929–34.

  51. Koller O, Zargaran S, Ney H, Bowden R. Deep sign: enabling robust statistical continuous sign language recognition via hybrid CNN-HMMs. Int J Comput Vis. 2018;126:1311–25.

    Article  Google Scholar 

  52. Kumar EK, Kishore P, Kumar MTK, Kumar DA. 3D sign language recognition with joint distance and angular coded color topographical descriptor on a 2–stream CNN. Neurocomputing. 2020;372:40–54.

    Article  Google Scholar 

  53. Kolkur S, Kalbande D, Shimpi P, Bapat C, Jatakia J. Human skin detection using RGB, HSV and YCbCr color models. 2017. http://arxiv.org/abs/1708.02694.

  54. Xu G, Xiao Y, Xie S, Zhu S. Face detection based on skin color segmentation and AdaBoost algorithm. In: 2017 IEEE 2nd advanced information technology, electronic and automation control conference (IAEAC). 2017. p. 1756–60.

  55. Khaled SM, Islam MS, Rabbani MG, Tabassum MR, Gias AU, Kamal MM et al. Combinatorial color space models for skin detection in sub-continental human images. In: International visual informatics conference. 2009. p. 532–42.

  56. Gonzalez RC, Woods RE, Masters BR. Digital image processing third edition. J Biomed Opt. 2008;14: 029901.

    Article  Google Scholar 

  57. Islalm MS, Rahman MM, Rahman MH, Arifuzzaman M, Sassi R, Aktaruzzaman M. Recognition Bangla sign language using convolutional neural network. In: 2019 international conference on innovation and intelligence for informatics, computing, and technologies (3ICT). 2019. p. 1–6.

Download references

Author information

Authors and Affiliations

  1. Gujarat Technological University, Nr. Vishwakarma Government Engineering College, Nr. Visat Three Roads, Visat-Gandhinagar Highway, Chandkheda, Ahmedabad, Gujarat, 382424, India

    Dhaval U. Patel

  2. Electronics and Communication Engineering Department, L. E. College (Diploma), Near ITI-Morbi, Ghuntu Road, Morbi, Gujarat, 363642, India

    Dhaval U. Patel

  3. Electronics and Communication Engineering Department, Bhagwan Mahavir Polytechnic, BMEF Campus VIP Road, Bharthana Road, Vesu, Surat, Gujarat, 395007, India

    Jay M. Joshi

Authors
  1. Dhaval U. Patel
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jay M. Joshi
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Dhaval U. Patel.

Ethics declarations

Conflicts of interest

The authors have no conflicts of interest to declare. All co-authors have seen and agree with the contents of the manuscript and there is no financial interest to report.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Patel, D.U., Joshi, J.M. Deep Leaning Based Static Indian-Gujarati Sign Language Gesture Recognition. SN COMPUT. SCI. 3, 380 (2022). https://doi.org/10.1007/s42979-022-01254-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-022-01254-2

Keywords