Skip to main content

Advertisement

Springer Nature Link
Log in

Bangla language modeling algorithm for automatic recognition of hand-sign-spelled Bangla sign language

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Because of using traditional hand-sign segmentation and classification algorithm, many diversities of Bangla language including joint-letters, dependent vowels etc. and representing 51 Bangla written characters by using only 36 hand-signs, continuous hand-sign-spelled Bangla sign language (BdSL) recognition is challenging. This paper presents a Bangla language modeling algorithm for automatic recognition of hand-sign-spelled Bangla sign language which consists of two phases. First phase is designed for hand-sign classification and the second phase is designed for Bangla language modeling algorithm (BLMA) for automatic recognition of hand-sign-spelled Bangla sign language. In first phase, we have proposed two step classifiers for hand-sign classification using normalized outer boundary vector (NOBV) and window-grid vector (WGV) by calculating maximum inter correlation coefficient (ICC) between test feature vector and pre-trained feature vectors. At first, the system classifies hand-signs using NOBV. If classification score does not satisfy specific threshold then another classifier based on WGV is used. The system is trained using 5,200 images and tested using another (5, 200 × 6) images of 52 hand-signs from 10 signers in 6 different challenging environments achieving mean accuracy of 95.83% for classification with the computational cost of 39.972 milliseconds per frame. In the Second Phase, we have proposed Bangla language modeling algorithm (BLMA) which discovers all “hidden characters” based on “recognized characters” from 52 hand-signs of BdSL to make any Bangla words, composite numerals and sentences in BdSL with no training, only based on the result of first phase. To the best of our knowledge, the proposed system is the first system in BdSL designed on automatic recognition of hand-sign-spelled BdSL for large lexicon. The system is tested for BLMA using hand-sign-spelled 500 words, 100 composite numerals and 80 sentences in BdSL achieving mean accuracy of 93.50%, 95.50% and 90.50% respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Sutton V. Introduction on deafness, sign language & sign-writing. See Signwritingorg Website, 2016

  2. WFD. Sign language. See World Federation of the Deaf Website, 2016

  3. Dey S. Bangladesh sign language day. See DPI-AP Website, 2014

  4. Jasim M, Zhang T, Hasanuzzaman M. A real-time computer vision-based static and dynamic hand gesture recognition system. International Journal on Image and Graphics, 2014, 14(01n02): 145006

    Article  Google Scholar 

  5. Majumder M, Hossain M, Mamtaz M, Ahmed M, Khan H, Ali M H, Khaled O, Iqbal M. Bengali Sign Language Dictionary. 2nd ed. Dhaka, Bangladesh: National Centre for Special Education and Ministry of Social Welfare in Cooperation with The Norwegian Association of The Deaf and Bangladersh National Federation of The Deaf, 1994

    Google Scholar 

  6. Begum S, Hasanuzzaman M. Computer vision-based bangladeshi sign language recognition system. In: Proceedings of the 12th International Conference on Computer and Information Technology. 2009, 414–419

  7. Rahaman M A, Jasim M, Zhang T, Ali M H, Hasanuzzaman M. A realtime hand-signs segmentation and classification system using fuzzy rule based RGB model and grid-pattern analysis. Frontiers of Computer Science, 2018, 12(6): 1258–1260

    Article  Google Scholar 

  8. Rahaman M A, Jasim M, Zhang T, Ali M H, Hasanuzzaman M. Realtime bengali and chinese numeral signs recognition using contour matching. In: Proceedings of the IEEE International Conference on Robotics and Biomimetics. 2015, 1215–1220

  9. Li J, Wang J, Ju Z. A novel hand gesture recognition based on highlevel features. International Journal of Humanoid Robotics, 2018, 15(2): 1750022

    Article  Google Scholar 

  10. Dong L, Liang Y, Kong G, Zhang Q, Cao X, Izquierdo E. Holons visual representation for image retrieval. IEEE Transactions on Multimedia, 2016, 18(4): 714–725

    Article  Google Scholar 

  11. Dong L, He L, Zhang Q. Discriminative light unsupervised learning network for image representation and classification. In: Proceedings of the 23rd ACM International Conference on Multimedia. 2015, 1235–1238

  12. Garcia-Ceja E, Brena R F. An improved three-stage classifier for activity recognition. International Journal of Pattern and Recognition and Artifical Intelligence, 2018, 32(1): 1860003

    Article  Google Scholar 

  13. Lee G, Mallipecldi R, Lee M. Trajectory-based vehicle tracking at low frame rates. Expert System with Application, 2017, 80: 46–57

    Article  Google Scholar 

  14. Mills M T, Bourbakis N G. A comparative survey on NLP/U methodologies for processing multi-documents. International Journal on Artificial Intelligence Tools, 2012, 21(4): 1250017

    Article  Google Scholar 

  15. Mills M, Psarologou A, Bourbakis N. Modeling natural language sentences into SPN graphs. In: Proceedings of the 25th IEEE International Conference on Tools with Artificial Intelligence. 2013, 889–896

  16. Santoni D, Pourabbas E. Automatic detection of words associations in texts based on joint distribution of words occurrences. Computational Intelligence, 2016, 32(4): 535–560

    Article  MathSciNet  Google Scholar 

  17. Lee G, Mallipeddi R, Lee M. Trajectory-based vehicle tracking at low frame rates. Expert Systems with Applications, 2017, 80: 46–57

    Article  Google Scholar 

  18. Liberman Y, Perry A. A visual tracking scheme for accurate object retrieval in low frame rate videos. International Journal on Artificial Intelligence Tools, 2016, 25(5): 1640003

    Article  Google Scholar 

  19. Dong L, Feng N, Mao M, He L, Wang J. E-grabcut: an economic method of iterative video object extraction. Frontiers of Computer Science, 2017, 11(4): 649–660

    Article  Google Scholar 

  20. Dong L, Feng N, Zhang Q. LSI: semantic label inference for nature image segmentation. Pattern Recognition, 2016, 59: 282–291

    Article  Google Scholar 

  21. Li C, Lin L, Zuo W, Wang W, Tang J. An approach to streaming video segmentation with sub-optimal low-rank decomposition. IEEE Transactions on Image Processing, 2016, 25(4): 1947–1960

    Article  MathSciNet  Google Scholar 

  22. Chen F S, Fu C M, Huang C L. Hand gesture recognition using a realtime tracking method and hidden markov models. Image Vision Computer, 2003, 21(8): 745–758

    Article  Google Scholar 

  23. Alon J, Athitsos V, Yuan Q, Sclaroff S. A unified framework for gesture recognition and spatiotemporal gesture segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(9): 1685–1699

    Article  Google Scholar 

  24. Asaari M S M, Rosdi B A, Suandi S A. Adaptive kalman filter incorporated eigenhand (akfie) for real-time hand tracking system. Multimedia Tools and Applications, 2015, 74(21): 9231–9257

    Article  Google Scholar 

  25. Khaled H, Sayed S G, Saad E S M, Ali H. Hand gesture recognition using modified 1 and background subtraction algorithms. Mathematical Problems in Engineering, 2015, (Article ID: 741068): 1–8

  26. Gurav R M, Kadbe P K. Vision based hand gesture recognition with haar classifier and adaboost algorithm. International Journal of Latest Trends in Engineering and Technology, 2015, 5(2): 155–160

    Google Scholar 

  27. Rahaman M A, Jasim M, Ali M H, Hasanuzzaman M. Realtime computer vision-based bengali sign language recognition. In: Proceedings of the 17th International Conference on Computer and Information Technology. 2014, 192–197

  28. Weng S K, Kuo C M, Tu S K. Video object tracking using adaptive kalman filter. Journal of Visual Communication and Image Representation, 2006, 17(6): 1190–1208

    Article  Google Scholar 

  29. Li N, Liu L, Xu D. Corner feature based object tracking using adaptive kalman filter. In: Proceedings of the 9th International Conference on Signal Processing. 2008, 1432–1435

  30. Luo Y, Celenk M. A new adaptive kalman filtering method for block-based motion estimation. In: Proceedings of the 15th International Conference on Systems, Signals and Image Processing. 2008, 89–92

  31. Karmokar B C, Alam K M R, Siddiquee M K. Bangladeshi sign language recognition employing neural network ensemble. International Journal of Computer Applications, 2012, 58(16): 43–46

    Article  Google Scholar 

  32. Jarman A M, Arshad S, Alam N, Islam M J. An automated bengali sign language recognition system based on fingertip finder algorithm. International Journal of Electronics and Informatics, 2015, 4(1): 1–10

    Google Scholar 

  33. Yasir F, Prasad P W C, Alsadoon A, Elchouemi A. Sift based approach on bangla sign language recognition. In: Proceedings of the 8th IEEE International Workshop on Computational Intelligence and Applications. 2015, 35–39

  34. Ayshee T F, Raka S A, Hasib Q R, Hossain M, Rahman R M. Fuzzy rule-based hand gesture recognition for bengali characters. In: Proceedings of the IEEE International Advance Computing Conference. 2014, 484–489

  35. Wang, Qiao Y, Tang X. Action recognition with trajectorypooled deepconvolutional descriptors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 4305–4314

  36. Feichtenhofer, Pinz A, Wildes R. Spatiotemporal residual networks for video action recognition. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. 2016, 3476–3484

  37. Asadi-Aghbolaghi M, Clapes A, Bellantonio M, Escalante H J, Ponce-López V, Baró X, Guyon I, Kasaei S, Escalera S. A survey on deep learning based approaches for action and gesture recognition in image sequences. In: Proceedings of the 12th IEEE Conference on Automatic Face and Gesture Recognition (FG 2017). 2017, 476–483

  38. Liu Z, Zhang C, Tian Y. 3D-based deep convolutional neural network for action recognition with depth sequences. Image and Vision Computing, 2016, 55(2): 93–100

    Article  Google Scholar 

  39. Varol G, Laptev I, Schmid C. Long-term temporal convolutions for action recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40(6): 1510–1517

    Article  Google Scholar 

  40. Zhu G, Zhang L, Mei L, Shao J, Song J, Shen P. Large-scale isolated gesture recognition using pyramidal 3D convolutional networks. In: Proceedings of the 23rd International Conference on Pattern Recognition. 2016, 19–24

  41. Wang P, Li W, Liu S, Gao Z, Tang C, Ogunbona P. Largescale isolated gesture recognition using convolutional neural networks. In: Proceedings of the 23rd International Conference on Pattern Recognition. 2016, 7–12

  42. Xu P. A real-time hand gesture recognition and humancomputer interaction system. 2017, arXiv preprint arXiv:1704.07296

  43. Rahaman M A, Jasim M, Ali M H, Hasanuzzaman M. Computer vision based bengali sign words recognition using contour analysis. In: Proceedings of the 18th International Conference on Computer and Information Technology. 2015, 335–340

  44. Park A, Yun S, Kim J, Min S, Jung K. Real-time visionbased korean finger spelling recognition system. International Journal of Computer, Electrical, Automation, Control and Information Engineering, 2008, 2(8): 2623–2628

    Google Scholar 

  45. Kane L, Khanna P. A framework for live and cross platform finger-spelling recognition using modified shape matrix variants on depth silhouettes. Computer Vision and Image Understanding, 2015, 141: 138–151

    Article  Google Scholar 

  46. Fang G, Gao W, Zhao D. Large-vocabulary continuous sign language recognition based on transition-movement models. IEEE Transactions on Systems, Man and Cybernetics, Part A, 2007, 37(1): 1–9

    Article  Google Scholar 

  47. Liwicki S, Everingham M. Automatic recognition of fingerspelled words in British sign language. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. 2009, 50–57

  48. Koller O, Forster V J, Ney V H. Continuous sign language recognition: towards large vocabulary statistical recognition systems handling multiple signers. Computer Vision and Image Understanding, 2015, 141: 108–125

    Article  Google Scholar 

  49. Rameshar S. Sadaran Vhasabiggan and Bangla Vhasa Dhaka, Bangladesh: Ananda Press, 1996

    Google Scholar 

  50. Mukhopaddhay A. Adorsho Bangla banan: ekti prostabona. See Galpersamay Website, 2017

  51. Kibria G. Bangla joint letter. See Daffodilvarsity.edu Website, 2013

  52. Chowdhuri M, Chowdhuri M H. Bangla Bhashar Bakoron (Grammar of Bengali language). 2nd ed. Dhaka, Bangladesh: Jatio Shikkhakromo Opatthopustok Board, 2014

    Google Scholar 

  53. Choudhury J. Bangla Banan Abhidhan. 3rd ed. Dhaka, Bangladesh: Bangla Academy, 2008

    Google Scholar 

  54. Ishaaque A. Samakalin Bangla Bhashar Abhidhan, 2nd ed. Dhaka, Bangladesh: Bangla Academy, 2003

    Google Scholar 

  55. CDD. Manual on Sign Supported Bangla. Dhaka, Bangladesh: Center for Disability in Development (CDD), 2002

    Google Scholar 

  56. Sourceforge.net. EmguCV. See Sourceforge Website, 2015

  57. Nagarajan S, Subashini T S. Static hand gesture recognition for sign language alphabets using edge oriented histogram and multi class SVM. International Journal of Computer Applications, 2013, 82(4): 28–35

    Article  Google Scholar 

Download references

Acknowledgements

This research was partially supported and funded by the Information and Communication Technology (ICT) Division, Ministry of Posts, Telecommunications and IT, Government of the People’s Republic of Bangladesh.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, University of Dhaka, Dhaka, 1000, Bangladesh

    Muhammad Aminur Rahaman, Mahmood Jasim, Md. Haider Ali & Md. Hasanuzzaman

Authors
  1. Muhammad Aminur Rahaman
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Mahmood Jasim
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Md. Haider Ali
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Md. Hasanuzzaman
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Muhammad Aminur Rahaman.

Additional information

Muhammad Aminur Rahaman received his BSc and MSc degree in Computer Science & Engineering from the Department of Computer Science & Engineering, Islamic University, Bangladesh in 2003 and 2004, respectively. He is a candidate for PhD degree under supervision of Prof. Dr. Md. Hasanuzzaman and Prof. Dr. Md. Haider Ali in the Department of Computer Science & Engineering, University of Dhaka, Bangladesh. He is a founder Director of Worldgaon (Pvt.) Limited which is a one of the famous software development company of Bangladesh. His current research interests including computer vision, sign language recognition and human-computer interaction. He is a member of IEEE.

Mahmood Jasim received his BSc and MSc degree in Computer Science & Engineering from the Department of Computer Science & Engineering, University of Dhaka, Bangladesh in 2011 and 2013, respectively. He joined the Department of Computer science and engineering as a lecturer in 2014. Currently he is pursuing his PhD degree in the University of Massachusetts Amherst. His current research interests include human-computer interaction, image processing, computer vision and artificial intelligence.

Md. Haider Ali received PhD degree from the Department of Electronics & Information Engineering, Toyohashi University of Technology, Japan in 2001. Prof. Ali has completed his Bachelor and Master degree from the Department of Applied Physics & Electronics (presently EEE) University of Dhaka 1984 and 1985 respectively. He is a professor of the Department of Computer Science & Engineering, University of Dhaka since June 2007. His current research interests include human face recognition and expression detection, post surgical expression simulation, soft-tissue deformation modeling, polygonal mesh simplification, narrow band video transmission/video conferencing, etc.

Md. Hasanuzzaman received PhD degree from the Department of Informatics, National Institute of Informatics (NII), The Graduate University for Advanced Studies, Japan. He graduated (with Honors) in 1993 from the Department of Applied Physics & Electronics, University of Dhaka, Bangladesh. He completed Masters of Science (MSc) in Computer Science in 1994 from the University of Dhaka. He joined as a Lecturer in the Department of Computer Science & Engineering, University of Dhaka, Bangladesh in 2000. Since March 2013, he has been serving as a professor in the Department of Computer Science & Engineering, University of Dhaka, Bangladesh. His current research interests include human-computer interaction, image processing, computer vision and artificial intelligence.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rahaman, M.A., Jasim, M., Ali, M.H. et al. Bangla language modeling algorithm for automatic recognition of hand-sign-spelled Bangla sign language. Front. Comput. Sci. 14, 143302 (2020). https://doi.org/10.1007/s11704-018-7253-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-018-7253-3

Keywords