Raima Islam
ABSTRACT
Deafness is one of the major health problems in Bangladesh, where almost 10% of the population falls under this category. The plight of deaf individuals to fit in with the mainstream population is significant. Unable to express themselves through sound, they have to rely on sign language to communicate.To create a more inclusive environment for this minority, we developed a model that recognizes Bengali characters and digits expressed using sign language in accordance with Bangla Sign Language (BdSL) guidelines. The model comprises two parts: a convolutional neural network that uses deep learning techniques, and an Extreme Gradient Boosting (XGBoost) classifier. Together, these components form the full architecture. The model’s execution was validated using the ‘Ishara-Lipi’ dataset, which is the first open-access digit and character dataset for BdSL. With the help of pre-processing techniques such as contrast-limited adaptive histogram equalization, using a more complex pre-trained CNN model like Inception-ResNet-v2, and optimizing the XGBoost model by using GridSearchCV, we achieved an accuracy of 86.67%, precision of 89%, recall and f1 score of 87%. Lastly, for digit recognition, we obtained an accuracy of 97.33%, precision of 98%, and recall and f1-score of 97%.
- Shahjalal Ahmed, Md Islam, Jahid Hassan, Minhaz Uddin Ahmed, Bilkis Jamal Ferdosi, Sanjay Saha, Md Shopon, 2019. Hand sign to Bangla speech: a deep learning in vision based system for recognizing hand sign digits and generating Bangla speech. arXiv preprint arXiv:1901.05613 (2019).Google Scholar
- Sunanda Das, Md Samir Imtiaz, Nieb Hasan Neom, Nazmul Siddique, and Hui Wang. 2023. A hybrid approach for Bangla sign language recognition using deep transfer learning model with random forest classifier. Expert Systems with Applications 213 (2023), 118914.Google ScholarDigital Library
- Jerome H Friedman. 2001. Greedy function approximation: a gradient boosting machine. Annals of statistics (2001), 1189–1232.Google Scholar
- Promila Haque, Badhon Das, and Nazmun Nahar Kaspy. 2019. Two-handed bangla sign language recognition using principal component analysis (PCA) and KNN algorithm. In 2019 International Conference on Electrical, Computer and Communication Engineering (ECCE). IEEE, 1–4.Google ScholarCross Ref
- Muttaki Hasan, Tanvir Hossain Sajib, and Mrinmoy Dey. 2016. A machine learning based approach for the detection and recognition of Bangla sign language. In 2016 International Conference on Medical Engineering, Health Informatics and Technology (MediTec). IEEE, 1–5.Google ScholarCross Ref
- Md Mehedi Hasan, Azmain Yakin Srizon, and Md Al Mehedi Hasan. 2020. Classification of Bengali sign language characters by applying a novel deep convolutional neural network. In 2020 IEEE Region 10 Symposium (TENSYMP). IEEE, 1303–1306.Google ScholarCross Ref
- Md Shafiqul Islalm, Md Moklesur Rahman, Md Hafizur Rahman, Md Arifuzzaman, Roberto Sassi, and Md Aktaruzzaman. 2019. Recognition bangla sign language using convolutional neural network. In 2019 international conference on innovation and intelligence for informatics, computing, and technologies (3ICT). IEEE, 1–6.Google ScholarCross Ref
- Md Sanzidul Islam, Sadia Sultana Sharmin Mousumi, Nazmul A Jessan, AKM Shahariar Azad Rabby, and Sayed Akhter Hossain. 2018. Ishara-lipi: The first complete multipurposeopen access dataset of isolated characters for bangla sign language. In 2018 International Conference on Bangla Speech and Language Processing (ICBSLP). IEEE, 1–4.Google ScholarCross Ref
- WW Kong and Surendra Ranganath. 2014. Towards subject independent continuous sign language recognition: A segment and merge approach. Pattern Recognition 47, 3 (2014), 1294–1308.Google ScholarDigital Library
- Wei-Meng Lee. 2021. Tuning the Hyperparameters of Your Machine Learning Model Using Grid Search CV. https://towardsdatascience.com/tuning-the-hyperparameters-of-your-machine-learning-model-using-gridsearchcv-7fc2bb76ff27.Google Scholar
- Jiaqing Liu, Kotaro Furusawa, Tomoko Tateyama, Yutaro Iwamoto, and Yen-wei Chen. 2019. An Improved Kinect-Based Real-Time Gesture Recognition Using Deep Convolutional Neural Networks for Touchless Visualization of Hepatic Anatomical Mode. Journal of Image and Graphics 7, 2 (2019), 45–49.Google ScholarCross Ref
- Mygel Andrei M Martija, Jakov Ivan S Dumbrique, and Prospero C Naval Jr. 2020. Underwater gesture recognition using classical computer vision and deep learning techniques. (2020).Google Scholar
- Md Humaion Kabir Mehedi, Ehteshamul Haque, Sameen Yasir Radin, Md Abrar Ur Rahman, Md Tanzim Reza, and Md. Golam Robiul Alam. 2022. Kidney Tumor Segmentation and Classification using Deep Neural Network on CT Images. In 2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA). 1–7. https://doi.org/10.1109/DICTA56598.2022.10034638Google ScholarCross Ref
- Trong-Nguyen Nguyen, Huu-Hung Huynh, and Jean Meunier. 2013. Static hand gesture recognition using artificial neural network. Journal of Image and Graphics 1, 1 (2013), 34–38.Google ScholarCross Ref
- Waseem Rawat and Zenghui Wang. 2017. Deep convolutional neural networks for image classification: A comprehensive review. Neural computation 29, 9 (2017), 2352–2449.Google Scholar
- Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A Alemi. 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI conference on artificial intelligence.Google ScholarDigital Library
- Jia Uddin, Fahmid Nasif Arko, Nujhat Tabassum, Taposhi Rabeya Trisha, and Fariha Ahmed. 2017. Bangla sign language interpretation using bag of features and Support Vector Machine. In 2017 3rd International Conference on Electrical Information and Communication Technology (EICT). IEEE, 1–4.Google ScholarCross Ref
- Jingzhong Wang, Xiaoqing Xu, and Meng Li. 2015. The study of gesture recognition based on SVM with LBP and PCA. Journal of Image and Graphics 3, 1 (2015), 16–19.Google ScholarCross Ref
- Haoyang Xu. 2020. The Problem with’Accurate’History: Complexity within Sallust’s Bellum Catilinae. Int’l J. Soc. Sci. Stud. 8 (2020), 81.Google ScholarCross Ref
Index Terms
- Optimization of Deep CNN-based Bangla Sign Language Recognition using XGBoost classifier
Recommendations
A hybrid approach for Bangla sign language recognition using deep transfer learning model with random forest classifier
AbstractSign language is the comprehensive medium of mass communication for hearing and speaking impaired individuals. As they cannot speak or hear, they are not able to use sound or vocal signals as an information medium for their communication. Rather, ...
Highlights- Demonstration of recent advancements in various Sign Language recognition research.
- Employment of proposed background elimination algorithm to remove unwanted features.
- Hybridization of transfer learning model with the Random ...
DeepNetDevanagari: a deep learning model for Devanagari ancient character recognition
AbstractDevanagari script is the most widely used script in India and other Asian countries. There is a rich collection of ancient Devanagari manuscripts, which is a wealth of knowledge. To make these manuscripts available to people, efforts are being ...
Handwritten Hangul recognition using deep convolutional neural networks
In spite of the advances in recognition technology, handwritten Hangul recognition (HHR) remains largely unsolved due to the presence of many confusing characters and excessive cursiveness in Hangul handwritings. Even the best existing recognizers do ...
Comments