Abstract
Sign language recognition and understanding are challenging tasks for many people who are not familiar with it, which limits communication between deaf-mute people and others. The system presented in this paper lowers the communication barrier, introducing an automatic translation layer that facilitates sign language understanding. The system uses a deep-learning model for sign language detection and a separate library for hand joint mapping. The application’s architecture was designed to allow users to access the system from desktop and mobile devices. The model’s results revealed an 82% accuracy, and after several tweaks on the activation function in our tests, we achieved perfect classification in our real word tests. The results of the system offered excellent accuracy, and its usability lowers the communication barrier between people, providing flexibility as the application is available for any device with a browser.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adeyanju, I.A., Bello, O.O., Adegboye, M.A.: Machine learning methods for sign language recognition: a critical review and analysis. Intell. Syst. Appl. 12, 200056 (2021)
Alzubaidi, L., et al.: Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. J. Big Data 8, 1–74 (2021)
Arruda, H., Silva, E.R., Lessa, M., Proença Jr., D., Bartholo, R.: Vosviewer and bibliometrix. J. Med. Libr. Assoc.: JMLA 110(3), 392 (2022)
Bantupalli, K., Xie, Y.: American sign language recognition using deep learning and computer vision. In: 2018 IEEE International Conference on Big Data (Big Data), pp. 4896–4899. IEEE (2018)
Han, J., Shao, L., Xu, D., Shotton, J.: Enhanced computer vision with Microsoft kinect sensor: a review. IEEE Trans. Cybern. 43(5), 1318–1334 (2013)
Larose, D.T., Larose, C.D.: k-nearest neighbor algorithm (2014)
Pang, B., Nijkamp, E., Wu, Y.N.: Deep learning with TensorFlow: a review. J. Educ. Behav. Stat. 45(2), 227–248 (2020)
Pathan, R.K., Biswas, M., Yasmin, S., Khandaker, M.U., Salman, M., Youssef, A.A.: Sign language recognition using the fusion of image and hand landmarks through multi-headed convolutional neural network. Sci. Rep. 13(1), 16975 (2023)
Rajasekhar, N., Yadav, M.G., Vedantam, C., Pellakuru, K., Navapete, C.: Sign language recognition using machine learning algorithm. In: 2023 International Conference on Sustainable Computing and Smart Systems (ICSCSS), pp. 303–306. IEEE (2023)
Rastgoo, R., Kiani, K., Escalera, S.: Sign language recognition: a deep survey. Expert Syst. Appl. 164, 113794 (2021)
Sherstinsky, A.: Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network. Phys. D 404, 132306 (2020)
Srivastava, S., Gangwar, A., Mishra, R., Singh, S.: Sign language recognition system using TensorFlow object detection API. In: Woungang, I., Dhurandher, S.K., Pattanaik, K.K., Verma, A., Verma, P. (eds.) ANTIC 2021. CCIS, vol. 1534, pp. 634–646. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-96040-7_48
Zhang, F., et al.: Mediapipe hands: on-device real-time hand tracking. arXiv preprint arXiv:2006.10214 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Costescu, C.M., Popescu, P.Ş., Mihăescu, M.C. (2025). Smart Sign Language Decoder. In: Julian, V., et al. Intelligent Data Engineering and Automated Learning – IDEAL 2024. IDEAL 2024. Lecture Notes in Computer Science, vol 15347. Springer, Cham. https://doi.org/10.1007/978-3-031-77738-7_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-77738-7_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-77737-0
Online ISBN: 978-3-031-77738-7
eBook Packages: Computer ScienceComputer Science (R0)