Abstract
Approximately 5% of the world’s population has hearing impairments and this number is expected to grow in the coming years due to demographic aging and the amount of noise we are exposed to. A significant fraction of this population has to endure severe impairments even since their childhood and sign languages are an effective mean of overcoming this barrier. Although sign languages are quite widespread among the deaf community, there are still situations in which the interaction with hearing people is difficult. This paper presents the sign language recognition module from an ongoing effort to develop a real-time Spanish sign language recognition system that could also work as a tutor. The proposed approach focuses on the definitions of the signs, first performing the classification of their constituents to end up recognizing full signs. Although the performance of the classification of the constituents can be quite weak, good user-independent sign recognition results are obtained.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Camgoz, N.C., Koller, O., Hadfield, S., Bowden, R.: Sign language transformers: joint end-to-end sign language recognition and translation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10023–10033 (2020)
Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., Sheikh, Y.: Openpose: realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans. Pattern Anal. Mach. Intell. 43(1), 172–186 (2019)
Carreira, J., Zisserman, A.: Quo vadis, action recognition? A new model and the kinetics dataset. In: proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6299–6308 (2017)
Elakkiya, R.: Machine learning based sign language recognition: a review and its research frontier. J. Ambient. Intell. Humaniz. Comput. 12(7), 7205–7224 (2021)
Gutierrez-Sigut, E., Costello, B., Baus, C., Carreiras, M.: LSE-sign: a lexical database for Spanish sign language. Behav. Res. Methods 48(1), 123–137 (2016)
Kratimenos, A., Pavlakos, G., Maragos, P.: Independent sign language recognition with 3d body, hands, and face reconstruction. In: ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4270–4274. IEEE (2021)
Liu, Z., Zhang, H., Chen, Z., Wang, Z., Ouyang, W.: Disentangling and unifying graph convolutions for skeleton-based action recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 143–152 (2020)
Lugaresi, C., et al.: MediaPipe: a framework for building perception pipelines. arXiv preprint arXiv:1906.08172 (2019)
Ma, Y., Zhou, G., Wang, S., Zhao, H., Jung, W.: SignFi: sign language recognition using WIFI. In: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 2(1), 1–21 (2018)
Pavlakos, G., et al.: Expressive body capture: 3D hands, face, and body from a single image. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10975–10985 (2019)
Rastgoo, R., Kiani, K., Escalera, S.: Hand sign language recognition using multi-view hand skeleton. Expert Syst. Appl. 150, 113336 (2020)
González, G.S., Sánchez, J.C., Díaz, M.M.B., Ata Pérez, A.: Recognition and classification of sign language for spanish. Computación y Sistemas 22(1), 271–277 (2018)
Sincan, O.M., Junior, J., Jacques, C.S., Escalera, S., Keles, H.Y.: Chalearn lap large scale signer independent isolated sign language recognition challenge: design, results and future research. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3472–3481 (2021)
Vazquez-Enriquez, M., Alba-Castro, J.L., Docio-Fernandez, L., Rodriguez-Banga, E.: Isolated sign language recognition with multi-scale spatial-temporal graph convolutional networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3462–3471 (2021)
Wadhawan, A., Kumar, P.: Sign language recognition systems: a decade systematic literature review. Arch. Comput. Meth. Eng. 28(3), 785–813 (2021)
Zhang, F., et al.: Mediapipe hands: On-device real-time hand tracking. arXiv preprint arXiv:2006.10214 (2020)
Acknowledgment
This work has been partially funded by the Basque Government, Spain, grant number IT900-16, and the Spanish Ministry of Science (MCIU), the State Research Agency (AEI), the European Regional Development Fund (FEDER), grant number RTI2018-093337-B-I00 (MCIU/AEI/FEDER, UE) and the Spanish Ministry of Science, Innovation and Universities (FPU18/04737 predoctoral grant). We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Rodríguez-Moreno, I., Martínez-Otzeta, J.M., Sierra, B. (2023). A Hierarchical Approach for Spanish Sign Language Recognition: From Weak Classification to Robust Recognition System. In: Arai, K. (eds) Intelligent Systems and Applications. IntelliSys 2022. Lecture Notes in Networks and Systems, vol 542. Springer, Cham. https://doi.org/10.1007/978-3-031-16072-1_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-16072-1_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16071-4
Online ISBN: 978-3-031-16072-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)