Abstract
This paper presents a study on under-actuation modelling applied to robotic hands aimed at sign language representation. Prior studies using a simulated TEO humanoid robot for representing sign language have shown positive comprehension and satisfaction responses among the deaf and hearing impaired community. The under-actuated mechanics of the robotic fingers were not contemplated in the simulated model, thus the correspondence problem arises as the previous joint space positions cannot be directly sent to the physical system. In addition to the 3:1 and 2:1 ratio of the under-actuation of the finger mechanisms, tendons and springs involve stiffness and elasticity that are difficult or unfeasible to model, and justify the need for a data-driven approach. Three motor command generators using three different neural network models are analysed and evaluated. Two of the generators are trained in a supervised fashion, and the third involves variational self-supervision and a transformation upon the latent space. The simulated joint space positions are translated into motor commands for the physical embodied robot to represent a sign language dactylology, which is in turn evaluated by deaf and hearing impaired end-users.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Using the tool https://github.com/HarisIqbal88/PlotNeuralNet.
References
Estevez, D., Victores, J.G., Fernandez-Fernandez, R., Balaguer, C.: Ironing, robotic, with 3D perception and force, torque feedback in household environments. In: 2017 IEEE/RSJ IROS International Conference on Intelligent Robots and Systems, Vancouver, BC, Canada, 24–28 September, pp. 6484–6489 (2017)
Kelion, L.: Toshiba’s robot is designed to be more human-like, BBC News, 9 March 2016. https://www.bbc.com/news/technology-35763917. Accessed 24 Feb 2019
Kose, H., Yorganci, R.: Tale of a robot: humanoid robot assisted sign language tutoring. In: 2011 11th IEEE-RAS International Conference on Humanoid Robots, Bled, Slovenia, 26–28 October (2011)
Goossens, M.: Optimisation of a humanoid sign language robot. B.S. thesis, Universiteit Antwerpen, Antwerpen, Belgium (2016)
Gago, J.J., Victores, J.G., Balaguer, C.: Sign language representation by TEO humanoid robot: end-user interest, comprehension and satisfaction. Electronics 8, 57 (2019)
Kyle, J.G., Woll, B.: Sign Language: The Study of Deaf People and Their Language. Cambridge University Press, Cambridge (1985)
Bridgwater, L.B., et al.: The robonaut 2 hand - designed to do work with tools. In: 2012 IEEE ICRA International Conference on Robotics and Automation (2012)
Carrozza, M.C., Dario, P., Vecchi, F., Roccella, S., Zecca, M., Sebastiani, F.: The cyber-hand: on the design of a cybernetic prosthetic hand intended to be interfaced to the peripheral nervous system. In: 2003 IEEE/RSJ IROS International Conference on Intelligent Robots and Systems (2003)
Birglen, L., Lalibert, T., Gosselin, C.M.: Underactuated Robotic Hands, vol. 40. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-77459-4
Casarez, C.S., Fearing, R.S.: Steering of an underactuated legged robot through terrain contact with an active tail. In: 2018 IEEE/RSJ IROS International Conference on Intelligent Robots and Systems (2018)
Jalani, J., Herrmann, G., Melhuish, C.: Robust trajectory following for underactuated robot fingers, pp. 495–500 (2010)
Chevallereau, C., Grizzle, J.W., Shih, C.: Asymptotically stable walking of a five-link underactuated 3D bipedal robot. IEEE Trans. Robot. 25(1), 37–50 (2009)
Kolathaya, S., Reher, J., Hereid, A., Ames, A.D.: Input to state stabilizing control Lyapunov functions for robust bipedal robotic locomotion. In: IEEE Annual American Control Conference (2018)
Buss, B.G., Ramezani, A., Hamed, K.A., Griffin, B.A., Galloway, K.S., Grizzle, J.W.: Experiments, preliminary walking, with underactuated 3D bipedal robot MARLO. In: 2014 IEEE/RSJ IROS International Conference on Intelligent Robots and Systems (2014)
Malvezzi, Monica., Salvietti, Gionata, Prattichizzo, Domenico: Evaluation of grasp stiffness in underactuated compliant hands exploiting environment constraints. In: Arakelian, Vigen, Wenger, Philippe (eds.) ROMANSY 22 – Robot Design, Dynamics and Control. CICMS, vol. 584, pp. 409–416. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-78963-7_51
Yao, S., Zhan, Q., Ceccarelli, M., Carbone, G., Lu, Z.: Analysis and grasp strategy modeling for underactuated multi-fingered robot hand. In: 2009 IEEE International Conference on Mechatronics and Automation (2009)
Li, Z., Du, R., Yu, H., Ren, H.: Statics modeling of an underactuated wire-driven flexible robotic arm. In: 5th IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics (2014)
Clevert, D., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (ELUs). In: 2016 ICLR International Conference Learning Representations (2016)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. In: 2014 ICLR International Conference on Learning Representations (2014)
Chollet, F.: Keras: the python deep learning library. Astrophysics Source Code Library, June 2018
Acknowledgments
The research leading to these results has received funding from RoboCity2030-DIH-CM Madrid Robotics Digital Innovation Hub (“Robótica aplicada a la mejora de la calidad de vida de los ciudadanos. Fase IV”; S2018/NMT-4331), funded by “Programas de Actividades I+D en la Comunidad de Madrid” and cofunded by Structural Funds of the EU. The authors thank CNSE (The Spanish Confederation of the Deaf) and LSE organizations such as Signapuntes Lengua de Signos and Mediación Comunicativa Cádiz for their kind collaboration with this project.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Gago, J.J., Łukawski, B., Victores, J.G., Balaguer, C. (2020). Under-Actuation Modelling in Robotic Hands via Neural Networks for Sign Language Representation with End-User Validation. In: Analide, C., Novais, P., Camacho, D., Yin, H. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2020. IDEAL 2020. Lecture Notes in Computer Science(), vol 12490. Springer, Cham. https://doi.org/10.1007/978-3-030-62365-4_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-62365-4_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-62364-7
Online ISBN: 978-3-030-62365-4
eBook Packages: Computer ScienceComputer Science (R0)