Abstract
The recognition of objects and their features is a fundamental task for social robots that could be improved with the combination of different sources of information, such as the ones provided by visual or speech understanding systems. In this paper, we present a first approach to fusion semantic localization and conversational skills for social robots which may act as assistants. Our solution is based on a mobile robot that is able to detect and recognize objects from an environment and store them in its base of knowledge to later act as an assistant for any user who is searching for any object. In the conversation the robot tries to help the user to find a specific object depending of the location and the features of the object which is looking for. The proposal has been empirically evaluated within a research lab where the robot recognizes objects in the environment and the users require, by means of speech commands, finding suitable objects that are placed in the environment.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bandera, A., Bandera, J.P., Bustos, P., Férnandez, F., García-Olaya, A., García-Polo, J., García-Varea, I., Manso, L.J., Marfil, R., Martínez-Gómez, J., Núñez, P., Perez-Lorenzo, J.M., Reche-Lopez, P., Romero-González, C., Viciana-Abad, R.: LifeBots I: building the software infrastructure for supporting lifelong technologies. In: Third Iberian Robotics Conference (ROBOT), Sevilla, Spain (2017)
Bradski, G., Kaehler, A.: Opencv. Dr. Dobbs J. Softw. Tools 3 (2000)
Chan, M., Estève, D., Escriba, C., Campo, E.: A review of smart homespresent state and future challenges. Comput. Methods Progr. Biomed. 91(1), 55–81 (2008)
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12(Aug), 2493–2537 (2011)
Durette, P.N.: Google text-to-speech (version 2.0.1) [software] (2018). https://github.com/pndurette/gTTS
Goodrich, M.A., Schultz, A.C., et al.: Human–robot interaction: a survey. Found. Trends® Hum. Comput. Interact. 1(3), 203–275 (2008)
Gross, H.-M., Schroeter, C., Mueller, S., Volkhardt, M., Einhorn, E., Bley, A., Martin, C., Langner, T., Merten, M.: Progress in developing a socially assistive mobile home robot companion for the elderly with mild cognitive impairment. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2430–2437. IEEE (2011)
Hartigan, J.A., Wong, M.A.: Algorithm as 136: a k-means clustering algorithm. J. Roy. Stat. Soc. Ser. C (Appl. Stat.) 28(1), 100–108 (1979)
Iocchi, L., Kraetzschmar, G., Nardi, D., Lima, P.U., Miraldo, P., Bastianelli, E., Capobianco, R.: Rockin@home: Domestic robots challenge. In: RoCKIn, chap. 3. IntechOpen, Rijeka (2017)
Littlewort, G.C., Bartlett, M.S., Fasel, I.R., Chenu, J., Kanda, T., Ishiguro, H., Movellan, J.R.: Towards social robots: automatic evaluation of human-robot interaction by facial expression classification. In: Advances in Neural Information Processing Systems, pp. 1563–1570 (2004)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)
SPARC Robotics: Robotics 2020 multi-annual roadmap for robotics in Europe. In: SPARC Robotics, EU-Robotics AISBL, The Hauge, The Netherlands (2016). Accessed 5 Feb 2018
Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans. Pattern Anal. Mach. Intell. 31(1), 39–58 (2009)
Zhang, A.: Speech recognition (version 3.8) [software] (2017). https://github.com/Uberi/speech_recognition
Acknowledgements
This work has been partially sponsored by the Spanish Ministry of Economy and Competitiveness under grant number TIN2015-65686-C5-3-R and by the Regional Council of Education, Culture and Sports of Castilla-La Mancha under grant number SBPLY/17/180501/000493, supported with Feder funds.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
González-Medina, D., Romero-González, C., García-Varea, I. (2019). Combination of Semantic Localization and Conversational Skills for Assistive Robots. In: Fuentetaja Pizán, R., García Olaya, Á., Sesmero Lorente, M., Iglesias Martínez, J., Ledezma Espino, A. (eds) Advances in Physical Agents. WAF 2018. Advances in Intelligent Systems and Computing, vol 855. Springer, Cham. https://doi.org/10.1007/978-3-319-99885-5_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-99885-5_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-99884-8
Online ISBN: 978-3-319-99885-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)