Skip to main content
Springer Nature Link
Log in

Comparative Analysis of Question Answering Models for HRI Tasks with NAO in Spanish

  • Conference paper
  • First Online:
Information Management and Big Data (SIMBig 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1410))

Included in the following conference series:

Abstract

Recent studies on Human Robot Interaction (HRI) have shown that different types of applications that combine metrics and techniques can help achieve a more efficient and organic interaction. This applications can be related to human care or go further and use a humanoid robot for nonverbal communication tasks. Similarly, for verbal communication, we found Question Answering, a Natural Language Processing task, that is in charge of capturing and interpret a question automatically and return a good representation of an answer. Also, recent work on creating Question Answering models, based on the Transformer architecture, have obtained state-of-the-art results. Our main goal in this project is to build a new Human Robot Interaction technique which uses a Question Answering system where we will test with college students. In the creation of the Question Answering model, we get results from state-of-the-art pre-trained models like BERT or XLNet, but also multilingual ones like m-BERT or XLM. We train them with a new Spanish dataset translated from the original SQuAD getting our best results with XLM-R, obtaining 68.1 F1 and 45.3 EM in the MLQA test dataset, and, 77.9 F1 and 58.3 EM for XQuAD test dataset. To validate the results obtained, we evaluated the project based on HRI metrics and a survey. The results demonstrate a high degree of acceptance in the students about the type of interaction that has been proposed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://drive.google.com/file/d/1JgADUQ0F0x9-gePfiftf7AXtxVxFpOli/view.

  2. 2.

    https://huggingface.co/.

  3. 3.

    https://github.com/zihangdai/xlnet/.

  4. 4.

    https://github.com/facebookresearch/SpanBERT/.

References

  1. Alenljung, B., Lindblom, J., Andreasson, R., Ziemke, T.: User experience in social human-robot interaction. Int. J. Ambient Comput. Intell. 8(2), 12–31 (2017)

    Article  Google Scholar 

  2. Aly, A., Griffiths, S.S., Stramandinoli, F.: Metrics and benchmarks in human-robot interaction: recent advances in cognitive robotics. Cogn. Syst. Res. 43, 313–323 (2017)

    Article  Google Scholar 

  3. Andreasson, R., Alenljung, B., Billing, E., Lowe, R.: Affective touch in human-robot interaction: conveying emotion to the nao robot. Int. J. Soc. Robot. 10, 473–491 (2018). https://doi.org/10.1007/s12369-017-0446-3

    Article  Google Scholar 

  4. Artetxe, M., Ruder, S., Yogatama, D.: On the cross-lingual transferability of monolingual representations. In: ACL (2020)

    Google Scholar 

  5. Bertacchini, F., Bilotta, E., Pantano, P.S.: Shopping with a robotic companion. Comput. Hum. Behav. 77, 382–395 (2017)

    Article  Google Scholar 

  6. Carrino, C.P., Costa-jussà, M.R., Fonollosa, J.A.R.: Automatic Spanish translation of the squad dataset for multilingual question answering. CoRR abs/1912.05200 (2019)

    Google Scholar 

  7. Conneau, A., et al.: Unsupervised cross-lingual representation learning at scale. In: ACL (2020)

    Google Scholar 

  8. Conneau, A., Lample, G.: Cross-lingual language model pretraining. In: NeurIPS (2019)

    Google Scholar 

  9. Dautenhahn, K.: Methodology and themes of human-robot interaction: a growing research field. Int. J. Adv. Robot. Syst. 4, 103–108 (2007)

    Google Scholar 

  10. Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT (2019)

    Google Scholar 

  11. Gao, J., Galley, M., Li, L.: Neural approaches to conversational AI. Found. Trends Inf. Retr. 13, 127–298 (2019)

    Article  Google Scholar 

  12. Joshi, M., Chen, D., Liu, Y., Weld, D.S., Zettlemoyer, L., Levy, O.: SpanBERT: improving pre-training by representing and predicting spans. Trans. ACL 8, 64–77 (2020)

    Google Scholar 

  13. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: ALBERT: a lite BERT for self-supervised learning of language representations. In: ICLR (2020)

    Google Scholar 

  14. Lee, J., et al.: BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36(4), 1234–1240 (2020)

    Google Scholar 

  15. Lewis, P.S.H., Oguz, B., Rinott, R., Riedel, S., Schwenk, H.: MLQA: evaluating cross-lingual extractive question answering. In: ACL (2020)

    Google Scholar 

  16. Lindblom, J., Andreasson, R.: Current challenges for UX evaluation of human-robot interaction. In: Schlick, C., Trzcieliński, S. (eds.) Advances in Ergonomics of Manufacturing: Managing the Enterprise of the Future. AISC, vol. 490, pp. 267–277. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41697-7_24

    Chapter  Google Scholar 

  17. Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach. CoRR abs/1907.11692 (2019)

    Google Scholar 

  18. Mattioli, T., Vendittelli, M.: Interaction force reconstruction for humanoid robots. IEEE Robot. Autom. Lett. 2(1), 282–289 (2017)

    Article  Google Scholar 

  19. Murphy, R.R., Schreckenghost, D.: Survey of metrics for human-robot interaction. In: HRI. IEEE/ACM (2013)

    Google Scholar 

  20. Pires, T., Schlinger, E., Garrette, D.: How multilingual is multilingual BERT? In: ACL (2019)

    Google Scholar 

  21. Quarteroni, S.: Natural language processing for industry - ELCA’s experience. Informatik Spektrum 41, 105–112 (2018). https://doi.org/10.1007/s00287-018-1094-1

    Article  Google Scholar 

  22. Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: SQuAD: 100,000+ questions for machine comprehension of text. In: EMNLP (2016)

    Google Scholar 

  23. Steinfeld, A., et al.: Common metrics for human-robot interaction. In: HRI (2006)

    Google Scholar 

  24. Szűcs, V., Karolyi, G., Tatar, A., Magyar, A.: Voice controlled humanoid robot based movement rehabilitation framework. In: CogInfoCom (2018)

    Google Scholar 

  25. Tiutiunnyk, S., Dyomkin, V.: Context-based question-answering system for the Ukrainian language. In: CEUR Workshop Proceedings (2020)

    Google Scholar 

  26. Ugarte, W., Boizumault, P., Loudni, S., Crémilleux, B.: Computing skypattern cubes. In: ECAI (2014)

    Google Scholar 

  27. Ugarte, W., Boizumault, P., Loudni, S., Crémilleux, B.: Computing skypattern cubes using relaxation. In: ICTAI. IEEE (2014)

    Google Scholar 

  28. Ugarte, W., Boizumault, P., Loudni, S., Crémilleux, B.: Modeling and mining optimal patterns using dynamic CSP. In: ICTAI. IEEE (2015)

    Google Scholar 

  29. Vaswani, A., et al.: Attention is all you need. In: NIPS (2017)

    Google Scholar 

  30. Yang, Z., Dai, Z., Yang, Y., Carbonell, J.G., Salakhutdinov, R., Le, Q.V.: XLNet: generalized autoregressive pretraining for language understanding. In: NeurIPS (2019)

    Google Scholar 

  31. Zaib, M., Sheng, Q.Z., Zhang, W.E.: A short survey of pre-trained language models for conversational AI-a new age in NLP. In: ACSW. ACM (2020)

    Google Scholar 

  32. Zhang, Y., Chen, X., Ai, Q., Yang, L., Croft, W.B.: Towards conversational search and recommendation: system ask, user respond. In: CIKM. ACM (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Universidad Peruana de Ciencias Aplicadas (UPC), Lima, Peru

    Enrique Burga-Gutierrez, Bryam Vasquez-Chauca & Willy Ugarte

Authors
  1. Enrique Burga-Gutierrez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bryam Vasquez-Chauca
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Willy Ugarte
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Willy Ugarte .

Editor information

Editors and Affiliations

  1. Stanford University, Stanford, CA, USA

    Juan Antonio Lossio-Ventura

  2. Visibilia, São Paulo, Brazil

    Jorge Carlos Valverde-Rebaza

  3. University of Valencia, Valencia, Spain

    Eduardo Díaz

  4. Universidad del Pacífico, Lima, Peru

    Hugo Alatrista-Salas

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Burga-Gutierrez, E., Vasquez-Chauca, B., Ugarte, W. (2021). Comparative Analysis of Question Answering Models for HRI Tasks with NAO in Spanish. In: Lossio-Ventura, J.A., Valverde-Rebaza, J.C., Díaz, E., Alatrista-Salas, H. (eds) Information Management and Big Data. SIMBig 2020. Communications in Computer and Information Science, vol 1410. Springer, Cham. https://doi.org/10.1007/978-3-030-76228-5_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-76228-5_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-76227-8

  • Online ISBN: 978-3-030-76228-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics