Skip to main content
Springer Nature Link
Log in

Generating Natural Language Responses in Robot-Mediated Referential Communication Tasks to Simulate Theory of Mind

  • Conference paper
  • First Online:
Social Robotics (ICSR 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13817))

Included in the following conference series:

Abstract

With advances in neural network-based computation, socially assistive robots have been endowed with the ability to provide natural conversation to users. However, the lack of transparency in the computation models results in unexpected robot behaviors and feedback, which may cause users to lose their trust in the robot. Theory of mind (ToM) in cooperative tasks has been considered as a key factor in understanding the relationship between user acceptance and the explainability of robot behaviors. Therefore, we develop a dialog system using previously collected data from a robot-mediated cooperative communication task data to simulate natural language smart feedback. The system is designed based on the mechanism of ToM and validated with a simulation test. Based on the result, we believe the designed dialog system bears the feasibility of simulating ToM and can be used as a research tool for further studying the importance of simulating ToM in human-robot communication.

Supported in part by National Institute of Health under the grant number R01AG077003.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Calder, M., Craig, C., Culley, D., De Cani, R., Donnelly, C.A., Douglas, R., Edmonds, B., Gascoigne, J., Gilbert, N., Hargrove, C., et al.: Computational modelling for decision-making: where, why, what, who and how. Royal Soc. Open Sci. 5(6), 172096 (2018)

    Article  Google Scholar 

  2. Chiu, H.M., et al.: Theory of mind predicts social interaction in children with autism spectrum disorder: A two-year follow-up study. J. Autism Dev. Disord. 1–11 (2022). https://doi.org/10.1007/s10803-022-05662-4

  3. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding (2018). https://doi.org/10.48550/ARXIV.1810.04805

  4. Foss, N., Stea, D.: Putting a realistic theory of mind into agency theory: implications for reward design and management in principal-agent relations. Eur. Manage. Rev. 11(1), 101–116 (2014)

    Article  Google Scholar 

  5. Fu, R., Guo, J., Qin, B., Che, W., Wang, H., Liu, T.: Learning semantic hierarchies via word embeddings. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1199–1209 (2014)

    Google Scholar 

  6. Grootendorst, M.: Keybert: Minimal keyword extraction with bert. (2020). https://doi.org/10.5281/zenodo.4461265

  7. John, A.E., Rowe, M.L., Mervis, C.B.: Referential communication skills of children with williams syndrome: understanding when messages are not adequate. Am. J. Intell. Dev. Disab. 114(2), 85–99 (2009)

    Article  Google Scholar 

  8. Jones, C.R., et al.: The association between theory of mind, executive function, and the symptoms of autism spectrum disorder. Autism Res. 11(1), 95–109 (2018)

    Article  Google Scholar 

  9. Kennedy, C.: Vagueness and grammar: the semantics of relative and absolute gradable adjectives. Linguist. Philos. 30(1), 1–45 (2007)

    Article  Google Scholar 

  10. Lin, C., Miller, T., Dligach, D., Bethard, S., Savova, G.: A bert-based universal model for both within-and cross-sentence clinical temporal relation extraction. In: Proceedings of the 2nd Clinical Natural Language Processing Workshop, pp. 65–71 (2019)

    Google Scholar 

  11. Liu, Z., et al.: A demonstration of human-robot communication based on multiskilled language-image analysis. In: 2021 IEEE/ACM Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), pp. 126–127. IEEE (2021)

    Google Scholar 

  12. Liu, Z., Paek, E.J., Yoon, S.O., Casenhiser, D., Zhou, W., Zhao, X.: Detecting alzheimer’s disease using natural language processing of referential communication task transcripts. J. Alzheimer’s Disease 86(3), 1–14 (2022)

    Google Scholar 

  13. Maridaki-Kassotaki, K., Antonopoulou, K.: Examination of the relationship between false-belief understanding and referential communication skills. Eur. J. Psychol. Educ. 26(1), 75–84 (2011)

    Article  Google Scholar 

  14. Miller, T.: Explanation in artificial intelligence: insights from the social sciences. Artif. Intell. 267, 1–38 (2019)

    Article  MATH  Google Scholar 

  15. Mou, L., Song, Y., Yan, R., Li, G., Zhang, L., Jin, Z.: Sequence to backward and forward sequences: a content-introducing approach to generative short-text conversation. arXiv preprint arXiv:1607.00970 (2016)

  16. Nilsen, E.S., Fecica, A.M.: A model of communicative perspective-taking for typical and atypical populations of children. Dev. Rev. 31(1), 55–78 (2011)

    Article  Google Scholar 

  17. O’Reilly, Z., Silvera-Tawil, D., Tan, D.W., Zurr, I.: Validation of a novel theory of mind measurement tool: the social robot video task. In: Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction, pp. 89–93 (2021)

    Google Scholar 

  18. Paal, T., Bereczkei, T.: Adult theory of mind, cooperation, machiavellianism: the effect of mindreading on social relations. Personality individ. Differ. 43(3), 541–551 (2007)

    Article  Google Scholar 

  19. Rai, A.: Explainable AI: From black box to glass box. J. Acad. Mark. Sci. 48(1), 137–141 (2020)

    Article  Google Scholar 

  20. Shi, P., Lin, J.: Simple bert models for relation extraction and semantic role labeling. arXiv preprint arXiv:1904.05255 (2019)

  21. Sidera, F., Perpiñà, G., Serrano, J., Rostan, C.: Why is theory of mind important for referential communication? Curr. Psychol. 37(1), 82–97 (2018)

    Article  Google Scholar 

  22. Song, Y., Luximon, Y.: Trust in AI agent: a systematic review of facial anthropomorphic trustworthiness for social robot design. Sensors 20(18), 5087 (2020)

    Article  Google Scholar 

  23. Sun, C., Qiu, X., Xu, Y., Huang, X.: How to fine-tune BERT for text classification? In: Sun, M., Huang, X., Ji, H., Liu, Z., Liu, Y. (eds.) CCL 2019. LNCS (LNAI), vol. 11856, pp. 194–206. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32381-3_16

    Chapter  Google Scholar 

  24. Whiten, A., Byrne, R.W.: The machiavellian intelligence hypotheses (1988)

    Google Scholar 

  25. Xu, J., Bu, Y., Ding, Y., Yang, S., Zhang, H., Yu, C., Sun, L.: Understanding the formation of interdisciplinary research from the perspective of keyword evolution: a case study on joint attention. Scientometrics 117(2), 973–995 (2018). https://doi.org/10.1007/s11192-018-2897-1

    Article  Google Scholar 

  26. Zhang, Z., Wu, Y., Zhao, H., Li, Z., Zhang, S., Zhou, X., Zhou, X.: Semantics-aware bert for language understanding. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 9628–9635 (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Tennessee, 1506 Middle Drive, Knoxville, TN, 37916, USA

    Ziming Liu & Xiaopeng Zhao

  2. City University of Hong Kong, 83 Tat Chee Ave., Kowloon Tong, Kowloon, 999077, Hong Kong

    Yigang Qin & Huiqi Zou

  3. University of Tennessee Health Science Center, 600 Henley Street, Knoxville, TN, 37902, USA

    Eun Jin Paek & Devin Casenhiser

  4. University of Tennessee, 916 Volunteer Boulevard, Knoxville, TN, 37996, USA

    Wenjun Zhou

Authors
  1. Ziming Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yigang Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huiqi Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eun Jin Paek
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Devin Casenhiser
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wenjun Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xiaopeng Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ziming Liu .

Editor information

Editors and Affiliations

  1. University of Florence, Florence, Italy

    Filippo Cavallo

  2. Qatar University, Doha, Qatar

    John-John Cabibihan

  3. University of Florence, Florence, Italy

    Laura Fiorini

  4. University of Florence, Florence, Italy

    Alessandra Sorrentino

  5. Wichita State University, Wichita, KS, USA

    Hongsheng He

  6. Qingdao University, Qingdao, China

    Xiaorui Liu

  7. National Institute of Advanced Industrial Science and Technology, Tsukuba, Japan

    Yoshio Matsumoto

  8. National University of Singapore, Singapore, Singapore

    Shuzhi Sam Ge

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liu, Z. et al. (2022). Generating Natural Language Responses in Robot-Mediated Referential Communication Tasks to Simulate Theory of Mind. In: Cavallo, F., et al. Social Robotics. ICSR 2022. Lecture Notes in Computer Science(), vol 13817. Springer, Cham. https://doi.org/10.1007/978-3-031-24667-8_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-24667-8_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-24666-1

  • Online ISBN: 978-3-031-24667-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics