Skip to main content
SpringerLink
Log in

Conversational Interactions with NPCs in LLM-Driven Gaming: Guidelines from a Content Analysis of Player Feedback

  • Conference paper
  • First Online:
Chatbot Research and Design (CONVERSATIONS 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14524))

Included in the following conference series:

  • 374 Accesses

Abstract

The growing capability and availability of large language models (LLMs) have led to their adoption in a number of domains. One application domain that could prove fruitful is to video games, where LLMs could be used to provide conversational responses from non-playable characters (NPCs) that are more dynamic and diverse. Additionally, LLMs could allow players the autonomy to converse in open-ended conversations potentially improving player immersion and agency. However, due to their recent commercial popularity, the consequences (both negative and positive) of using LLMs in video games from a player’s perspective is currently unclear. On from this, we analyse player feedback to the use of LLM-driven NPC responses in a commercially available video game. We discuss findings and implications, and generate guidelines for designers incorporating LLMs into NPC dialogue.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 49.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Note on terminology: An “NPC” can be thought of as an embodied conversational agent that a user interacts with in a virtual environment, and a “player” can be thought of as a user that talks to said conversational agent.

  2. 2.

    “poetic” review here.

  3. 3.

    “enigmas” review here.

  4. 4.

    As confirmed from both private correspondence with the game development studio, and a Steam forum [developer] post here.

  5. 5.

    See Discord thread for player confusion surrounding existence of location.

  6. 6.

    See Discord thread for inconsistent fingerprint responses.

  7. 7.

    See screenshots in Discord for contradictory NPC responses regarding knowledge of murder victim.

  8. 8.

    See second sentence of Steam review here.

  9. 9.

    See https://github.com/josephrocca/OpenCharacters for such an implementation.

References

  1. Bansal, H., Dang, J., Grover, A.: Peering through preferences: unraveling feedback acquisition for aligning large language models. arXiv preprint arXiv:2308.15812 (2023)

  2. Bowey, J.T., Friehs, M.A., Mandryk, R.L.: Red or blue pill: fostering identification and transportation through dialogue choices in RPGs. In: Proceedings of the 14th International Conference on the Foundations of Digital Games, pp. 1–11 (2019)

    Google Scholar 

  3. Braun, V., Clarke, V.: Using thematic analysis in psychology. Qual. Res. Psychol. 3(2), 77–101 (2006)

    Article  Google Scholar 

  4. Bumblebee-Studios: Vaudeville on Steam. Steam (June 2023). https://store.steampowered.com/app/2240920/Vaudeville/

  5. Chen, L., Zaharia, M., Zou, J.: How is chatgpt’s behavior changing over time? arXiv preprint arXiv:2307.09009 (2023)

  6. Cox, S.R., Abdul, A., Ooi, W.T.: Prompting a large language model to generate diverse motivational messages: a comparison with human-written messages. In: Proceedings of the 11th International Conference on Human-Agent Interaction (2023)

    Google Scholar 

  7. Cox, S.R., Lee, Y.C., Ooi, W.T.: Comparing how a chatbot references user utterances from previous chatting sessions: an investigation of users privacy concerns and perceptions. In: Proceedings of the 11th International Conference on Human-Agent Interaction (2023)

    Google Scholar 

  8. Csepregi, L.M.: The Effect of Context-aware LLM-based NPC Conversations on Player Engagement in Role-playing Video Games

    Google Scholar 

  9. El-Kassas, W.S., Salama, C.R., Rafea, A.A., Mohamed, H.K.: Automatic text summarization: a comprehensive survey. Expert Syst. Appl. 165, 113679 (2021)

    Article  Google Scholar 

  10. Frommel, J., Fischbach, F., Rogers, K., Weber, M.: Emotion-based dynamic difficulty adjustment using parameterized difficulty and self-reports of emotion. In: Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play, pp. 163–171 (2018)

    Google Scholar 

  11. Hunicke, R.: The case for dynamic difficulty adjustment in games. In: Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, pp. 429–433 (2005)

    Google Scholar 

  12. Inworld: Inworld - The most advanced Character Engine for AI NPCs. https://inworld.ai/

  13. Inworld: Inworld Origins on Steam. Steam (July 2023). https://store.steampowered.com/app/2199920/Inworld_Origins/

  14. Jiang, H., Zhang, X., Cao, X., Kabbara, J., Roy, D.: PersonaLLM: Investigating the Ability of GPT-3.5 to Express Personality Traits and Gender Differences (2023)

    Google Scholar 

  15. Khadpe, P., Krishna, R., Fei-Fei, L., Hancock, J.T., Bernstein, M.S.: Conceptual metaphors impact perceptions of human-AI collaboration. Proc. ACM Hum.-Comput. Interact. 4(CSCW2), 1–26 (2020)

    Article  Google Scholar 

  16. Kim, B.J., Choi, Y.S.: Automatic baseball commentary generation using deep learning. In: Proceedings of the 35th Annual ACM Symposium on Applied Computing, pp. 1056–1065 (2020)

    Google Scholar 

  17. Kway, L., Mitchell, A.: Emotional agency in storygames. In: Proceedings of the 13th International Conference on the Foundations of Digital Games, pp. 1–10 (2018)

    Google Scholar 

  18. Lin, D., Bezemer, C.P., Zou, Y., Hassan, A.E.: An empirical study of game reviews on the Steam platform. Empir. Softw. Eng. 24, 170–207 (2019)

    Article  Google Scholar 

  19. Liu, N.F., et al.: Lost in the middle: how language models use long contexts. arXiv preprint arXiv:2307.03172 (2023)

  20. Mateas, M., Stern, A.: Façade: an experiment in building a fully-realized interactive drama. In: Game Developers Conference, vol. 2, pp. 4–8. Citeseer (2003)

    Google Scholar 

  21. Mehta, M., Dow, S., Mateas, M., MacIntyre, B.: Evaluating a conversation-centered interactive drama. In: Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 1–8 (2007)

    Google Scholar 

  22. Mirowski, P., Mathewson, K.W., Pittman, J., Evans, R.: Co-writing screenplays and theatre scripts with language models: evaluation by industry professionals. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 1–34 (2023)

    Google Scholar 

  23. Mori, Y., Miyake, Y.: Ethical issues in automatic dialogue generation for non-player characters in digital games. In: 2022 IEEE International Conference on Big Data (Big Data), pp. 5132–5139. IEEE (2022)

    Google Scholar 

  24. OpenAI: Inworld AI (January 2023). https://openai.com/customer-stories/inworld-ai

  25. Packer, C., Fang, V., Patil, S.G., Lin, K., Wooders, S., Gonzalez, J.E.: Memgpt: towards llms as operating systems. arXiv preprint arXiv:2310.08560 (2023)

  26. Paduraru, C., Cernat, M., Stefanescu, A.: Conversational agents for simulation applications and video games. In: Proceedings of 18th International Conference on Software Technologies (ICSOFT 2023) (2023)

    Google Scholar 

  27. Park, J.S., O’Brien, J.C., Cai, C.J., Morris, M.R., Liang, P., Bernstein, M.S.: Generative agents: interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442 (2023)

  28. Perlin, K., Goldberg, A.: Improv: a system for scripting interactive actors in virtual worlds. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, pp. 205–216 (1996)

    Google Scholar 

  29. Phillips, C., Klarkowski, M., Frommel, J., Gutwin, C., Mandryk, R.L.: Identifying commercial games with therapeutic potential through a content analysis of Steam reviews. Proc. ACM Hum. Comput. Interact. 5(CHI PLAY), 1–21 (2021)

    Google Scholar 

  30. Safdari, M., et al.: Personality traits in large language models. arXiv preprint arXiv:2307.00184 (2023)

  31. van Stegeren, J., Myśliwiec, J.: Fine-tuning GPT-2 on annotated RPG quests for NPC dialogue generation. In: Proceedings of the 16th International Conference on the Foundations of Digital Games, pp. 1–8 (2021)

    Google Scholar 

  32. Sun, Y., Li, Z., Fang, K., Lee, C.H., Asadipour, A.: Language as Reality: A Co-Creative Storytelling Game Experience in 1001 Nights using Generative AI. arXiv preprint arXiv:2308.12915 (2023)

  33. Tanenbaum, K., Tanenbaum, T.J.: Commitment to meaning: a reframing of agency in games (2009)

    Google Scholar 

  34. Vanhatupa, J.M.: Guidelines for personalizing the player experience in computer role-playing games. In: Proceedings of the 6th International Conference on Foundations of Digital Games, pp. 46–52 (2011)

    Google Scholar 

  35. Värtinen, S., Hämäläinen, P., Guckelsberger, C.: Generating role-playing game quests with GPT language models. IEEE Trans. Games (2022)

    Google Scholar 

  36. Weibel, D., Wissmath, B.: Immersion in computer games: the role of spatial presence and flow. Inter. J. Comput. Games Technol. 2011, 6–6 (2011)

    Article  Google Scholar 

  37. Weir, N., Thomas, R., D’Amore, R., Hill, K., Van Durme, B., Jhamtani, H.: Ontologically Faithful Generation of Non-Player Character Dialogues. arXiv preprint arXiv:2212.10618 (2022)

  38. Wu, W., Jiang, C., Jiang, Y., Xie, P., Tu, K.: Do plms know and understand ontological knowledge? In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3080–3101 (2023)

    Google Scholar 

  39. Xi, Y., et al.: Kuileixi: a chinese open-ended text adventure game. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations, pp. 175–184 (2021)

    Google Scholar 

  40. Xie, L., Wu, Z., Xu, P., Li, W., Ma, X., Li, Q.: RoleSeer: understanding informal social role changes in MMORPGs via visual analytics. In: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pp. 1–17 (2022)

    Google Scholar 

  41. Yan, V.: Yandere AI Girlfriend Simulator (2023). https://helixngc7293.itch.io/yandere-ai-girlfriend-simulator

  42. Zagal, J.P., Ladd, A., Johnson, T.: Characterizing and understanding game reviews. In: Proceedings of the 4th international Conference on Foundations of Digital Games, pp. 215–222 (2009)

    Google Scholar 

  43. Zagal, J.P., Tomuro, N.: Cultural differences in game appreciation: a study of player game reviews. In: Proceedings of the 8th international Conference on Foundations of Digital Games, pp. 86–93 (2013)

    Google Scholar 

  44. Zhang, A.X., Cranshaw, J.: Making sense of group chat through collaborative tagging and summarization. Proc. ACM Hum.-Comput. Interact. 2(CSCW), 1–27 (2018)

    Google Scholar 

Download references

Acknowledgement

We would like to thank Alex Mitchell for discussions regarding video game design literature and our generated design guidelines, Ashraf Abdul for their assistance in thematic analysis, and Bumblebee Studios for being friendly and open in answering queries regarding Vaudeville.

Author information

Authors and Affiliations

  1. National University of Singapore, Singapore, Singapore

    Samuel Rhys Cox & Wei Tsang Ooi

Authors
  1. Samuel Rhys Cox
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Tsang Ooi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Samuel Rhys Cox .

Editor information

Editors and Affiliations

  1. SINTEF, Oslo, Norway

    Asbjørn Følstad

  2. University of Amsterdam, Amsterdam, The Netherlands

    Theo Araujo

  3. CERTH-ITI, Thessaloniki, Greece

    Symeon Papadopoulos

  4. Department of Computer Science, Durham University, Durham, UK

    Effie L.-C. Law

  5. Design Informatics, University of Edinburgh, Edinburgh, UK

    Ewa Luger

  6. Centre for AI Research, University of Agder, Grimstad, Norway

    Morten Goodwin

  7. TH Lübeck – University of Applied Sciences, Lübeck, Germany

    Sebastian Hobert

  8. SINTEF & University of Oslo, Oslo, Norway

    Petter Bae Brandtzaeg

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cox, S.R., Ooi, W.T. (2024). Conversational Interactions with NPCs in LLM-Driven Gaming: Guidelines from a Content Analysis of Player Feedback. In: Følstad, A., et al. Chatbot Research and Design. CONVERSATIONS 2023. Lecture Notes in Computer Science, vol 14524. Springer, Cham. https://doi.org/10.1007/978-3-031-54975-5_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-54975-5_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-54974-8

  • Online ISBN: 978-3-031-54975-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation