From Playing the Story to Gaming the System: Repeat Experiences of a Large Language Model-Based Interactive Story

Abstract

A distinctive aspect of interactive stories is how they enable players to impact the story based on their choices, with repeated playthroughs potentially yielding different stories. While some work has been done to investigate repeat experiences of interactive stories, the focus has mainly been on simpler linear and branching narrative systems. Recent advances in large language model (LLM) systems such as ChatGPT have sparked renewed interest in how users engage with artificial intelligence systems, and how LLM systems could be used in works of entertainment such as games and interactive stories. In this paper, we detail the findings of a qualitative observational study of 10 participants playing and replaying the interactive text adventure game AI Dungeon which is based upon the natural-language-generating LLMs GPT-3 and GPT-J. The study provides insights into how the player’s mental model of the system shifts from assuming the system is a linear or branching narrative to one with seemingly limitless narrative possibilities. This study also uncovers three related sources of motivations for repeat engagement with an LLM-based interactive story, which are impacted by the player’s shifting mental model of the system: narrative closure, narrative incoherence, and expectations for variation between playthroughs. Finally, this study highlights how uncertainty in terms of both how the system works and what objectives the player has could impact the player’s choice of actions when engaging with an LLM-based interactive story such as AI Dungeon.