skip to main content
10.1145/3654777.3676372acmotherconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article

Story-Driven: Exploring the Impact of Providing Real-time Context Information on Automated Storytelling

Published: 11 October 2024 Publication History

Abstract

Stories have long captivated the human imagination with narratives that enrich our lives. Traditional storytelling methods are often static and not designed to adapt to the listener’s environment, which is full of dynamic changes. For instance, people often listen to stories in the form of podcasts or audiobooks while traveling in a car. Yet, conventional in-car storytelling systems do not embrace the adaptive potential of this space. The advent of generative AI is the key to creating content that is not just personalized but also responsive to the changing parameters of the environment. We introduce a novel system for interactive, real-time story narration that leverages environment and user context in correspondence with estimated arrival times to adjust the generated story continuously. Through two comprehensive real-world studies with a total of 30 participants in a vehicle, we assess the user experience, level of immersion, and perception of the environment provided by the prototype. Participants’ feedback shows a significant improvement over traditional storytelling and highlights the importance of context information for generative storytelling systems.

Supplemental Material

MP4 File
Video figure
ZIP File
Exemplary stories and currently anonymized presentation video

References

[1]
Gregory D Abowd, Anind K Dey, Peter J Brown, Nigel Davies, Mark Smith, and Pete Steggles. 1999. Towards a better understanding of context and context-awareness. In Handheld and Ubiquitous Computing: First International Symposium, HUC’99 Karlsruhe, Germany, September 27–29, 1999 Proceedings 1. Springer, 304–307.
[2]
Mieke Bal. [n. d.]. Narratology: introduction to the theory of narrative (3. ed ed.). University of Toronto Press.
[3]
Matthias Baldauf, Schahram Dustdar, and Florian Rosenberg. 2007. A survey on context-aware systems. International Journal of ad Hoc and ubiquitous Computing 2, 4 (2007), 263–277.
[4]
Roland Barthes and Roland Barthes. [n. d.]. S-Z (nachdr ed.). Number 687 in Suhrkamp-Taschenbuch Wissenschaft. Suhrkamp.
[5]
Nina Begus. 2023. Experimental Narratives: A Comparison of Human Crowdsourced Storytelling and AI Storytelling. arXiv preprint arXiv:2310.12902 (2023).
[6]
David Bethge, Daniel Bulanda, Adam Kozlowski, Thomas Kosch, Albrecht Schmidt, and Tobias Grosse-Puppendahl. 2024. HappyRouting: Learning Emotion-Aware Route Trajectories for Scalable In-The-Wild Navigation. arxiv:2401.15695 [cs.HC]
[7]
Arthur Brisbane. [n. d.]. Newspaper Copy That People Must Read, Advertising’s Relation to the Growth of Reading Ability—the Thunderstorm and "Yellow" Journalism—an Example of the Power of Comparison in Writing. ([n. d.]), 17.
[8]
Rick Busselle and Helena Bilandzic. 2009. Measuring narrative engagement. Media psychology 12, 4 (2009), 321–347.
[9]
Guanling Chen and David Kotz. 2000. A survey of context-aware mobile computing research. (2000).
[10]
Zexin Chen, Eric Zhou, Kenneth Eaton, Xiangyu Peng, and Mark Riedl. 2023. Ambient Adventures: Teaching ChatGPT on Developing Complex Stories. arXiv preprint arXiv:2308.01734 (2023).
[11]
Haoran Chu and Sixiao Liu. [n. d.]. Can AI Tell Good Stories? Narrative Transportation and Persuasion with ChatGPT. ([n. d.]). https://psyarxiv.com/c3549/download?format=pdf
[12]
John Joon Young Chung, Wooseok Kim, Kang Min Yoo, Hwaran Lee, Eytan Adar, and Minsuk Chang. 2022. TaleBrush: Sketching Stories with Generative Pretrained Language Models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (, New Orleans, LA, USA, ) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 209, 19 pages. https://doi.org/10.1145/3491102.3501819
[13]
Andy Coenen, Luke Davis, Daphne Ippolito, Emily Reif, and Ann Yuan. 2021. Wordcraft: a human-ai collaborative editor for story writing. arXiv preprint arXiv:2107.07430 (2021).
[14]
Lajos Matyas Csepregi. 2021. The effect of context-aware llm-based npc conversations on player engagement in role-playing video games. Unpublished manuscript (2021).
[15]
Anind K Dey. 1998. Context-aware computing: The CyberDesk project. In Proceedings of the AAAI 1998 Spring Symposium on Intelligent Environments. AAAI Press Menlo Park, CA, 51–54.
[16]
Fiona Draxler, Daniel Buschek, Mikke Tavast, Perttu Hämäläinen, Albrecht Schmidt, Juhi Kulshrestha, and Robin Welsch. 2023. Gender, age, and technology education influence the adoption and appropriation of LLMs. arXiv preprint arXiv:2310.06556 (2023).
[17]
Panagiotis Fotaris, Theodoros Mastoras, and Petros Lameras. 2023. Designing Educational Escape Rooms With Generative AI: A Framework and ChatGPT Prompt Engineering Guide. In 17th European Conference on Games Based Learning.
[18]
Louie Giray. 2023. Prompt Engineering with ChatGPT: A Guide for Academic Writers. Annals of Biomedical Engineering (2023), 1–5.
[19]
Mustafa Can Gursesli, Pittawat Taveekitworachai, Febri Abdullah, Mury F Dewantoro, Antonio Lanata, Andrea Guazzini, Van Khôi Lê, Adrien Villars, and Ruck Thawonmas. 2023. The Chronicles of ChatGPT: Generating and Evaluating Visual Novel Narratives on Climate Change Through ChatGPT. In International Conference on Interactive Digital Storytelling. Springer, 181–194.
[20]
Anton Gustafsson, John Bichard, Liselott Brunnberg, Oskar Juhlin, and Marco Combetto. [n. d.]. Believable Environments: Generating Interactive Storytelling in Vast Location-Based Pervasive Games. In Proceedings of the 2006 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (New York, NY, USA, 2006-06-14) (ACE ’06). Association for Computing Machinery, 24–es. https://doi.org/10.1145/1178823.1178852
[21]
Chi-yang Hsu, Yun-Wei Chu, Ting-Hao Huang, and Lun-Wei Ku. [n. d.]. Plot and Rework: Modeling Storylines for Visual Storytelling. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 (Online, 2021-08), Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Association for Computational Linguistics, 4443–4453. https://doi.org/10.18653/v1/2021.findings-acl.390
[22]
Ting-Hao Kenneth Huang, Francis Ferraro, Nasrin Mostafazadeh, Ishan Misra, Aishwarya Agrawal, Jacob Devlin, Ross Girshick, Xiaodong He, Pushmeet Kohli, Dhruv Batra, C. Lawrence Zitnick, Devi Parikh, Lucy Vanderwende, Michel Galley, and Margaret Mitchell. [n. d.]. Visual Storytelling. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (San Diego, California, 2016-06), Kevin Knight, Ani Nenkova, and Owen Rambow (Eds.). Association for Computational Linguistics, 1233–1239. https://doi.org/10.18653/v1/N16-1147
[23]
Catherine Kanellopoulou, Katia Lida Kermanidis, and Andreas Giannakoulopoulos. [n. d.]. The Dual-Coding and Multimedia Learning Theories: Film Subtitles as a Vocabulary Teaching Tool. 9, 3 ([n. d.]), 210. https://doi.org/10.3390/educsci9030210 Number: 3 Publisher: Multidisciplinary Digital Publishing Institute.
[24]
Mohamed Kari, Tobias Grosse-Puppendahl, Alexander Jagaciak, David Bethge, Reinhard Schütte, and Christian Holz. [n. d.]. SoundsRide: Affordance-Synchronized Music Mixing for In-Car Audio Augmented Reality. In The 34th Annual ACM Symposium on User Interface Software and Technology (Virtual Event USA, 2021-10-10). ACM, 118–133. https://doi.org/10.1145/3472749.3474739
[25]
Taewook Kim, Hyomin Han, Eytan Adar, Matthew Kay, and John Joon Young Chung. [n. d.]. Authors’ Values and Attitudes Towards AI-bridged Scalable Personalization of Creative Language Arts. In Proceedings of the CHI Conference on Human Factors in Computing Systems (Honolulu HI USA, 2024-05-11). ACM, 1–16. https://doi.org/10.1145/3613904.3642529 arxiv:2403.00439 [cs]
[26]
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems 35 (2022), 22199–22213.
[27]
Vikram Kumaran, Jonathan Rowe, Bradford Mott, and James Lester. 2023. SCENECRAFT: automating interactive narrative scene generation in digital games with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol. 19. 86–96.
[28]
Mina Lee, Katy Ilonka Gero, John Joon Young Chung, Simon Buckingham Shum, Vipul Raheja, Hua Shen, Subhashini Venugopalan, Thiemo Wambsganss, David Zhou, Emad A Alghamdi, 2024. A Design Space for Intelligent and Interactive Writing Assistants. arXiv preprint arXiv:2403.14117 (2024).
[29]
Zhiyu Lin and Mark Riedl. 2021. Plug-and-blend: A framework for controllable story generation with blended control codes. arXiv preprint arXiv:2104.04039 (2021).
[30]
Richard E. Mayer. [n. d.]. Multimedia learning. Cambridge University Press. https://doi.org/10.1017/CBO9781139164603 Pages: xi, 210.
[31]
Munan Ning, Yujia Xie, Dongdong Chen, Zeyin Song, Lu Yuan, Yonghong Tian, Qixiang Ye, and Li Yuan. 2023. Album Storytelling with Iterative Story-aware Captioning and Large Language Models. arXiv preprint arXiv:2305.12943 (2023).
[32]
Allan Paivio. [n. d.]. Dual coding theory: Retrospect and current status. 45, 3 ([n. d.]), 255–287. https://doi.org/10.1037/h0084295 Place: Canada Publisher: Canadian Psychological Association.
[33]
Jeongyoon Park, Jumin Shin, Gayeon Kim, and Byung-Chull Bae. 2023. Designing a Language Model-Based Authoring Tool Prototype for Interactive Storytelling. In Interactive Storytelling: 16th International Conference on Interactive Digital Storytelling, ICIDS 2023, Kobe, Japan, November 11–15, 2023, Proceedings, Part II (Kobe, Japan). Springer-Verlag, Berlin, Heidelberg, 239–245. https://doi.org/10.1007/978-3-031-47658-7_22
[34]
Bastian Pfleging, Maurice Rang, and Nora Broy. [n. d.]. Investigating User Needs for Non-Driving-Related Activities during Automated Driving. In Proceedings of the 15th International Conference on Mobile and Ubiquitous Multimedia (New York, NY, USA, 2016-12-12) (MUM ’16). Association for Computing Machinery, 91–99. https://doi.org/10.1145/3012709.3012735
[35]
Hannah Rashkin, Asli Celikyilmaz, Yejin Choi, and Jianfeng Gao. 2020. Plotmachines: Outline-conditioned generation with dynamic plot state tracking. arXiv preprint arXiv:2004.14967 (2020).
[36]
Daniel C Richardson, Nicole K Griffin, Lara Zaki, Auburn Stephenson, Jiachen Yan, Thomas Curry, Richard Noble, John Hogan, Jeremy I Skipper, and Joseph T Devlin. 2020. Engagement in video and audio narratives: Contrasting self-report and physiological measures. Scientific Reports 10, 1 (2020), 11298.
[37]
Alejandro Rivero-Rodriguez, Paolo Pileggi, and Ossi Antero Nykänen. 2016. Mobile context-aware systems: technologies, resources and applications. International Journal of Interactive Mobile Technologies 10, 2 (2016), 25–32.
[38]
Marie-Laure Ryan. [n. d.]. Narrative as virtual reality: immersion and interactivity in literature and electronic media (transferred to digital print. 2001 - [im kolophon: milton keynes: lightning source, 2010] ed.). Johns Hopkins Univ. Press.
[39]
Nick Ryan, Jason Pascoe, and David Morse. 1999. Enhanced reality fieldwork: the context aware archaeological assistant. (1999).
[40]
Bill N Schilit and Marvin M Theimer. 1994. Disseminating active map information to mobile hosts. IEEE network 8, 5 (1994), 22–32.
[41]
Martin Schrepp, Andreas Hinderks, and Jörg Thomaschewski. 2017. Design and evaluation of a short version of the user experience questionnaire (UEQ-S). International Journal of Interactive Multimedia and Artificial Intelligence, 4 (6), 103-108. (2017).
[42]
Nisha Simon and Christian Muise. 2022. TattleTale: Storytelling with Planning and Large Language Models. In ICAPS Workshop on Scheduling and Planning Applications.
[43]
Nicholas Suwono, Justin Chen, Tun Hung, Ting-Hao Huang, I-Bin Liao, Yung-Hui Li, Lun-Wei Ku, and Shao-Hua Sun. [n. d.]. Location-Aware Visual Question Generation with Lightweight Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (Singapore, 2023-12), Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, 1415–1432. https://doi.org/10.18653/v1/2023.emnlp-main.88
[44]
Xu Tan, Tao Qin, Frank Soong, and Tie-Yan Liu. 2021. A survey on neural speech synthesis. arXiv preprint arXiv:2106.15561 (2021).
[45]
Ronald B Tobias. 2012. 20 master plots: And how to build them. Penguin.
[46]
Runyu Wang, Keng Leng Siau, and Zili Zhang. 2023. Using AI and ChatGPT in Brand Storytelling. (2023).
[47]
Yichen Wang, Kevin Yang, Xiaoming Liu, and Dan Klein. 2023. Improving Pacing in Long-Form Story Planning. arXiv preprint arXiv:2311.04459 (2023).
[48]
Christian Weiß. 2011. V2X communication in Europe–From research projects towards standardization and field testing of vehicle communication technology. Computer Networks 55, 14 (2011), 3103–3119.
[49]
Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, and Douglas C Schmidt. 2023. A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382 (2023).
[50]
Polly W. Wiessner. [n. d.]. Embers of society: Firelight talk among the Ju/’hoansi Bushmen. 111, 39 ([n. d.]), 14027–14035. https://doi.org/10.1073/pnas.1404212111 Publisher: Proceedings of the National Academy of Sciences.
[51]
Bob G. Witmer and Michael J. Singer. [n. d.]. Measuring Presence in Virtual Environments: A Presence Questionnaire. 7, 3 ([n. d.]), 225–240. https://doi.org/10.1162/105474698565686
[52]
Daijin Yang, Yanpeng Zhou, Zhiyuan Zhang, Toby Jia-Jun Li, and Ray LC. 2022. AI as an Active Writer: Interaction strategies with generated text in human-AI collaborative fiction writing. In Joint Proceedings of the ACM IUI Workshops, Vol. 10. CEUR-WS Team.
[53]
Kevin Yang, Dan Klein, Nanyun Peng, and Yuandong Tian. 2022. Doc: Improving long story coherence with detailed outline control. arXiv preprint arXiv:2212.10077 (2022).
[54]
Kevin Yang, Yuandong Tian, Nanyun Peng, and Dan Klein. [n. d.]. Re3: Generating Longer Stories With Recursive Reprompting and Revision. https://doi.org/10.48550/arXiv.2210.06774 arxiv:2210.06774 [cs]
[55]
Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. 2019. Plan-and-write: Towards better automatic storytelling. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 7378–7385.
[56]
Min-Hsuan Yeh, Vincent Chen, Ting-Hao Huang, and Lun-Wei Ku. [n. d.]. Multi-VQG: Generating Engaging Questions for Multiple Images. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (Abu Dhabi, United Arab Emirates, 2022-12), Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (Eds.). Association for Computational Linguistics, 277–290. https://doi.org/10.18653/v1/2022.emnlp-main.19
[57]
Zheng Zhang, Ying Xu, Yanhao Wang, Bingsheng Yao, Daniel Ritchie, Tongshuang Wu, Mo Yu, Dakuo Wang, and Toby Jia-Jun Li. 2022. Storybuddy: A human-ai collaborative chatbot for parent-child interactive storytelling with flexible parental involvement. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–21.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
UIST '24: Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology
October 2024
2334 pages
ISBN:9798400706288
DOI:10.1145/3654777
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 October 2024

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Auditive Interfaces
  2. Driving Experience
  3. Large-Language-Models
  4. Storytelling

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

UIST '24

Acceptance Rates

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 311
    Total Downloads
  • Downloads (Last 12 months)311
  • Downloads (Last 6 weeks)74
Reflects downloads up to 06 Jan 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media