Skip to main content
Springer Nature Link
Log in

Narrative Cognition in Mixed Reality Systems: Towards an Empirical Framework

  • Conference paper
  • First Online:
Virtual, Augmented and Mixed Reality (HCII 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12770))

Included in the following conference series:

  • 3611 Accesses

Abstract

In this paper, we propose an interdisciplinary theoretical and empirical framework to investigate the particular faculties related to human “narrative cognition”, in general, and in relation to MRT in particular. In order to contextualize our approach, we shortly review the cognitive turn in narratology, as well as state of the art in different domains that have undertaken psychophysiological studies that either characterize aspects that are relevant to narrative cognition, or which investigate mixed reality experiences. The idea is to bring together knowledge and insights from narratology, different branches of semiotics and cognitive sciences, with empirical strategies that bridge the gap between first-person phenomenological approaches and psychophysiological and behavioural methods. We propose a rationale in order to combine tools and techniques from MRT/VR/AR, interactive digital narratives and storytelling, with a suite of integrated psychophysiological methods (such as EEG, HR, GSR and eye tracking) and phenomenological-subjective approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Broniatowski, D.A., Reyna, V.F.: Gist and verbatim in narrative memory. In: Finlayson, M.A., Fisseni, B., Löwe, B., Meister, J.C. (eds.) 2013 Workshop on Computational Models of Narrative, pp. 43–51. Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik, Dagstuhl (2013). https://doi.org/10.4230/OASIcs.CMN.2013.43

  2. Bolioli, A., Casu, M., Lana, M., Roda, R.: Exploring the betrothed lovers. In: Finlayson, M.A., Fisseni, B., Löwe, B., Meister, J.C. (eds.) 2013 Workshop on Computational Models of Narrative, pp. 30–35. Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik, Dagstuhl (2013). https://doi.org/10.4230/OASIcs.CMN.2013.30

  3. Bhatt, M., Suchan, J., Schultz, C.: Cognitive interpretation of everyday activities: toward perceptual narrative based visuo-spatial scene interpretation. In: Workshop on Computational Models of Narrative, pp. 24–29. Leibniz-Zentrum für Informatik (2013). https://doi.org/10.4230/OASIcs.CMN.2013.24

  4. Damiano, R., Lieto, A.: Ontological representations of narratives: a case study on stories and actions. In: Proceedings of 2013 Workshop on Computational Models of Narrative, pp. 76–93 (2013). https://doi.org/10.4230/OASIcs.CMN.2013.76

  5. Dumas, J.E., Szilas, N., Richle, U., Boggini, T.: Interactive simulations to help teenagers cope when a parent has a traumatic brain injury. Comput. Entertain. 8, 1–3 (2011). https://doi.org/10.1145/1899687.1899692

    Article  Google Scholar 

  6. Szilas, N.: Interactive drama on computer: beyond linear narrative. In: AAAI Fall Symposium on Narrative Intelligence, pp. 150–156 (1999)

    Google Scholar 

  7. Schoenau-Fog, H., Bruni, L.E., Khalil, F.F., Faizi, J.: First person victim: developing a 3D interactive dramatic experience. In: Aylett, R., Lim, M.Y., Louchart, S., Petta, P., Riedl, M. (eds) Interactive Storytelling, ICIDS 2010, Lecture Notes in Computer Science, vol. 6432. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-16638-9_32

  8. Baceviciute, S., Rützou Albæk, K.R., Arsovski, A., Bruni, L.E.: Digital interactive narrative tools for facilitating communication with children during counseling: a case for audiology. In: Oyarzun, D., Peinado, F., Young, R.M., Elizalde, A., Méndez, G. (eds) Interactive Storytelling, ICIDS 2012, Lecture Notes in Computer Science, vol. 7648. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-34851-8_5

  9. Mulholland, P., Wolff, A., Zdrahal, Z., Li, N., Corneli, J.: Constructing and connecting storylines to tell museum stories. In: Koenitz, H., Sezen, T.I., Ferri, G., Haahr, M., Sezen, D., Çatak, G. (eds) Interactive Storytelling, ICIDS 2013, Lecture Notes in Computer Science, vol. 8230. Springer, Cham (2013). https://doi.org/10.1007/978-3-319-02756-2_14

  10. Kalogeras, S.: Media-education convergence: applying transmedia storytelling edutainment in e-learning environments. Int. J. Inf. Commun. Technol. Educ. 9, 1–11 (2013)

    Article  Google Scholar 

  11. Marsh, T., Nickole, L.Z., Klopfer, E., Xuejin, C., Osterweil, S., Haas, J.: Fun and learning: blending design and development dimensions in serious games through narrative and characters. In: Ma, M., Oikonomou, A., Jain, L. (eds.) Serious Games and Edutainment Applications. Springer, London (2011). https://doi.org/10.1007/978-1-4471-2161-9_14

  12. Heiden, W., Räder, M., Fassbender, E.: Interactive storytelling in academic teaching. In: Aylett, R., Lim, M.Y., Louchart, S., Petta, P., Riedl, M. (eds.) Interactive Storytelling, pp. 216–221. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-16638-9_27

    Chapter  Google Scholar 

  13. Guger, C., Holzner, C., Grönegress, C., Edlinger, G., Slater, M.: Control of a Smart Home with a Brain-Computer Interface (2008)

    Google Scholar 

  14. Si, M., Marsella, S., Miller, L.: Interactive stories for health interventions. In: Aylett, R., Lim, M.Y., Louchart, S., Petta, P., Riedl, M. (eds.) Interactive Storytelling, pp. 291–292. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-16638-9_46

    Chapter  Google Scholar 

  15. Szilas, N., Richle, U., Boggini, T., Dumas, J.: Using highly interactive drama to help young people cope with traumatic situations. In: Aylett, R., Lim, M.Y., Louchart, S., Petta, P., Riedl, M. (eds.) Interactive Storytelling, pp. 279–282. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-16638-9_42

    Chapter  Google Scholar 

  16. Barthes, R., Duisit, L.: An introduction to the structural analysis of narrative. New Lit. Hist. 6, 237 (1975). https://doi.org/10.2307/468419

    Article  Google Scholar 

  17. Bruner, J.: Acts of Meaning. Harvard University Press, Cambridge (1990)

    Google Scholar 

  18. Polkinghorne, D.E.: Narrative Knowing and the Human Sciences. Suny Press, Albany (1988)

    Google Scholar 

  19. Ricoeur, P.: Time and Narrative. University of Chicago Press, Chicago (1984)

    Google Scholar 

  20. Finlayson, M.A., et al.: Workshop on computational models of narrative. In: CMN 2013, Saarbrücken/WadernSchloss Dagstuhl-Leibniz-Zentrum für Informatik, Hamburg (2013)

    Google Scholar 

  21. Sanford, A.J., Emmott, C.: Mind, Brain and Narrative. Cambridge University Press, Cambridge (2012)

    Book  Google Scholar 

  22. Ryan, M.-L.: Narrative as Virtual Reality: Immersion and Interactivity in Literature and Electronic Media. Johns Hopkins University Press, Baltimore (2001)

    Google Scholar 

  23. Ryan, M.-L.: Avatars of Story. U of Minnesota Press, Minneapolis (2006)

    Google Scholar 

  24. Herman, D.: Cognitive narratology. In: Handbook of Narratology, pp. 46–64. de Gruyter, Berlin (2009)

    Google Scholar 

  25. Jahn, M.: Frames, preferences, and the reading of third-person narratives: towards a cognitive narratology. Poet. Today. 18, 441 (1997). https://doi.org/10.2307/1773182

    Article  Google Scholar 

  26. Cobley, P.: Narrative. Routledge, London (2014)

    Google Scholar 

  27. Zlatev, J.: Cognitive semiotics: an emerging field for the transdisciplinary study of meaning. Public J. Semiot. 4, 2–24 (2012). https://doi.org/10.37693/pjos.2012.4.8837

    Article  Google Scholar 

  28. Ranta, M.: Stories in pictures (and non-pictorial objects): a narratological and cognitive psychological approach. Contemp. Aesthetics 9, 6 (2011)

    Google Scholar 

  29. Matuk, C.F.: Narratives in mind and media: a cognitive semiotic account of novices interpreting visual science media (2010)

    Google Scholar 

  30. Lee, Y.: Narrative cognition and modeling in new media communication from Peirce’s semiotic perspective. Semiotica 2012, 181–195 (2012). https://doi.org/10.1515/sem-2012-0029

    Article  Google Scholar 

  31. Bruni, L.E.: Heterarchical semiosis: from signal transduction to narrative intelligibility. In: Trifonas, P.P. (ed.) International Handbook of Semiotics, pp. 1079–1097. Springer, Netherlands (2015). https://doi.org/10.1007/978-94-017-9404-6_49

  32. Bruni, L.E., Baceviciute, S.: On the embedded cognition of non-verbal narratives. Sign. Syst. Stud. 42, 359–375 (2014). https://doi.org/10.12697/SSS.2014.42.2-3.09

    Article  Google Scholar 

  33. Ryan, M.L.: Narratology and cognitive science: a problematic relation. Style. 44, 469–495 (2010)

    Google Scholar 

  34. Defense Advanced Research Projects Agency: Narrative Networks. https://www.darpa.mil/program/narrative-networks. Accessed 10 Feburary 2020

  35. Bruni, L.E., Baceviciute, S., Arief, M.: Narrative cognition in interactive systems: suspense-surprise and the P300 ERP component. In: Mitchell, A., Fernández-Vara, C., Thue, D. (eds.) Lecture Notes in Computer Science, pp. 164–175. Springer International Publishing, Cham (2014). https://doi.org/10.1007/978-3-319-12337-0_17

  36. Speer, N.K., Reynolds, J.R., Swallow, K.M., Zacks, J.M.: Reading stories activates neural representations of visual and motor experiences. Psychol. Sci. 20, 989–999 (2009). https://doi.org/10.1111/j.1467-9280.2009.02397.x

    Article  Google Scholar 

  37. Yarkoni, T., Speer, N.K., Zacks, J.M.: Neural substrates of narrative comprehension and memory. Neuroimage 41, 1408–1425 (2008). https://doi.org/10.1016/j.neuroimage.2008.03.062

    Article  Google Scholar 

  38. Mar, R.A.: The neuropsychology of narrative: story comprehension, story production and their interrelation. Neuropsychologia 42, 1414–1434 (2004). https://doi.org/10.1016/j.neuropsychologia.2003.12.016

    Article  Google Scholar 

  39. Zacks, J.M., Speer, N.K., Swallow, K.M., Maley, C.J.: The brain’s cutting-room floor: segmentation of narrative cinema. Front. Hum. Neurosci. 4 (2010). https://doi.org/10.3389/fnhum.2010.00168

  40. Lambrecht, J., Spring, D.K., Münte, T.F.: The focus of attention at the virtual cocktail party—electrophysiological evidence. Neurosci. Lett. 489, 53–56 (2011). https://doi.org/10.1016/j.neulet.2010.11.066

    Article  Google Scholar 

  41. Münte, T.F., Spring, D.K., Szycik, G.R., Noesselt, T.: Electrophysiological attention effects in a virtual cocktail-party setting. Brain Res. 1307, 78–88 (2010). https://doi.org/10.1016/j.brainres.2009.10.044

    Article  Google Scholar 

  42. Nager, W., Dethlefsen, C., Münte, T.F.: Attention to human speakers in a virtual auditory environment: Brain potential evidence. Brain Res. 1220, 164–170 (2008). https://doi.org/10.1016/j.brainres.2008.02.058

    Article  Google Scholar 

  43. Jaeger, M., Mirkovic, B., Bleichner, M.G., Debener, S.: Decoding the attended speaker from EEG using adaptive evaluation intervals captures fluctuations in attentional listening. Front. Neurosci. 14, 1–16 (2020). https://doi.org/10.3389/fnins.2020.00603

    Article  Google Scholar 

  44. Stuldreher, I.V., Thammasan, N., van Erp, J.B.F., Brouwer, A.-M.: Physiological synchrony in EEG, electrodermal activity and heart rate detects attentionally relevant events in time. Front. Neurosci. 14, 1–11 (2020). https://doi.org/10.3389/fnins.2020.575521

    Article  Google Scholar 

  45. Shavit-Cohen, K., Zion Golumbic, E.: The dynamics of attention shifts among concurrent speech in a naturalistic multi-speaker virtual environment. Front. Hum. Neurosci. 13, 1–12 (2019). https://doi.org/10.3389/fnhum.2019.00386

  46. Di Liberto, G.M., Nie, J., Yeaton, J., Khalighinejad, B., Shamma, S.A., Mesgarani, N.: Neural representation of linguistic feature hierarchy reflects second-language proficiency. Neuroimage 227, 117586 (2021). https://doi.org/10.1016/j.neuroimage.2020.117586

    Article  Google Scholar 

  47. Jospe, K., Genzer, S., Klein Selle, N., Ong, D., Zaki, J., Perry, A.: The contribution of linguistic and visual cues to physiological synchrony and empathic accuracy. Cortex. 132, 296–308 (2020). https://doi.org/10.1016/j.cortex.2020.09.001

  48. Ju, Y.S., Hwang, J.S., Kim, S.J., Suk, H.J.: Study of eye gaze and presence effect in virtual reality. In: Communications in Computer and Information Science, pp. 446–449. Springer International Publishing, Heidelberg (2019). https://doi.org/10.1007/978-3-030-23528-4_60

  49. Bruni, L.E., Baceviciute, S.: Narrative intelligibility and closure in interactive systems. In: Koenitz, H., Sezen, T.I., Ferri, G., Haahr, M., Sezen, D., Çatak, G. (eds.) Interactive Storytelling, pp. 13–24. Springer International Publishing, Cham (2013). https://doi.org/10.1007/978-3-319-02756-2_2

  50. Duncan, C.C., et al.: Event-related potentials in clinical research: guidelines for eliciting, recording, and quantifying mismatch negativity, P300, and N400. Clin. Neurophysiol. 120, 1883–1908 (2009). https://doi.org/10.1016/j.clinph.2009.07.045

    Article  Google Scholar 

  51. McPherson, W.B., Holcomb, P.J.: An electrophysiological investigation of semantic priming with pictures of real objects. Psychophysiology 36, 53–65 (1999). https://doi.org/10.1017/S0048577299971196

    Article  Google Scholar 

  52. West, W.C., Holcomb, P.J.: Event-related potentials during discourse-level semantic integration of complex pictures. Cogn. Brain Res. 13, 363–375 (2002). https://doi.org/10.1016/S0926-6410(01)00129-X

    Article  Google Scholar 

  53. Proverbio, A.M., Riva, F.: RP and N400 ERP components reflect semantic violations in visual processing of human actions. Neurosci. Lett. 459, 142–146 (2009). https://doi.org/10.1016/j.neulet.2009.05.012

    Article  Google Scholar 

  54. Sitnikova, T., Holcomb, P.J., Kiyonaga, K.A., Kuperberg, G.R.: Two neurocognitive mechanisms of semantic integration during the comprehension of visual real-world events. J. Cogn. Neurosci. 20, 2037–2057 (2008). https://doi.org/10.1162/jocn.2008.20143

    Article  Google Scholar 

  55. Hein, G., Doehrmann, O., Muller, N.G., Kaiser, J., Muckli, L., Naumer, M.J.: Object familiarity and semantic congruency modulate responses in cortical audiovisual integration areas. J. Neurosci. 27, 7881–7887 (2007). https://doi.org/10.1523/JNEUROSCI.1740-07.2007

    Article  Google Scholar 

  56. Federmeier, K.: Picture the difference: electrophysiological investigations of picture processing in the two cerebral hemispheres. Neuropsychologia 40, 730–747 (2002). https://doi.org/10.1016/S0028-3932(01)00193-2

    Article  Google Scholar 

  57. Wu, Y.C., Coulson, S.: Meaningful gestures: electrophysiological indices of iconic gesture comprehension. Psychophysiology 42, 654–667 (2005). https://doi.org/10.1111/j.1469-8986.2005.00356.x

    Article  Google Scholar 

  58. Sitnikova, T., Kuperberg, G., Holcomb, P.J.: Semantic integration in videos of real–world events: an electrophysiological investigation. Psychophysiology 40, 160–164 (2003). https://doi.org/10.1111/1469-8986.00016

    Article  Google Scholar 

  59. Chao, L.L., Nielsen-Bohlman, L., Knight, R.T.: Auditory event-related potentials dissociate early and late memory processes. Electroencephalogr. Clin. Neurophysiol. Potentials Sect. 96, 157–168 (1995). https://doi.org/10.1016/0168-5597(94)00256-E.

  60. Cohn, N., Paczynski, M., Holcomb, P., Jackendoff, R., Kuperberg, G.: Comics on the brain: structure and meaning in sequential image comprehension. In: Proceedings of the Annual Meeting of the Cognitive Science Society (2011)

    Google Scholar 

  61. Cohn, N., Paczynski, M., Jackendoff, R., Holcomb, P.J., Kuperberg, G.R.: (Pea)nuts and bolts of visual narrative: structure and meaning in sequential image comprehension. Cogn. Psychol. 65, 1–38 (2012). https://doi.org/10.1016/j.cogpsych.2012.01.003

    Article  Google Scholar 

  62. Barrett, S.E., Rugg, M.D.: Event-related potentials and the semantic matching of pictures. Brain Cogn. 14, 201–212 (1990). https://doi.org/10.1016/0278-2626(90)90029-N

    Article  Google Scholar 

  63. Hamm, J.P., Johnson, B.W., Kirk, I.J.: Comparison of the N300 and N400 ERPs to picture stimuli in congruent and incongruent contexts. Clin. Neurophysiol. 113, 1339–1350 (2002). https://doi.org/10.1016/S1388-2457(02)00161-X

    Article  Google Scholar 

  64. Gilroy, S.W., et al.: A brain-computer interface to a plan-based narrative. In: Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, pp. 1997–2005. AAAI Press (2013)

    Google Scholar 

  65. Ekman, I., et al.: Review on psychophysiological methods in game research. In: Proceedings of the 2010 International DiGRA Nordic Conference: Experiencing Games: Games, Play, and Players, DiGRA Nordic 2010 (2010)

    Google Scholar 

  66. Nijholt, A.: BCI for games: a ‘state of the art’ survey. In: Stevens, S.M., Saldamarco, S.J. (eds.) Lecture Notes in Computer Science, pp. 225–228. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-89222-9_29

  67. Gjøl, B.A., Jørgensen, N.V., Thomsen, M.R., Bruni, L.E.: Predictability and plausibility in interactive narrative constructs: a case for an ERP study. In: Rouse, R., Koenitz, H., Haahr, M. (eds.) Interactive Storytelling, pp. 121–133. Springer International Publishing, Cham (2018). https://doi.org/10.1007/978-3-030-04028-4_9

  68. Coderre, E.L., O’Donnell, E., O’Rourke, E., Cohn, N.: Predictability modulates neurocognitive semantic processing of non-verbal narratives. Sci. Rep. 10, 10326 (2020). https://doi.org/10.1038/s41598-020-66814-z

    Article  Google Scholar 

  69. Coco, M.I., Araujo, S., Petersson, K.M.: Disentangling stimulus plausibility and contextual congruency: electro-physiological evidence for differential cognitive dynamics. Neuropsychologia 96, 150–163 (2017). https://doi.org/10.1016/j.neuropsychologia.2016.12.008

    Article  Google Scholar 

  70. Ki, J.J., Kelly, S.P., Parra, L.C.: Attention strongly modulates reliability of neural responses to naturalistic narrative stimuli. J. Neurosci. 36, 3092–3101 (2016). https://doi.org/10.1523/JNEUROSCI.2942-15.2016

    Article  Google Scholar 

  71. Cohen, S.S., Parra, L.C.: Memorable audiovisual narratives synchronize sensory and supramodal neural responses. Eneuro. 3, ENEURO.0203–16.2016 (2016). https://doi.org/10.1523/ENEURO.0203-16.2016

  72. Wang, R.W.Y., Chang, Y.-C., Chuang, S.-W.: EEG spectral dynamics of video commercials: impact of the narrative on the branding product preference. Sci. Rep. 6, 36487 (2016). https://doi.org/10.1038/srep36487

    Article  Google Scholar 

  73. Chun, C., Park, B., Shi, C.: Re-living suspense: emotional and cognitive responses during repeated exposure to suspenseful film. Front. Psychol. 11, 2786 (2020). https://doi.org/10.3389/fpsyg.2020.558234

    Article  Google Scholar 

  74. Foulsham, T., Wybrow, D., Cohn, N.: Reading without words: eye movements in the comprehension of comic strips. Appl. Cogn. Psychol. 30, 566–579 (2016). https://doi.org/10.1002/acp.3229

    Article  Google Scholar 

  75. Barraza, J.A., Alexander, V., Beavin, L.E., Terris, E.T., Zak, P.J.: The heart of the story: peripheral physiology during narrative exposure predicts charitable giving. Biol. Psychol. 105, 138–143 (2015). https://doi.org/10.1016/j.biopsycho.2015.01.008

    Article  Google Scholar 

  76. Jääskeläinen, I.P., Sams, M., Glerean, E., Ahveninen, J.: Movies and narratives as naturalistic stimuli in neuroimaging. Neuroimage 224, 117445 (2021). https://doi.org/10.1016/j.neuroimage.2020.117445

    Article  Google Scholar 

  77. Rothe, S., Hollerer, T., Hubmann, H.: CVR-analyzer: a tool for analyzing cinematic virtual reality viewing patterns. In: 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 403–404. IEEE (2018). https://doi.org/10.1109/ISMAR-Adjunct.2018.00117

  78. Rothe, S., Pothmann, P., Drewe, H., Hussmann, H.: Interaction techniques for cinematic virtual reality. In: 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 1733–1737. IEEE (2019). https://doi.org/10.1109/VR.2019.8798189

  79. Rothe, S., L. Chuang, L.: Implications of eye tracking research to cinematic virtual reality. In: Symposium on Eye Tracking Research and Applications, pp. 1–3. ACM, New York (2020). https://doi.org/10.1145/3379157.3391658

  80. Ding, N., Zhou, W., Fung, A.Y.H.: Emotional effect of cinematic VR compared with traditional 2D film. Telematics Inform. 35, 1572–1579 (2018). https://doi.org/10.1016/j.tele.2018.04.003

    Article  Google Scholar 

  81. Marwecki, S., Wilson, A.D., Ofek, E., Gonzalez Franco, M., Holz, C.: Mise-unseen: using eye-tracking to hide virtual reality scene changes in plain sight. In: Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, pp. 777–789. ACM, New York (2019). https://doi.org/10.1145/3332165.3347919

  82. Škola, F., et al.: Virtual reality with 360-video storytelling in cultural heritage: study of presence, engagement, and immersion. Sensors. 20, 5851 (2020). https://doi.org/10.3390/s20205851

    Article  Google Scholar 

  83. Kwok, T.C.K., Kiefer, P., Schinazi, V.R., Adams, B., Raubal, M.: Gaze-guided narratives. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–12. ACM, New York (2019). https://doi.org/10.1145/3290605.3300721

  84. Bee, N., et al.: Discovering eye gaze behavior during human-agent conversation in an interactive storytelling application. In: International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction on - ICMI-MLMI 2010, p. 1. ACM Press, New York (2010). https://doi.org/10.1145/1891903.1891915

  85. Gere, C.: Digital Culture. Reaktion Books, London (2009)

    Google Scholar 

  86. Rhumbo. http://rhumbo.eu/webs/rhumbo/. Accessed 01 Feburary 2021

  87. Bruni, L.E.: Hierarchical categorical perception in sensing and cognitive processes. Biosemiotics 1, 113–130 (2008). https://doi.org/10.1007/s12304-008-9001-9

    Article  Google Scholar 

  88. Bruni, L.E.: Semiotic freedom: emergence and teleology in biological and cognitive interfaces. Am. J. Semiot. 24, 57–73 (2008). https://doi.org/10.5840/ajs2008241/35

    Article  Google Scholar 

  89. Vincent, D., Perrin, L.: On the narrative vs non-narrative functions of reported speech: a socio-pragmatic study. J. Socioling. 3, 291–313 (1999). https://doi.org/10.1111/1467-9481.00080

    Article  Google Scholar 

  90. Goodman, N.: Twisted tales: story, study and symphony. In: Mitchell, W.J.T. (ed.) Narrative, Chicago (1981)

    Google Scholar 

  91. Hutto, D.D.: Narrative and understanding persons. R. Inst. Philos. Suppl. 60, 1–16 (2007). https://doi.org/10.1017/S135824610700001X

    Article  Google Scholar 

  92. Strawson, G.: Against narrativity. Ratio 17, 428–452 (2004). https://doi.org/10.1111/j.1467-9329.2004.00264.x

    Article  Google Scholar 

  93. Herman, D.: Basic Elements of Narrative. Wiley, New York (2009)

    Book  Google Scholar 

  94. Goodson, I., Gill, S.R.: Narrative Pedagogy: Life History and Learning. Peter Lang, Bern (2011)

    Google Scholar 

  95. White, M., Wijaya, M., White, M.K., Epston, D.: Narrative Means to Therapeutic Ends. WW Norton & Company, New York (1990)

    Google Scholar 

  96. Kleinreesink, E., Moelker, R., Richardson, R.: Books and Bikes. Amsterdam University Press, Amsterdam (2012)

    Book  Google Scholar 

  97. Corman, S.R.: Narrating the Exit from Afghanistan. Tempe: Center for Strategic Communication (2013)

    Google Scholar 

  98. Varela, F.J., Rosch, E., Thompson, E.: The Embodied Mind. MIT Press, Cambridge (1993)

    Google Scholar 

Download references

Acknowledgments

The authors acknowledge the financial support of Rhumbo (European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie Grant Agreement No 813234).

Author information

Authors and Affiliations

  1. The Augmented Cognition Lab, Aalborg University, 2450, Copenhagen, Denmark

    Luis Emilio Bruni, Hossein Dini & Aline Simonetti

  2. Department of Market and Marketing Research, University of Valencia, 46022, Valencia, Spain

    Aline Simonetti

Authors
  1. Luis Emilio Bruni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hossein Dini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aline Simonetti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luis Emilio Bruni .

Editor information

Editors and Affiliations

  1. U.S. Army Research Laboratory, Aberdeen Proving Ground, MD, USA

    Jessie Y. C. Chen

  2. U.S. Army Combat Capabilities Development Command Soldier Center, Orlando, FL, USA

    Gino Fragomeni

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bruni, L.E., Dini, H., Simonetti, A. (2021). Narrative Cognition in Mixed Reality Systems: Towards an Empirical Framework. In: Chen, J.Y.C., Fragomeni, G. (eds) Virtual, Augmented and Mixed Reality. HCII 2021. Lecture Notes in Computer Science(), vol 12770. Springer, Cham. https://doi.org/10.1007/978-3-030-77599-5_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-77599-5_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-77598-8

  • Online ISBN: 978-3-030-77599-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics