Skip to main content
SpringerLink
Log in

Visualizing a User’s Cognitive and Emotional Journeys: A Fintech Case

  • Conference paper
  • First Online:
Design, User Experience, and Usability. Interaction Design (HCII 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12200))

Included in the following conference series:

Abstract

In this article, we propose a visualization approach that presents the user’s cognitive and emotional states in conjunction with the actual journey of the user on a web interface. Specifically, we have designed a new visualization method which contextualizes the user’s physiological and behavioral data while interacting with a web-based information system in the financial services industry. The proposed approach brings together the user’s behavior with his/her cognitive and emotional states to produce a rich overview of his/her experience. Combining these methods produces key insights into the user experience and facilitates an understanding of the evolution of the experience since it highlights where the user was on the interface when s/he experienced a given cognitive and emotional state. Results from an illustrative case suggest that the proposed visualization method is useful in conveying where participants deviate from the optimal path and facilitates the identification of usability issues on web interfaces.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Rust, R.T., Lemon, K.N.: E-service and the consumer. Int. J. Electron. Commer. 5(3), 85–101 (2001)

    Article  Google Scholar 

  2. Gopalani, A.: The service-enabled customer experience: a jump-start to competitive advantage. J. Bus. Strategy 32(3), 4–12 (2011)

    Article  Google Scholar 

  3. Pine, B.J., Gilmore, J.H.: The Experience Economy. Harvard Business Press, Cambridge (2011)

    Google Scholar 

  4. De Keyser, A., et al.: A framework for understanding and managing the customer experience. Mark. Sci. Inst. Working Pap. Ser. 15(121), 1–48 (2015)

    Google Scholar 

  5. Stickdorn, M., et al.: This is Service Design Thinking: Basics, Tools, Cases, vol. 1. Wiley, Hoboken (2011)

    Google Scholar 

  6. Rawson, A., Duncan, E., Jones, C.: The truth about customer experience. Harvard Bus. Rev. 91, 90–98 (2013)

    Google Scholar 

  7. Browne, J.: Customer Journey Mapping: What Is It For?. F.R. Inc., Editor (2012)

    Google Scholar 

  8. Bitner, M.J., Ostrom, A.L., Morgan, F.N.: Service blueprinting: a practical technique for service innovation. Calif. Manag. Rev. 50(3), 66–94 (2008)

    Article  Google Scholar 

  9. Patrício, L., Fisk, R.P., Falcão e Cunha, J.: Designing multi-interface service experiences: the service experience blueprint. J. Serv. Res. 10(4), 318–334 (2008)

    Article  Google Scholar 

  10. Verma, R., et al.: Customer experience modeling: from customer experience to service design. J. Serv. Manag. 23(3), 362–376 (2012)

    Article  Google Scholar 

  11. Lemon, K.N., Verhoef, P.C.: Understanding customer experience throughout the customer journey. J. Mark. 80(6), 69–96 (2016)

    Article  Google Scholar 

  12. Rosenbaum, M.S., Otalora, M.L., Ramírez, G.C.: How to create a realistic customer journey map. Bus. Horiz. 60(1), 143–150 (2017)

    Article  Google Scholar 

  13. Helkkula, A., Kelleher, C., Pihlström, M.: Characterizing value as an experience: implications for service researchers and managers. J. Serv. Res. 15(1), 59–75 (2012)

    Article  Google Scholar 

  14. Venkatraman, V., et al.: New scanner data for brand marketers: how neuroscience can help better understand differences in brand preferences. J. Consum. Psychol. 22(1), 143–153 (2012)

    Article  Google Scholar 

  15. Lewinski, P.: Automated facial coding software outperforms people in recognizing neutral faces as neutral from standardized datasets. Front. Psychol. 6, 1386 (2015)

    Google Scholar 

  16. Plassmann, H., et al.: Consumer neuroscience: applications, challenges, and possible solutions. J. Mark. Res. 54, 427–435 (2015)

    Article  Google Scholar 

  17. Schmitt, B.: Experiential marketing. J. Mark. Manag. 15(1–3), 53–67 (1999)

    Article  Google Scholar 

  18. Verhoef, P.C., et al.: Customer experience creation: determinants, dynamics and management strategies. J. Retail. 85(1), 31–41 (2009)

    Article  Google Scholar 

  19. Vom Brocke, J., Hevner, A., Léger, P. M., Walla, P., Riedl, R.: Advancing a Neurois Research Agenda with Four Areas of Societal Contributions. Eur. J. Inf. Syst. 29(1), 9–24 (2020)

    Article  Google Scholar 

  20. Neslin, S.A., et al.: Challenges and opportunities in multichannel customer management. J. Serv. Res. 9(2), 95–112 (2006)

    Article  Google Scholar 

  21. Segelström, F.: Stakeholder Engagement for Service Design: How Service Designers Identify and Communicate Insights. Linköping University Electronic Press, Linköping (2013)

    Book  Google Scholar 

  22. Norton, D.W., Pine, B.J.: Using the customer journey to road test and refine the business model. Strategy Leadersh. 41(2), 12–17 (2013)

    Article  Google Scholar 

  23. Halvorsrud, R., Kvale, K., Følstad, A.: Improving service quality through customer journey analysis. J. Serv. Theory Pract. 26(6), 840–867 (2016)

    Article  Google Scholar 

  24. Holmlid, S., Evenson, S.: Bringing service design to service sciences, management and engineering. In: Hefley, B., Murphy, W. (eds.) Service Science, Management and Engineering Education for the 21st Century. SSRI, pp. 341–345. Springer, Boston (2008). https://doi.org/10.1007/978-0-387-76578-5_50

    Chapter  Google Scholar 

  25. Temkin, B.D.: Mapping the Customer Journey: Best Practices for Using an Important Customer Experience Tool. Forrester Research Inc., Cambridge (2010)

    Google Scholar 

  26. Marquez, J.J., Downey, A., Clement, R.: Walking a mile in the user’s shoes: customer journey mapping as a method to understanding the user experience. Internet Ref. Serv. Q. 20(3–4), 135–150 (2015)

    Article  Google Scholar 

  27. Tseng, M.M., Qinhai, M., Su, C.J.: Mapping customers’ service experience for operations improvement. Bus. Process Manag. J. (1999)

    Google Scholar 

  28. Dirican, A.C., Göktürk, M.: Psychophysiological measures of human cognitive states applied in human computer interaction. Procedia Comput. Sci. 3, 1361–1367 (2011)

    Article  Google Scholar 

  29. Riedl, R., Léger, P.-M.: Fundamentals of NeuroIS. SNPBE. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-662-45091-8

    Book  Google Scholar 

  30. De Guinea, A.O., Webster, J.: An investigation of information systems use patterns: technological events as triggers, the effect of time and consequences for performance. MIS Q. 37(4), 1165–1188 (2013)

    Article  Google Scholar 

  31. De Guinea, A.O., Titah, R., Léger, P.-M.: Explicit and implicit antecedents of users’ behavioral beliefs in information systems: a neuropsychological investigation. J. Manag. Inf. Syst. 30(4), 179–210 (2014)

    Article  Google Scholar 

  32. Leger, P.-M., Riedl, R., vom Brocke, J.: Emotions and ERP information sourcing: the moderating role of expertise. Ind. Manag. Data Syst. 114, 456–471 (2014)

    Article  Google Scholar 

  33. Kahneman, D., Riis, J.: Living, and Thinking About It: Two Perspectives on Life in the Science of Well-Being, pp. 285–304. Oxford University Press, Oxford (2005)

    Google Scholar 

  34. Lourties, S., Léger, P.-M., Sénécal, S., Fredette, M., Chen, S.L.: Testing the convergent validity of continuous self-perceived measurement systems: an exploratory study. In: Nah, F.F.-H., Xiao, B.S. (eds.) HCIBGO 2018. LNCS, vol. 10923, pp. 132–144. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91716-0_11

    Chapter  Google Scholar 

  35. Ivonin, L., et al.: Beyond cognition and affect: sensing the unconscious. Behav. Inf. Technol. 34(3), 220–238 (2014)

    Article  Google Scholar 

  36. Zijlstra, F.R.H., et al.: Temporal factors in mental work: effects of interrupted activities. J. Occup. Organ. Psychol. 72, 163–185 (1999)

    Article  Google Scholar 

  37. Bailey, B.P., et al.: A framework for specifying and monitoring user tasks. Comput. Hum. Behav. 22(4), 709–732 (2006)

    Article  Google Scholar 

  38. Bailey, B.P., Konstan, J.A.: On the need for attention-aware systems: measuring effects of interruption on task performance, error rate, and affective state. Comput. Hum. Behav. 22(4), 685–708 (2006)

    Article  Google Scholar 

  39. Picard, R.W.: Affective Computing, in M.I.T Media Laboratory Perceptual Computing Section (1995)

    Google Scholar 

  40. Ward, R.D., Marsden, P.H.: Physiological responses to different WEB page designs. Int. J. Hum. Comput. Stud. 59(1–2), 199–212 (2003)

    Article  Google Scholar 

  41. Bentley, T., Johnson, L., von Baggo, K.: Evaluation using Cued-Recall Debrief to Elicit Information about a user’s affective experiences. In: OZCHI. Canberra, Australia (2005)

    Google Scholar 

  42. Giroux-Huppé, C., et al.: Impact d’une expérience psychophysiologique négative sur la satisfaction des consommateurs dans un contexte d’épicerie en ligne, in Marketing. HEC Montréal, Montréal (2019)

    Google Scholar 

  43. Guinea, A.O., Markus, L.: Why break the habit of a lifetime? Rethinking the roles of intention, habit, and emotion in continuing information technology use. MIS Q. 33(3), 433–444 (2009)

    Article  Google Scholar 

  44. Léger, P.-M., et al.: Neurophysiological correlates of cognitive absorption in an enactive training context. Comput. Hum. Behav. 34, 273–283 (2014)

    Article  Google Scholar 

  45. Russell, J.A.: A circumplex model of affect. J. Pers. Soc. Psychol. 39(6), 1161–1178 (1980)

    Article  Google Scholar 

  46. Boucsein, W.: Electrodermal Activity. Springer, Heidelberg (2012). https://doi.org/10.1007/978-1-4614-1126-0

    Book  Google Scholar 

  47. Forne, M.: Physiology as a Tool for UX and Usability Testing, in KTH Computer Science and Communication. Royal Institute of Technology, Stockholm (2012)

    Google Scholar 

  48. Colombetti, G.: Appraising valence. J. Conscious. Stud. 12(8–10), 103–126 (2005)

    Google Scholar 

  49. Maia, C.L.B., Furtado, E.S.: A Study about psychophysiological measures in user experience monitoring and evaluation. In: Proceedings of the 15th Brazilian Symposium on Human Factors in Computing Systems (2016)

    Google Scholar 

  50. Burton-Jones, A., Gallivan, M.J.: Toward a deeper understanding of system usage in organizations: a multilevel perspective. MIS Q. 31(4), 657–679 (2007)

    Article  Google Scholar 

  51. Den Uyl, M.J., Van Kuilenburg, H: The facereader: online facial expression recognition. In: Proceedings of Measuring Behavior (2005)

    Google Scholar 

  52. Haapalainen, E., et al.: Psycho-physiological measures for assessing cognitive load. In: Proceedings of the 12th ACM International Conference on Ubiquitous Computing. ACM (2010)

    Google Scholar 

  53. Charland, P., et al.: Assessing the multiple dimensions of engagement to characterize learning: a neurophysiological perspective. JoVE J. Visualized Exp. 101, e52627 (2015)

    Google Scholar 

  54. Beatty, J., Lucero-Wagoner, B.: The pupillary system. Handb. Psychophysiol. 2, 142–162 (2000)

    Google Scholar 

  55. Wilson, G.F.: An analysis of mental workload in pilots during flight using multiple psychophysiological measures. Int. J. Aviat. Psychol. 12(1), 3–18 (2002)

    Article  MathSciNet  Google Scholar 

  56. Ikehara, C.S., Crosby, M.E.: Assessing cognitive load with physiological sensors. In: Proceedings of the 38th Annual Hawaii International Conference on System Sciences. IEEE (2005)

    Google Scholar 

  57. Iqbal, S.T., et al.: Towards an index of opportunity: understanding changes in mental workload during task execution. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM (2005)

    Google Scholar 

  58. Léger, P.-M., Charland, P., Sénécal, S., Cyr, S.: Predicting properties of cognitive pupillometry in human–computer interaction: a preliminary investigation. In: Davis, F.D., Riedl, R., vom Brocke, J., Léger, P.-M., Randolph, A.B. (eds.) Information Systems and Neuroscience. LNISO, vol. 25, pp. 121–127. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-67431-5_14

    Chapter  Google Scholar 

  59. Desrochers, C., et al.: The arithmetic complexity of online grocery shopping: the moderating role of product pictures. Ind. Manag. Data Syst. 119(6), 1206–1222 (2019)

    Article  Google Scholar 

  60. Richardson, A.: Using customer journey maps to improve customer experience. Harvard Bus. Rev. 15(1), 2–5 (2010)

    Google Scholar 

  61. Coursaris, C.K., Kim, D.J.: A meta-analytical review of empirical mobile usability studies. J. Usability Stud. 6(3), 117–171 (2011)

    Google Scholar 

  62. Reichheld, F.F., Markey, R.: The Ultimate Question 2.0 How Net Promoter Companies Thrive in a CustomerDriven World. Harvard Business Review Press, Boson (2011)

    Google Scholar 

  63. Alvarez, J., et al.: Towards Agility and Speed in Enriched UX Evaluation Projects, in Human-Computer Interaction. IntechOpen, London (2019)

    Google Scholar 

  64. Vasseur, A., P.-M. Léger, Sénécal, S.: Eye-tracking for information systems research: a literature review. In: SIGHCI, vol. 8 (2019)

    Google Scholar 

  65. Van Gerven, P.W., et al.: Memory load and the cognitive pupillary response in aging. Psychophysiology 41(2), 167–174 (2004)

    Article  Google Scholar 

  66. Hyönä, J., Tommola, J., Alaja, A.-M.: Pupil dilation as a measure of processing load in simultaneous interpretation and other language tasks. Q. J. Exp. Psychol. Sect. A 48(3), 598–612 (2007)

    Article  Google Scholar 

  67. Hassenzahl, M., Tractinsky, N.: User experience - a research agenda. Behav. Inf. Technol. 25(2), 91–97 (2006)

    Article  Google Scholar 

  68. Braithwaite, J.J., et al.: A Guide for Analysing Electrodermal Activity (EDA) & Skin Conductance Responses (SCRs) for Psychological Experiments. U.o.B. Behavioural Brain Sciences Centre, Editor (2015)

    Google Scholar 

  69. Hassenzahl, M., Burmester, M., Koller, F.: AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In: Szwillus, G., Ziegler, J. (eds.) Mensch & Computer 2003. BGCACM, vol. 57, pp. 187–196. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-322-80058-9_19

    Chapter  Google Scholar 

  70. Stern, R.M., et al.: Psychophysiological Recording. Oxford University Press, USA (2001)

    Google Scholar 

  71. Ekman, P., Friesen, W.V.: Unmasking the face: A Guide to Recognizing Emotions from Facial Clues. M. Books, Editor (2003)

    Google Scholar 

  72. Lojiens, L., Krips, O.: FaceReader Methodology Note. N.I. Technology, Editor (2018)

    Google Scholar 

  73. Courtemanche, F., et al.: Method of and system for processing signals sensed from a user. Google Patents (2019)

    Google Scholar 

  74. Léger, P.-M., Courtemanche, F., Fredette, M., Sénécal, S.: A cloud-based lab management and analytics software for triangulated human-centered research. In: Davis, F.D., Riedl, R., vom Brocke, J., Léger, P.-M., Randolph, A.B. (eds.) Information Systems and Neuroscience. LNISO, vol. 29, pp. 93–99. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-01087-4_11

    Chapter  Google Scholar 

  75. Posner, J., Russell, J.A., Peterson, B.S.: The circumplex model of affect an integrative approach to affective neuroscience, cognitive development, and psychopathology. Dev. Psychopathol. 17(3), 714–734 (2005)

    Article  Google Scholar 

  76. Lallemand, C., Gronier, G.: Focus Group, in Méthodes de design UX. Eyrolles, Paris (2016)

    Google Scholar 

  77. Debus, M.: Handbook for excellence in focus group research. Academy for Educational Development, Durham (2007)

    Google Scholar 

  78. Anzieu, D., Martin, J.-Y.: La dynamique des groupes restreints. Presses Universitaires de France, Paris (1968)

    Google Scholar 

  79. Lamontagne, C., et al.: User test: how many users are needed to find the psychophysiological pain points in a journey map? In: Ahram, T., Taiar, R., Colson, S., Choplin, A. (eds.) IHIET 2019. AISC, vol. 1018, pp. 136–142. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-25629-6_22

    Chapter  Google Scholar 

  80. Gentile, C., Spiller, N., Noci, G.: How to Sustain the Customer Experience: An Overview of Experience Components That Co-Create Value with the Customer. Eur. Manage. J. 25(5), 395–410 (2007)

    Article  Google Scholar 

  81. Russell, J. A., Weiss, A., Mendelsohn, G. A.: Affect Grid: A Single-Item Scale of Pleasure and Arousal. J. Pers. Soc. Psychol. 57(3), 493 (1989)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. HEC Montréal, Montréal, H3T 2A7, Canada

    Marianne Veilleux, Sylvain Sénécal, Bertrand Demolin, Frédérique Bouvier, Constantinos Coursaris & Pierre-Majorique Léger

  2. Mouvement Desjardins, Montréal, H5B 1B4, Canada

    Marie-Laure Di Fabio

Authors
  1. Marianne Veilleux
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sylvain Sénécal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bertrand Demolin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Frédérique Bouvier
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marie-Laure Di Fabio
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Constantinos Coursaris
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Pierre-Majorique Léger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marianne Veilleux .

Editor information

Editors and Affiliations

  1. Aaron Marcus and Associates, Berkeley, CA, USA

    Aaron Marcus

  2. World Usability Day and Bentley User Experience Center, Newton Center, MA, USA

    Elizabeth Rosenzweig

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Veilleux, M. et al. (2020). Visualizing a User’s Cognitive and Emotional Journeys: A Fintech Case. In: Marcus, A., Rosenzweig, E. (eds) Design, User Experience, and Usability. Interaction Design. HCII 2020. Lecture Notes in Computer Science(), vol 12200. Springer, Cham. https://doi.org/10.1007/978-3-030-49713-2_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-49713-2_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-49712-5

  • Online ISBN: 978-3-030-49713-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation