research-article

Asserting Real-Time Emotions through Cued-Recall: Is it Valid?

Authors:

Effie Lai-Chong Law,

Matthias Heintz,

Poul Svante EriksenAuthors Info & Claims

NordiCHI '16: Proceedings of the 9th Nordic Conference on Human-Computer Interaction

Article No.: 37, Pages 1 - 10

https://doi.org/10.1145/2971485.2971516

Published: 23 October 2016 Publication History

Abstract

Asserting emotions through free-recall is commonly used to evaluate user experience (UX) of interactive systems. From psychology we know that free-recall of emotions leads to a significant memory bias where participants rely on a few of the most intense episodes when asserting an overall experience. It is argued that cued-recall can reduce the memory bias in UX evaluations. Yet, this has not been studied empirically. We present a systematic empirical study based on 38 participants. We measured emotions in terms of objective galvanic skin responses (GSR) and subjective Self-Assessment Manikin (SAM) ratings. We found significant correlations between emotions experienced in real-time and those experienced during cued-recall. This validates the use of cued-recall for UX evaluations. An implication is that HCI researchers and practitioners now have cued-recall as an alternative that significantly reduces the memory bias and enables highly detailed measurements of emotions while not disturbing participants during system interaction.

References

[1]

Andreassi, J.L. Psychophysiology: Human Behavior and Physiological Response. Lawrence Erlbaum, Mahwah, 2000.

[2]

Bargas-Avila, J.A. and Hornbæk, K. Old wine in new bottles or novel challenges. In Proc. CHI. ACM, New York, 2011, pp. 2689--2698.

Digital Library

[3]

Bentley, T., Johnston, L., and von Baggo, K. Evaluation Using Cued-recall Debrief to Elicit Information About a User's Affective Experiences. In Proc. OzCHI. CHISIG Australia, Narrabundah, 2005, pp. 1--10.

Digital Library

[4]

Bevan, N. Classifying and Selecting UX and Usability Measures. In Proc. Int. WS on Valid Useful User Experience Measurement (VUUM). IRIT, Toulouse, 2008.

[5]

Bruun, A. and Ahm, S. Mind the Gap! Comparing Retrospective and Concurrent Ratings of Emotion in User Experience Evaluation. In 15th IFIP TC13 Conference on Human-Computer Interaction (INTERACT). Springer-Verlag, 2015.

[6]

Buchanan, T.W. Retrieval of Emotional Memories. Psychological bulletin, 133, 5 (September 2007), 761--779.

[7]

Burmester, M., Mast, M., Jäger, K., and Homans, H. Valence Method for Formative Evaluation of User Experience. In Proc. DIS. ACM, New York, 2010, pp. 364--367.

Digital Library

[8]

Bylsma, L.M., Taylor-Clift, A., and Rottenberg, J. Emotional reactivity to daily events in major and minor depression. Journal of abnormal psychology, 120, 1 (February 2011), 155--167.

[9]

Cockburn, A., Quinn, P., and Gutwin, C. Examining the Peak-End Effects of Subjective Experience. In Proc. CHI'15. ACM, New York, NY, USA, 2015, pp. 357--366.

Digital Library

[10]

Cohen, J. Statistical power analysis for the behavioral sciences. Lawrence Earlbaum Associates, Hillsdale, NJ, 1988.

[11]

Csikszentmihalyi, M. and Larson, R. Validity and Reliability of the Experience-Sampling Method. In Flow and the Foundations of Positive Psychology SE - 3. Springer Netherlands, 2014, pp. 35--54.

[12]

Desmet, P. Funology. In M.A. Blythe, K. Overbeeke, A.F. Monk and P.C. Wright, eds., Kluwer Academic Publishers, Norwell, MA, USA, 2004, pp. 111--123.

Digital Library

[13]

Desmet, P.M.A., Overbeeke, K., and Tax, S. Designing products with added emotional value; development and application of an approach for research through design. The Design Journal, 4, 1 (2001), 32--47.

[14]

Diener, E. and Tay, L. Review of the Day Reconstruction Method (DRM). Social Indicators Research, 116, 1 (2014), 255--267.

[15]

Ekkekakis, P. The Measurement of Affect, Mood, and Emotion. Cambridge University Press, Cambridge, 2013.

[16]

Goldberg, L.R. The development of markers for the Big-Five factor structure. Psychological Assessment, 4, 1 (1992), 26--42.

[17]

Hassenzahl, M. and Ullrich, D. To Do or Not to Do: Differences in User Experience and Retrospective Judgments Depending on the Presence or Absence of Instrumental Goals. Interact. Comput., 19, 4 (2007), 429--437.

Digital Library

[18]

James, W. What Is An Emotion? Mind, os-IX, 34 (1884), 188--205.

[19]

Kahneman, D., Fredrickson, B.L., Schreiber, C.A., and Redelmeier, D.A. When More Pain Is Preferred to Less: Adding a Better End. Psychological Science, 4, 6 (November 1993), 401--405.

[20]

Kujala, S. and Miron-Shatz, T. Emotions, Experiences and Usability in Real-life Mobile Phone Use. In Proc. CHI. ACM, New York, 2013, pp. 1061--1070.

Digital Library

[21]

Kujala, S., Roto, V., Väänänen-Vainio-Mattila, K., Karapanos, E., and Sinnelä, A. {UX} Curve: A method for evaluating long-term user experience. Interacting with Computers, 23, 5 (2011), 473--483.

Digital Library

[22]

Lang, P.J. Behavioral treatment and bio-behavioral assessment: computer applications. In J.B. Sidowski, J.H. Johnson and T.H. Williams, eds., Technology in Mental Health Care Delivery Systems. Ablex, Norwood, 1980, pp. 119--137.

[23]

Lang, P.J. The emotion probe: Studies of motivation and attention. American Psychologist, 50, 5 (1995), 372--385.

[24]

Lazarus, R.S. Thoughts on the Relations between Emotions and Cognition. American Phychologist, 37, 10 (1982), 1019--1024.

[25]

Lerner, J.S. and Keltner, D. Beyond valence: Toward a model of emotion-specific influences on judgement and choice. Cognition and Emotion, 14, 4 (2000), 473--493.

[26]

Mahlke, S., Minge, M., and Thüring, M. Measuring Multiple Components of Emotions in Interactive Contexts. In CHI EA. ACM, New York, 2006, pp. 1061--1066.

Digital Library

[27]

Mahlke, S., Minge, M., and Thüring, M. Measuring Multiple Components of Emotions in Interactive Contexts. In CHI EA. ACM, New York, 2006, pp. 1061--1066.

Digital Library

[28]

Mandryk, R.L. and Atkins, M.S. A Fuzzy Physiological Approach for Continuously Modeling Emotion During Interaction with Play Technologies. Int. J. Hum.-Comput. Stud., 65, 4 (April 2007), 329--347.

Digital Library

[29]

Murray, B.D., Holland, A.C., and Kensinger, E.A. Episodic Memory and Emotion. In M.D. Robinson, E.R. Watkins and E. Harmon-Jones, eds., Handbook of Cognition and Emotion. Guilford Press, 2013, pp. 156--175.

[30]

Omodei, M.M. and McLennan, J. Studying complex decision making in natural settings: Using a head-mounted video camera to study competitive orienteering. Perceptual and Motor Skills, 79, 3, Pt 2 (1994), 1411--1425.

[31]

Philippot, P., Baeyens, C., and Douilliez, C. Specifying emotional information: Regulation of emotional intensity via executive processes. Emotion (Washington, D.C.), 6, 4 (November 2006), 560--571.

[32]

Redelmeier, D.A. and Kahneman, D. Patients' memories of painful medical treatments: real-time and retrospective evaluations of two minimally invasive procedures. Pain, 66, 1 (1996), 3--8.

[33]

Scherer, K.R. What are emotions? And how can they be measured? Social Science Information, 44, 4 (2005), 695--729.

[34]

Scollon, C.N., Kim-Prieto, C., and Diener, E. Experience Sampling: Promises and Pitfalls, Strengths and Weaknesses. Journal of Happiness Studies, 4, 1 (2003), 5--34.

[35]

Smith, A.P.R., Henson, R.N.A., Rugg, M.D., and Dolan, R.J. Modulation of retrieval processing reflects accuracy of emotional source memory. Learning & Memory, 12, 5 (2005), 472--479.

[36]

Springett, M. Assessing user experiences within interaction: experience as a qualitative state and experience as a causal event. In Proc. Int. WS on Valid Useful User Experience Measurement (VUUM). IRIT, Toulouse, 2008.

[37]

Stone, A.A., Schwartz, J.E., Neale, J.M., et al. A comparison of coping assessed by ecological momentary assessment and retrospective recall. Journal of personality and social psychology, 74, 6 (June 1998), 1670--1680.

[38]

Ståhl, A., Höök, K., Svensson, M., Taylor, A.S., and Combetto, M. Experiencing the Affective Diary. Personal Ubiquitous Comput., 13, 5 (2009), 365--378.

Digital Library

[39]

Thüring, M. and Mahlke, S. Usability, aesthetics and emotions in human--technology interaction. Int. J. Psychology, 42, 4 (2007), 253--264.

[40]

Tähti, M. and Arhippainen, L. A Proposal for collecting Emotions and Experiences. Interactive Experiences in HCI, 2, (2004), 195--198.

[41]

Ward, R. and Marsden, P. Physiological responses to different WEB page designs. Int. J. Human-Computer Studies, 59, 1-2 (July 2003), 199--212.

Digital Library

Cited By

Howell NHartsoe WAmin JNamani V(2024)Reflective Design for Informal Participatory Algorithm Auditing: A Case Study with Emotion AIProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685411(1-17)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685411
Yamagishi TYasunaka YTakayama ENomaru TKomatsu TWatanabe K(2024)Ah-Aloud Method to Comprehend Time-Series Emotion Observation During Gameplay: An Initial Investigation with Japanese SpeakersProceedings of the ACM on Human-Computer Interaction10.1145/36770568:CHI PLAY(1-23)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677056
Esposito ADesolda GLanzilotti R(2024)The fine line between automation and augmentation in website usability evaluationScientific Reports10.1038/s41598-024-59616-014:1Online publication date: 2-May-2024
https://doi.org/10.1038/s41598-024-59616-0
Show More Cited By

Index Terms

Asserting Real-Time Emotions through Cued-Recall: Is it Valid?
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Do You Feel the Same? On the Robustness of Cued-Recall Debriefing for User Experience Evaluation

Cued Recall Debriefing (CRD) is a form of retrospective think aloud approach. It involves re-immersing users to a level where emotional responses are comparable to those experienced during actual interaction with a system. To validate whether the ...
Measuring multiple components of emotions in interactive contexts
CHI EA '06: CHI '06 Extended Abstracts on Human Factors in Computing Systems

The study of users' emotional behavior in HCI has been receiving increasing attention for the last few years. This paper focuses on emotions as an important part of a user's overall experience when interacting with a system. Based on the multi-component ...
Real-time classification of evoked emotions using facial feature tracking and physiological responses

We present automated, real-time models built with machine learning algorithms which use videotapes of subjects' faces in conjunction with physiological measurements to predict rated emotion (trained coders' second-by-second assessments of sadness or ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

NordiCHI '16: Proceedings of the 9th Nordic Conference on Human-Computer Interaction

October 2016

1045 pages

ISBN:9781450347631

DOI:10.1145/2971485

General Chairs:
Staffan Björk
University of Gothenburg
,
Eva Eriksson
Chalmers University of Technology
,
Program Chairs:
Morten Fjeld
Chalmers University of Technology
,
Susanne Bødker
University of Aarhus
,
Publications Chairs:
Wolmet Barendregt
University of Gothenburg
,
Mohammad Obaid
Chalmers University of Technology

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Chalmers University of Technology
SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

NordiCHI '16

NordiCHI '16: 9th Nordic Conference on Human-Computer Interaction

October 23 - 27, 2016

Gothenburg, Sweden

Acceptance Rates

NordiCHI '16 Paper Acceptance Rate 58 of 231 submissions, 25%;

Overall Acceptance Rate 379 of 1,572 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
320
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)3

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Howell NHartsoe WAmin JNamani V(2024)Reflective Design for Informal Participatory Algorithm Auditing: A Case Study with Emotion AIProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685411(1-17)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685411
Yamagishi TYasunaka YTakayama ENomaru TKomatsu TWatanabe K(2024)Ah-Aloud Method to Comprehend Time-Series Emotion Observation During Gameplay: An Initial Investigation with Japanese SpeakersProceedings of the ACM on Human-Computer Interaction10.1145/36770568:CHI PLAY(1-23)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677056
Esposito ADesolda GLanzilotti R(2024)The fine line between automation and augmentation in website usability evaluationScientific Reports10.1038/s41598-024-59616-014:1Online publication date: 2-May-2024
https://doi.org/10.1038/s41598-024-59616-0
Kitson AMuntean RDiPaola SRiecke B(2022)Lucid Loop: Exploring the Parallels between Immersive Experiences and Lucid DreamingProceedings of the 2022 ACM Designing Interactive Systems Conference10.1145/3532106.3533538(865-880)Online publication date: 13-Jun-2022
https://dl.acm.org/doi/10.1145/3532106.3533538
Bruun ALaw ENielsen THeintz M(2021)Do You Feel the Same? On the Robustness of Cued-Recall Debriefing for User Experience EvaluationACM Transactions on Computer-Human Interaction10.1145/345347928:4(1-45)Online publication date: 23-Jul-2021
https://dl.acm.org/doi/10.1145/3453479
Kitson AChirico AGaggioli ARiecke B(2020)A Review on Research and Evaluation Methods for Investigating Self-TranscendenceFrontiers in Psychology10.3389/fpsyg.2020.54768711Online publication date: 16-Nov-2020
https://doi.org/10.3389/fpsyg.2020.547687
Schmidt TSchlindwein MLichtner KWolff C(2020)Investigating the Relationship Between Emotion Recognition Software and Usability Metricsi-com10.1515/icom-2020-000919:2(139-151)Online publication date: 6-Aug-2020
https://doi.org/10.1515/icom-2020-0009
Elsborg MBruun AJensen R(2020)Supporting Anxiety Patients’ Self-Reflection through Visualization of Physiological Data.Proceedings of the 32nd Australian Conference on Human-Computer Interaction10.1145/3441000.3441037(742-747)Online publication date: 2-Dec-2020
https://dl.acm.org/doi/10.1145/3441000.3441037
Sabie DSabie SAhmed SBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Memory through Design: Supporting Cultural Identity for Immigrants through a Paper-Based Home Drafting ToolProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376636(1-16)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376636
Johanssen JBernius JBruegge B(2019)Toward usability problem identification based on user emotions derived from facial expressionsProceedings of the 4th International Workshop on Emotion Awareness in Software Engineering10.1109/SEmotion.2019.00008(1-7)Online publication date: 28-May-2019
https://dl.acm.org/doi/10.1109/SEmotion.2019.00008
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten