Morten Hertzum
Kasper Hornbæk
Skip Abstract Section
Abstract
When computers unexpectedly delay or thwart goal attainment, frustration ensues. The central studies of the extent, content, and impact of such frustration were done more than 15 years ago. We revisit this issue after computers have become more mature and computer use is more extensive. To this end, we had 234 crowdsourced participants log the frustrating episodes they experienced with their computers during one hour of computer use. The average time lost due to frustrating episodes was between 11% and 20% of the one-hour period. Though this is less time lost than in the earlier studies, frustration remains a common user experience. While shorter, the median level of frustration during the episodes was high (7 on a 9-point scale). The frustration level correlated with task importance and time lost but was unaffected by computer experience and largely unaffected by computer self-efficacy. In addition, participants indicated that 84% of the episodes had happened before, that 87% could happen again, and that they were unable to resolve 26% of the episodes. This high rate of recurrence and lack of control likely added to the frustration level. The episodes spanned various issues pertaining to performance (49%), usability (36%), and utility (16%).
- [1] . 2009. Undo and erase events as indicators of usability problems. In Proceedings of the CHI2009 Conference on Human Factors in Computing Systems. ACM, New York, 659–668.
DOI: Google Scholar
Digital Library
- [2] . 2005. Jevons’ paradox. Ecological Economics 54, 1 (2005), 9–21.
DOI: Google ScholarCross Ref
- [3] . 2022. Auto-cucumber: The impact of autocorrection failures on users’ frustration. In Proceedings of the Graphics Interface 2022 Conference. Canadian Information Processing Society, Mississauga, ON.Google Scholar
- [4] . 1992. Frustration theory - Many years later. Psychological Bulletin 112, 3 (1992), 396–399.
DOI: Google ScholarCross Ref - [5] . 2019. How serious is the ‘carelessness’ problem on Mechanical Turk? International Journal of Social Research Methodology 22, 5 (2019), 441–449.
DOI: Google ScholarCross Ref - [6] . 2010. Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive-affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies 68, 4 (2010), 223–241.
DOI: Google ScholarDigital Library - [7] . 2001. Bad is stronger than good. Review of General Psychology 5, 4 (2001), 323–370.
DOI: Google ScholarCross Ref - [8] . 1989. Frustration-aggression hypothesis: Examination and reformulation. Psychological Bulletin 106, 1 (1989), 59–73.
DOI: Google ScholarCross Ref - [9] . 2003. Social and psychological influences on computer user frustration. In Media Access: Social and Psychological Dimensions of New Technology Use. (Eds.), Routledge, New York, 91–103.
DOI: Google ScholarCross Ref - [10] . 2006. A model for computer frustration: The role of instrumental and dispositional factors on incident, session, and post-session frustration and mood. Computers in Human Behavior 22, 6 (2006), 941–961.
DOI: Google ScholarCross Ref - [11] . 2018. Funology 2: From Usability to Enjoyment. Second edition. Springer, Cham.Google ScholarCross Ref
- [12] . 2007. How emotion is made and measured. International Journal of Human-Computer Studies 65, 4 (2007), 275–291.
DOI: Google ScholarDigital Library - [13] . 2017. Frustration-aggression theory. In The Wiley Handbook of Violence and Aggression, Peter Sturney (Ed.), Wiley, New York, 40:01–40:12.
DOI: Google ScholarCross Ref - [14] . 2016. Understanding the relationship between frustration and the severity of usability problems: What can psychophysiological data (not) tell us? In Proceedings of the CHI2016 Conference on Human Factors in Computing Systems. ACM, New york, 3975–3987.
DOI: Google ScholarDigital Library - [15] . 2004. Determining causes and severity of end-user frustration. International Journal of Human-Computer Interaction 17, 3 (2004), 333–356.
DOI: Google ScholarCross Ref - [16] . 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20, 1 (1960), 37–46.
DOI: Google ScholarCross Ref - [17] . 2014. Validity and reliability of the experience-sampling method. In Flow and the Foundations of Positive Psychology, Mihaly Csikszentmihalyi (Ed.), Springer, Netherlands, 35–54.Google ScholarCross Ref
- [18] . 1939. Frustration and Aggression. Yale University Press, New Haven, CT.Google ScholarCross Ref
- [19] . 1990. Learned helplessness, causal attribution, and response to frustration. The Journal of General Psychology 117, 1 (1990), 47–58.
DOI: Google ScholarCross Ref - [20] . 1989. Stimulating change through usability testing. ACM SIGCHI Bull. 21, 1 (1989), 37–44.
DOI: Google ScholarDigital Library - [21] . 2021. Individual differences in frustration and performance with online shopping activities. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 65, 1 (2021), 148–153.
DOI: Google ScholarCross Ref - [22] . 2018. The Upgrade: Why doctors hate their computers. New Yorker November 12 (2018), 62–73. Retrieved from https://www.newyorker.com/magazine/2018/11/12/why-doctors-hate-their-computers. Accessed February 12, 2023.Google Scholar
- [23] . 1980. A plea for friendly software. ACM SIGSOFT Software Engineering Notes 5, 4 (1980), 4–5.
DOI: Google ScholarDigital Library - [24] . 2018. End-user frustrations and failures in digital technology: Exploring the role of fear of missing out, internet addiction and personality. Heliyon 4, 11 (2018), e00872.
DOI: Google ScholarCross Ref - [25] . 2010. Faster progress bars: Manipulating perceived duration with visual augmentations. In Proceedings of the CHI2010 Conference on Human Factors in Computing Systems. ACM, New York, 1545–1548.
DOI: Google ScholarDigital Library - [26] . 2004. The interplay of beauty, goodness, and usability in interactive products. Human-Computer Interact. 19, 4 (2004), 319–349.
DOI: Google ScholarDigital Library - [27] . 2010. Frustration: A common user experience. In DHRS2010: Proceedings of the 10th Danish Human-Computer Interaction Research Symposium. Computer Science Research Report #132, Morten Hertzum and Magnus Hansen (Eds.). Roskilde University, Roskilde, DK, 11–14.Google Scholar
- [28] . 2021. Reference values and subscale patterns for the task load index (TLX): A meta-analytic review. Ergonomics 64, 7 (2021), 869–878.
DOI: Google ScholarCross Ref - [29] . 2022. Associations among workload dimensions, performance, and situational characteristics: A meta-analytic review of the task load index. Behaviour and Information Technology 41, 16 (2022), 3506–3518.
DOI: Google ScholarCross Ref - [30] . 2013. When filling the wait makes it feel longer: A paradigm shift perspective for managing online delay. MIS Quarterly. 37, 2 (2013), 383–406. Retrieved from https://www.jstor.org/stable/43825915.Google ScholarDigital Library
- [31] . 2004. A survey method for characterizing daily life experience: The day reconstruction method. Science 306, 5702 (2004), 1776–1780.
DOI: Google ScholarCross Ref - [32] . 1984. Attractive quality and must-be quality. Journal of the Japanese Society for Quality Control 14, 2 (1984), 39–48.Google Scholar
- [33] . 2010. A longitudinal study of usability in health care: Does time heal? International Journal of Medical Informatics 79, 6 (2010), e135–e143.
DOI: Google ScholarCross Ref - [34] . 1995. The Trouble with Computers: Usefulness, Usability, and Productivity. MIT Press, Cambridge, MA.Google ScholarDigital Library
- [35] . 2017. Research Methods in Human-Computer Interaction (2nd ed.). Morgan Kaufmann, Cambridge, MA.Google Scholar
- [36] . 2006. Severity and impact of computer user frustration: A comparison of student and workplace users. Interacting with Computers 18, 2 (2006), 187–207.
DOI: Google ScholarDigital Library - [37] . 2006. Workplace user frustration with computers: An exploratory investigation of the causes and severity. Behaviour and Information Technology 25, 3 (2006), 239–251.
DOI: Google ScholarCross Ref - [38] . 2005. Usability over time. In Proceedings of the SIGDOC2005 Conference on Design of Communication. ACM, New York, 151–158.
DOI: Google ScholarDigital Library - [39] . 2003. User recalled occurrences of usability errors: Implications on the user experience. In Proceedings of the CHI2003 Conference on Human Factors in Computing Systems: Extended Abstracts. ACM, New York, 736–737.
DOI: Google ScholarDigital Library - [40] . 2007. Toward a more accurate view of when and how people seek help with computer applications. In Proceedings of the SIGDOC2007 Conference on Design of Communication. ACM, New York, 95–102.
DOI: Google ScholarDigital Library - [41] . 2012. Short-term methodology for long-term usability. In Proceedings of the SIGDOC2012 Conference on Design of Communication. ACM, New York, 205–211.
DOI: Google ScholarDigital Library - [42] . 2022. The Platform Economy in Europe: Results from the Second ETUI Internet and Platform Work Survey. Working paper 2022.05. European Trade Union Institute, Brussels.Google Scholar
- [43] . 2022. Characterizing the global crowd workforce: A cross-country comparison of crowdworker demographics. Human Computation 9, 1 (2022), 22–57.
DOI: Google ScholarCross Ref - [44] . 1998. A psychological investigation of long retrieval times on the world wide web. Interacting with Computers 10, 1 (1998), 77–86.
DOI: Google ScholarCross Ref - [45] . 2005. The impact of aging on self-efficacy and computer skill acquisition. Journal of Managerial Issues 17, 2 (2005), 212–228. Retrieved from https://www.jstor.org/stable/40604496.Google Scholar
- [46] . 2014. Emotional dimensions of user experience: A user psychological analysis. International Journal of Human-Computer Interaction 30, 4 (2014), 303–320.
DOI: Google ScholarCross Ref - [47] . 2008. Designing and Engineering Time: The Psychology of Time Perception in Software. Addison-Wesley, Upper Saddle River, NJ.
DOI: Google ScholarCross Ref - [48] . 1985. Human factors and usability - Whence and whither? In Software-Erqonomie’85: Mensch-Computer-lnteraktion. 13–31.Google Scholar
- [49] . 2020. Digital failure: An emerging reason of anger expression among adolescents. Industrial Psychiatry Journal 29, 2 (2020), 335–338.
DOI: Google ScholarCross Ref - [50] . 2008. Ecological momentary assessment. Annual Review of Clinical Psychology 4, (2008), 1–32.
DOI: Google ScholarCross Ref - [51] . 2000. Universal usability. Communications of the ACM 43, 5 (2000), 84–91.
DOI: Google ScholarDigital Library - [52] . 2009. Card sorting: Designing Usable Categories. Rosenfeld, New York.Google Scholar
- [53] . 2004. Variation in communication loads on clinical staff in the emergency department. Annals of Emergency Medicine 44, 3 (2004), 268–273.
DOI: Google ScholarCross Ref - [54] . 2015. Does Herzberg's notion of hygienes and motivators apply to user experience? ACM Transactions on Computer-Human Interaction 22, 4 (2015), 1–24.
DOI: Google ScholarDigital Library - [55] . 2016. The neural basis of frustration state. In Neuroimaging Personality, Social Cognition, and Character, John R. Absher and Jasmin Cloutier (Eds.). Academic Press, Amsterdam, 223–243.
DOI: Google ScholarCross Ref - [56] . 2014. The effects of a computer malfunction on subsequent task performance. Behaviour and Information Technology 33, 9 (2014), 874–881.
DOI: Google ScholarDigital Library
Index Terms
- Frustration: Still a Common User Experience
Recommendations
The effectiveness of social agents in reducing user frustration
CHI EA '06: CHI '06 Extended Abstracts on Human Factors in Computing SystemsA study was conducted to evaluate the effectiveness of social agents in reducing user frustration. The particular type of agent studied reacted to users' facial expressions while they browsed through a shopping website. While highly frustrated users ...
Empathic agents to reduce user frustration: The effects of varying agent characteristics
There is now growing interest in the development of computer systems which respond to users' emotion and affect. We report three small-scale studies (with a total of 42 participants), which investigate the extent to which affective agents, using ...
Emotional user experience
Emotional experience has become an important topic in human-technology interaction research and design. Nevertheless, such research and design often lacks a proper explanatory basis and methodologically robust operationalisation. In this article, a ...
Comments