Abstract
This chapter focuses on the User Engagement Scale (UES), a 31 item experiential questionnaire that measures six dimensions of user engagement: novelty, felt involvement, focused attention, aesthetic appeal, perceived usability, and endurability. The UES has been adopted and adapted to measure engagement in various domains, including online search, news, and video; educational, haptic, consumer, and social networking applications; and video games. Over 50 studies are synthesized to examine the dimensionality, reliability, validity and generalizability of the UES. In addition to illuminating the strengths and weaknesses of the UES as a measurement tool, this chapter uses the UES to explore the reciprocal relationship between theory and the development and application of measurement instruments in user experience research
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Agarwal, R., Karahanna, E.: Time flies when you’re having fun: cognitive absorption and beliefs about information technology usage. Manag. Inf. Syst. Q. 24 (4), 665–694 (2000)
Arapakis, I., Lalmas, M., Cambazoglu, B., Marcos, M., Jose, J.M.: User engagement in online news: under the scope of sentiment, interest, affect, and gaze. J. Assoc. Inf. Sci. Technol. 65 (10), 1988–2005 (2014)
Arapakis, I., Lalmas, M., Valkanas, G.: Understanding within-content engagement through pattern analysis of mouse gestures. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pp. 1439–1448. ACM, New York (2014)
Arapakis, I., Bai, X., Cambazoglu, B.B.: Impact of response latency on user behavior in web search. In: Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 103–112. ACM, New York (2014)
Arguello, J., Wu, W.-C., Kelly, D., Edwards, A.: Task complexity, vertical display and user interaction in aggregated search. In: Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 435–444. ACM, New York (2012)
Bangcuyo, R.G., Smith, K.J., Zumach, J.L., Pierce, A.M., Guttman, G.A., Simons, C.T.: The use of immersive technologies to improve consumer testing: the role of ecological validity, context and engagement in evaluating coffee. Food Qual. Prefer. 41, 84–95 (2015)
Banhawi, F., Ali, N.M.: Measuring user engagement attributes in social networking application. In: Proceedings of the 2011 International Conference on Semantic Technology and Information Retrieval (STAIR), pp. 297–301. IEEE, New York (2011)
Bödker, S.: When second wave HCI meets third wave challenges. In: Proceedings of the 4th Nordic Conference on Human-Computer Interaction (NordCHI ’06), pp. 1–8. ACM, New York (2006). doi:10.1145/1182475.1182476
Bron, M., van Gorp, J., Nack, F., Baltussen, L.B., de Rijke, M.: Aggregated search interface preferences in multi-session search tasks. In: Proceedings of the 36th International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 123–132. ACM, New York (2013)
Brooke, J.: SUS-A quick and dirty usability scale. In: Jordan, P.W., Thomas, B., Weerdmeester, B.A., McClelland, A.L. (eds.) Usability Evaluation in Industry, pp. 4–7. Taylor and Francis, London (1996)
Busselle, R., Bilandzic, H.: Measuring narrative engagement. Media Psychol. 12 (4), 321–347 (2009)
Bustillo, J., Garaizar, P.: Scratching the surface of digital literacy…but we need to go deeper. In: 2014 IEEE Frontiers in Education Conference Proceedings, pp. 1440–1443. IEEE, New York (2014)
Choi, H.S.: The impact of visuospatial characteristics of video games on improvements in cognitive abilities. Unpublished Doctoral Dissertation, North Carolina State University (2013)
Cian, L., Krishna, A., Elder, R.S.: This logo moves me: dynamic imagery from static image. J. Mark. Res. 51 (2), 184–197 (2013)
Creswell, J.W.: Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage, Los Angeles (2003)
De Moor, K., Mazza, F., Hupont, I., Ríos Quintero, M., Mäki, T., Varela, M.: Chamber QoE: a multi-instrumental approach to explore affective aspects in relation to quality of experience. In: Proceedings of SPIE 9014, Human Vision and Electronic Imaging XIX, 90140U (2014). doi:10.1117/12.2042243
Deci, E.L., Ryan, R.M.: Handbook of Self-Determination Research. University Rochester Press, New York (2012)
Devellis, R.F.: Scale Development: Theory and Applications, 2nd edn. Sage, Thousand Oaks (2003)
Fuchsberger, V., Sellner, W., Moser, C., Tscheligi, M.: Benefits and hurdles for older adults in intergenerational online interactions. In: Miesenberger, K., Karshmer, A., Penaz, P., Zagler, W. (eds.) Computers Helping People with Special Needs. Lecture Notes in Computer Science, vol. 7382, pp. 697–704. Springer, Berlin/Heidelberg (2012)
Grafsgaard, J.F.: Multimodal affect modeling in task-oriented tutorial dialogue. Unpublished Doctoral Dissertation, North Carolina State University (2014)
Halpern, D.: Towards a networked public sphere: how social media triggers civic engagement through news consumption and political discussion. Unpublished Doctoral Dissertation, Rutgers University (2013)
Hong, T.: Internet health search: when process complements goals. J. Am. Soc. Inf. Sci. Technol. 63 (11), 2283–2293 (2012)
Jacques, R.D.: The nature of engagement and its role in hypermedia evaluation and design. Unpublished Doctoral Dissertation, South Bank University (1996)
Kajalainen, K.: Increasing the enjoyment of online video increasing the enjoyment of online video content with topical interactivity. Unpublished Doctoral Dissertation, Aalto University (2015)
Kelly, D.: Methods for evaluating interactive information retrieval systems with users. Found. Trends Inf. Retr. 3 (1–2), 1–224 (2009)
Kwak, N., Williams, A.E., Wang, X., Lee, H.: Talking politics and engaging politics: an examination of the interactive relationships between structural features of political talk and discussion engagement. Commun. Res. 32 (1), 87–111 (2005)
Lalmas, M. O’Brien, H., Yom-Tov, E.: Measuring user engagement. Synth. Lect. Inf. Concepts Retr. Serv. 6 (4), 1–132 (2014)
Lawrence, B., Fournier, S., Brunel, F.: When companies don’t make the ad: a multimethod inquiry into the differential effectiveness of consumer-generated advertising. J. Advert. 42 (4), 292–307 (2013)
Lee, Y.-C., Lin, W.-C., Cherng, F.-Y., Wang, H.-C., Sung, C.-Y., King, J.-T.: Using time-anchored peer comments to enhance social interaction in online educational videos. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 689–698. ACM, New York (2015)
Levesque, V., Oram, L., MacLean, K., Cockburn, A., Marchuk, N.D., Johnson, D., Colgate, J.E., Peshkin, M.A.: Enhancing physicality in touch interaction with programmable friction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2481–2490. ACM, New York (2011)
Lindgaard, G., Kirakowski, J.: Introduction to the special issue: the tricky landscape of developing rating scales in HCI. Interact. Comput. 25 (4), 271–277 (2013)
McCay-Peet, L., Lalmas, M., Navalpakkam, V.: On salience, affect and focused attention. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 541–550. ACM, New York (2012). doi:10.1145/2207676.2207751
Moshfeghi, Y., Matthews, M., Blanco, R., Jose, J.M.: Influence of timeline and named-entity components on user engagement. In: Serdyukov, P., Braslavski, P., Kuznetsov, S., Kamps, J., Rüger, S., Agichtein, E., Segalovich, I., Yilmaz, E. (eds.) Advances in Information Retrieval. Lecture Notes in Computer Science, vol. 7814, pp. 305–317. Springer, Berlin/Heidelberg (2013)
O’Brien, H.: Defining and measuring user experiences with technology. Unpublished Doctoral Dissertation, Dalhousie University (2008)
O’Brien, H.: The influence of hedonic and utilitarian motivations on user engagement: the case of online shopping experiences. Interact. Comput. 22 (5), 344–352 (2010)
O’Brien, H.: The role of story and media in user engagement with online news. In: Proceedings of the Annual Conference of Canadian Association of Information Science (CAIS), CAIS, Victoria (2013)
O’Brien, H., Cairns, P.: An empirical evaluation of the user engagement scale (UES) in online news environments. Inf. Process. Manag. 51 (4), 413–427 (2015)
O’Brien, H., Lebow, M.: A mixed methods approach to measuring user experience in online news interactions. J. Assoc. Inf. Sci. Technol. 64 (8), 1543–1556 (2013)
O’Brien, H., Toms, E.G.: What is user engagement? A conceptual framework for defining user engagement with technology. J. Am. Soc. Inf. Sci. Technol. 59 (6), 938–955 (2008)
O’Brien, H., Toms, E.G.: The development and evaluation of a survey to measure user engagement in e-commerce environments. J. Am. Soc. Inf. Sci. Technol. 61 (1), 50–69 (2010)
O’Brien, H., Toms, E.G.: Is there a universal instrument for measuring interactive information retrieval? The case of the user engagement scale. In: Proceedings of Information Interaction in Context (IIiX), pp. 335–340. ACM, Rutgers (2010). doi:10.1145/1840784.1840835
O’Brien, H., Toms, E.G.: Examining the generalizability of the user engagement scale (UES) in exploratory search. Inf. Process. Manag. 49 (5), 1092–1107 (2013)
Okoro, E.M.: A study of different representation conventions during investigatory sensemaking. Unpublished Masters Thesis, Middlesex University (2014)
Parra, D., Brusilovsky, P.: User-controllable personalization: a case study with SetFusion. Int. J. Hum. Comput. Stud. 77, 43–67 (2014)
Peterson, R.A.: Constructing Effective Questionnaires. Sage Publications, Thousand Oaks (2000)
Reitz, A.R.: Online consumer engagement: understanding the antecedents and outcomes. Unpublished Doctoral Dissertation, Colorado State University (2012)
Seedorf, S., Thum, C., Schulze, T., Pfrogner, L.: Social co-browsing in online shopping: the impact of real-time collaboration on user engagement. In: Proceedings of the Twenty Second European Conference on Information Systems, Tel Aviv 2014. AIS Electronic Library (2014)
Sharek, D.J.: Investigating real-time predictors of engagement: implications for adaptive video games and online training. Unpublished Doctoral Dissertation, North Carolina State University (2012)
Shirzad, N.: The use of physiological signals and motor performance metrics in task difficulty adaptation: improving engagement in robot-assisted movement therapy. Unpublished Doctoral Dissertation, The University of British Columbia (2013)
Tabachnick, B.G., Fidell, L.S.: Using Multivariate Statistics, 6th International edition (cover) edn. Sage Publications, Thousand Oaks (2013)
Vail, A.K., Boyer, K.E., Wiebe, E.N., Lester, J.C.: The Mars and Venus effect: the influence of user gender on the effectiveness of adaptive task support. In: Ricci, F., Bontcheva, K., Conlan, O., Lawless, S. (eds.) User Modeling, Adaptation and Personalization. Lecture Notes in Computer Science, vol. 9146, pp. 265–276. Springer, Berlin/Heidelberg (2015)
Warnock, D., Lalmas, M.: An exploration of cursor tracking data. arXiv preprint (2015). arXiv:1502.00317
Webster, J., Ho, H.: Audience engagement in multimedia presentations. Data Base Adv. Inf. Syst. 28 (2), 63–77 (1997)
Whitman, L.: The effectiveness of interactivity in multimedia software tutorials. Unpublished Doctoral Dissertation, North Carolina State University (2013)
Wiebe, E.N., Lamb, A., Hardy, M., Sharek, D.: Measuring engagement in video game-based environments: investigation of the user engagement scale. Comput. Educ. 32, 123–132 (2014)
Witmer, B.G., Singer, M.J.: Measuring presence in virtual environments: a presence questionnaire. Presence: Teleoper. Virtual Environ. 7 (3), 225–240 (1998)
Zhu., Y., Heynderick, I., Redi, J.A.: Alone or together: measuring users’ viewing experience in different social contexts. In: Proceedings of SPIE 9014, Human Vision and Electronic Imaging XIX, 90140W (2014). doi:10.1117/12.2042867
Acknowledgements
I wish to express appreciation to the Social Science and Humanities Research Council of Canada and the Networks of Centres of Excellence in Graphics, Animation and New Media Project for funding support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
O’Brien, H. (2016). Translating Theory into Methodological Practice. In: O'Brien, H., Cairns, P. (eds) Why Engagement Matters. Springer, Cham. https://doi.org/10.1007/978-3-319-27446-1_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-27446-1_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-27444-7
Online ISBN: 978-3-319-27446-1
eBook Packages: Computer ScienceComputer Science (R0)