skip to main content
10.1145/3584931.3611292acmconferencesArticle/Chapter ViewAbstractPublication PagescscwConference Proceedingsconference-collections
extended-abstract

Understanding and Evaluating UX Outcomes at Scale

Published: 14 October 2023 Publication History

Abstract

Evaluating the success of applications from a summative perspective is essential to many industry researchers’ roles, yet a thorough understanding of what to measure and how to bridge business frameworks remains elusive. New technologies and novel ways of interacting with applications have garnered domain-specific interest in evaluating these experiences, but there is yet to be clarity or a path forward for a general methodological approach. While experience outcomes, UX OKRs, UX metrics, UX health, etc., are essential elements in HCI research and design, it has not become a core topic in HCI courses. This workshop aims to address this gap by bringing together academics and industry researchers involved in the measurement of success around large-scale applications – applications with a large user base. The objective of this workshop will be to engage in dynamic discussions around current evaluative experiences, how we move forward to produce valuable and unified updates to this topic, and a plan to do so. The workshop will specifically look at the value of writing a State of the Union paper, developing a community (e.g., SIG), and exploring education and training opportunities around this topic.

References

[1]
[1] Avinash Kaushik. 2013. See-Think-Care-Do: A Content, Marketing, Measurement Business Framework. (July 2013). Retrieved June 30, 2023 from https://www.kaushik.net/avinash/see-think-do-content-marketing-measurement-business-framework/
[2]
[2] Caitlin Sadowski, Margaret-Anne Storey, and Robert Feldt. 2019. A Software Development Productivity Framework. In: Sadowski C., Zimmermann T. (eds) Rethinking Productivity in Software Engineering. Apress, Berkeley, CA. https://doi.org/10.1007/978-1- 4842-4221-6_5
[3]
[3] Dmitry Alexandrovsky, Susanne Putze, Valentin Schwind, Elisa D. Mekler, Jan David Smeddinck, Denise Kahl, Antonio Krüger, 7 and Rainer Malaka. 2021. Evaluating User Experiences in Mixed Reality. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA '21). Association for Computing Machinery, New York, NY, USA, Article 87, 1–5. https://doi.org/10.1145/3411763.3441337
[4]
[4] Emeline Brulé, Oussama Metatla, Katta Spiel, Ahmed Kharrufa, and Charlotte Robinson. 2019. Evaluating Technologies with and for Disabled Children. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). Association for Computing Machinery, New York, NY, USA, Paper SIG08, 1–6. https://doi.org/10.1145/32900607.3311757
[5]
[5] Kerry Rodden, Hilary Hutchinson, and Xin Fu. 2010. Measuring the user experience on a large scale: user-centered metrics for web applications. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). Association for Computing Machinery, New York, NY, USA, 2395–2398. https://doi.org/10.1145/1753326.1753687
[6]
[6] Roman Bednarik, Ann Blandford, Feng Feng, Antti Huotarinen, Matti Iso-Mustajärvi, Ahreum Lee, Federico Nicolosi, Jeremy Opie, Soojeong Yoo, and Bin Zheng. 2022. Integration of Human Factors in Surgery: Interdisciplinary Collaboration in Design, Development, and Evaluation of Surgical Technologies. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA '22). Association for Computing Machinery, New York, NY, USA, Article 112, 1–7. https://doi.org/10.1145/3491101.3503709
[7]
[7] Sayan Sarcar, Cosmin Munteanu, Neil Charness, Jussi Jokinen, Xiangshi Ren, and Emma Nicol. 2021. Designing Interactions for the Ageing Populations – Addressing Global Challenges. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA '21). Association for Computing Machinery, New York, NY, USA, Article 110, 1–4. https://doi.org/10.1145/3411763.3441326
[8]
[8] Sean Byrnes. 2019. What is Self-Service Data Science? Forbes, Innovation, Retrieved June 30, 2023 from https://www.forbes.com/sites/forbestechcouncil/2019/08/22/what-is-self-service-data-science/
[9]
[9] Serena Hillman, Samira Jain, Vichita Jienjitlert, and Paula Bach. 2022. The BLUE Framework: Designing User-Centered In-Product Feedback for Large Scale Applications. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA '22). Association for Computing Machinery, New York, NY, USA, Article 21, 1–8. https://doi.org/10.1145/3491101.3503558
[10]
[10] Student Services McGill University, 8 Tips for Making your Workshop more Accessible, Retrieved from https://www.mcgill.ca/skills21/files/skills21/accessibility_workshop_pdf.pdf

Cited By

View all
  • (2024)Beyond Theory: A UX Outcomes Casebook for HCI EducationExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3643980(1-3)Online publication date: 11-May-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CSCW '23 Companion: Companion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing
October 2023
596 pages
ISBN:9798400701290
DOI:10.1145/3584931
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 October 2023

Check for updates

Author Tags

  1. Experience outcomes
  2. KPIs
  3. OKRs
  4. Summative research
  5. UX health
  6. UX metrics
  7. UX outcomes
  8. user-centered metrics

Qualifiers

  • Extended-abstract
  • Research
  • Refereed limited

Conference

CSCW '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 2,235 of 8,521 submissions, 26%

Upcoming Conference

CSCW '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)55
  • Downloads (Last 6 weeks)3
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Beyond Theory: A UX Outcomes Casebook for HCI EducationExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3643980(1-3)Online publication date: 11-May-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media