research-article

How to evaluate technologies for health behavior change in HCI research

Authors:

Predrag Klasnja,

Sunny Consolvo,

Wanda PrattAuthors Info & Claims

CHI '11: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 3063 - 3072

https://doi.org/10.1145/1978942.1979396

Published: 07 May 2011 Publication History

Abstract

New technologies for encouraging physical activity, healthy diet, and other types of health behavior change now frequently appear in the HCI literature. Yet, how such technologies should be evaluated within the context of HCI research remains unclear. In this paper, we argue that the obvious answer to this question - that evaluations should assess whether a technology brought about the intended change in behavior - is too limited. We propose that demonstrating behavior change is often infeasible as well as unnecessary for a meaningful contribution to HCI research, especially when in the early stages of design or when evaluating novel technologies. As an alternative, we suggest that HCI contributions should focus on efficacy evaluations that are tailored to the specific behavior-change intervention strategies (e.g., self-monitoring, conditioning) embodied in the system and studies that help gain a deep understanding of people's experiences with the technology.

References

[1]

Abowd, G. D., Mynatt, E. D., and Rodden, T. The human experience {of ubiquitous computing}. IEEE pervasive computing 1, 1 (2002), 48--57.

Digital Library

[2]

Anderson, I., Maitland, J., Sherwood, S., et al. Shakra: Tracking and Sharing Daily Activity Levels with Unaugmented Mobile Phones. Mobile Networks and Applications 12, 2-3 (2007), 185--199.

Digital Library

[3]

Bargh, J. A., Gollwitzer, P. M., Lee-Chai, A., Barndollar, K., and Trötschel, R. The automated will: Nonconscious activation and pursuit of behavioral goals. Journal of Personality and Social Psychology 81, 6 (2001), 1014--1027.

[4]

Barkhuus, L. and Rode, J. From mice to men: 24 years of evaluation in CHI. ACM CHI '07 - Alt.CHI, (2007).

Digital Library

[5]

van der Berg, M. H., Schoones, J. W., and Vliet Vlieland, T. P. M. Internet-based physical activity interventions: A systematic review of the literature. Journal of Medical Internet Research 9, 3 (2007).

[6]

Bickmore, T. W., Caruso, L., and Clough-Gorr, K. Acceptance and usability of a relational agent interface by older urban results. Proceedings of CHI 2005, ACM Press (2005).

Digital Library

[7]

Consolvo, S., Everitt, K. M., Smith, I., and Landay, J. A. Design requirements for technologies that encourage physical activity. Proceedings of CHI 2006, ACM Press (2006), 457--466.

Digital Library

[8]

Consolvo, S., Klasnja, P., McDonald, D. W., et al. Flowers or a robot army?: encouraging awareness & activity with personal, mobile displays. Proceedings of the 10th international conference on Ubiquitous computing, ACM (2008), 54--63.

Digital Library

[9]

Consolvo, S., McDonald, D. W., Toscos, T., et al. Activity sensing in the wild: a field trial of ubifit garden. Proceeding of CHI 2008, ACM (2008), 1797--1806.

Digital Library

[10]

Csikszentmihalyi, M. and Larson, R. Validity and reliability of the Experience Sampling Method. Journal of Nervous and Mental Disease 175, 9 (1987), 526--536.

[11]

Deci, E. L. and Ryan, R. M. The "What" and "Why" of Goal Pursuits: Human Needs and the Self-Determination of Behavior. Psychological Inquiry: An International Journal for the Advancement of Psychological Theory 11, 4 (2000), 227--268.

[12]

Denning, T., Andrew, A., Chaudhri, R., et al. BALANCE: towards a usable pervasive wellness application with accurate activity inference. Proceedings of the 10th workshop on Mobile Computing Systems & Applications, ACM (2009), 1--6.

Digital Library

[13]

DeShazo, J., Harris, L., Turner, A., and Pratt, W. Grounded in theory, user-centered, and someplace else: Designing and remotely testing mobile diabetes video games. Journal of Telemedicine and Telecare 16, 7 (2010), 378--382.

[14]

Dixon, M. and Fogarty, J. Prefab: implementing advanced behaviors using pixel-based reverse engineering of interface structure. Proceedings of CHI 2010, ACM (2010), 1525--1534.

Digital Library

[15]

Fogg, B. J. Persuasive Technology: Using Computers to Change What We Think and Do. Morgan Kaufmann, New York, 2003.

[16]

Froehlich, J., Chen, M. Y., Consolvo, S., Harrison, B. L., and Landay, J. A. MyExperience: A system for in situ tracing and capturing of user feedback on mobile phones. Proceedings of the 5th international conference on Mobile systems, applications and services (MobiSys '07), ACM Press (2007), 57--70.

Digital Library

[17]

Gasser, R., Brodbeck, D., Degen, M., Luthiger, J., Wyss, R., and Reichlin, S. Persuasiveness of a Mobile Lifestyle Coaching Application Using Social Facilitation. Proceedings of the First International Conference on Persuasive Technology, Springer-Verlag (2006), 27--38.

Digital Library

[18]

Greenberg, S. and Buxton, B. Usability evaluation considered harmful (some of the time). Proceeding of CHI 2008, ACM (2008), 111--120.

Digital Library

[19]

Higgins, E. T. Knowledge activation: Accessibility, applicability, and salience. In E. T. Higgins and A. W. Kruglanski, eds., Social psychology: Handbook of basic principles. Guilford Press, New York, 1996, 133--168.

[20]

Hoffman, C., Rice, D., and Sung, H. Y. Persons with chronic conditions. Their prevalence and costs. JAMA: The Journal of the American Medical Association 276, 18 (1996), 1473--1479.

[21]

Hurling, R., Catt, M., De Boni, M., et al. Using internet and mobile phone technology to deliver an automated physical activity program: Randomized controlled trial. Journal of Medical Internet Research 9, (2007), e7.

[22]

Kopp, J. Self-monitoring: A literature review of research and practice. Social Work Research & Abstracts 24, (1988), 8--20.

[23]

Lee, G., Tsai, C., Griswold, W. G., Raab, F., and Patrick, K. PmEB: a mobile phone application for monitoring caloric balance. CHI '06 extended abstracts on Human factors in computing systems, ACM (2006), 1013--1018.

Digital Library

[24]

Lin, J. L., Mamykina, L., Lindtner, S., Delajoux, G., and Strub, H. B. Fish'n'Steps: Encouraging physical activity with an interactive computer game. Proceedings of Ubicomp 2006, Springer (2006), 261--278.

Digital Library

[25]

Logue, A. W. Self-control and health behavior. In W. K. Bickel and R. E. Vuchinich, eds., Reframing health behavior change with behavioral economics. Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 2000, 167--192.

[26]

Mamykina, L., Mynatt, E., Davidson, P., and Greenblatt, D. MAHI: investigation of social scaffolding for reflective thinking in diabetes management. Proceeding of CHI 2008, ACM (2008), 477--486.

Digital Library

[27]

Marcus, B. H., Dubbert, P. M., Forsyth, L. H., et al. Physical activity behavior change: issues in adoption and maintenance. Health Psychology: Official Journal of the Division of Health Psychology, American Psychological Association 19, 1 Suppl (2000), 32--41.

[28]

Marlatt, G. A. and George, W. H. Relapse prevention and the maintenance of optimal health. In S. A. Shumaker, E. B. Schron, J. K. Ockene and W. L. McBee, eds., The handbook of health behavior change. Springer, New York, 1998, 33--58.

[29]

Mattila, E., Pärkkä, J., Hermersdorf, M., et al. Mobile diary for wellness management - Results on usage and usability in two user studies. IEEE Transactions on Information Technology in Biomedicine 12, 4 (2008), 501--512.

Digital Library

[30]

Moore, S. M., Dolansky, M. A., Ruland, C. M., Pashkow, F. J., and Blackburn, G. G. Predictors of women's exercise maintenance after cardiac rehabilitation. Journal of Cardiopulmonary Rehabilitation 23, 1 (2003), 40--49.

[31]

Morris, M., Kathawala, Q., Leen, T. K., et al. Mobile therapy: Case study evaluations of a cell phone application for emotional self-awareness. Journal of Medical Internet Research 12, 2 (2010), e10.

[32]

Muraven, M. and Baumeister, R. F. Self-regulation and depletion of limited resources: Does self-control resemble a muscle? Psychological Bulletin 126, 2 (2000), 247--259.

[33]

Muraven, M., Tice, D. M., and Baumeister, R. F. Self-control as limited resource: Regulatory depletion patterns. Journal of Personality and Social Psychology 74, 3 (1998), 774--789.

[34]

Nawyn, J., Intille, S. S., and Larson, K. Embedding behavior modification strategies into a consumer electronic device: a case study. Proceedings of UbiComp 2006, Springer (2006), 297--314.

Digital Library

[35]

Nelson, R. O. Assessment and therapeutic functions of self-monitoring. In M. Hersen, R. M. Eisler and P. M. Miller, eds., Progress in behavior modification. Academic Press, New York, 1977.

[36]

Prochaska, J. O., Johnson, S., and Lee, P. The transtheoretical model of behavior change. In S. A. Shumaker, E. B. Schron, J. K. Ockene and W. L. McBee, eds., The handbook of health behavior change. Springer, New York, 1998, 59--84.

[37]

Radtke, K. L. Exercise compliance in cardiac rehabilitation. Rehabilitation Nursing: The Official Journal of the Association of Rehabilitation Nurses 14, 4 (1989), 182--186, 195.

[38]

Rogers, Y., Connelly, K., Tedesco, L., et al. Why It's Worth the Hassle: The Value of In-Situ Studies When Designing Ubicomp. In UbiComp 2007: Ubiquitous Computing. 2007, 336--353.

Digital Library

[39]

Smith, B. K., Frost, J., Albayrak, M., and Sudhakar, R. Integrating glucometers and digital photography as experience capture tools to enhance patient understanding and communication of diabetes self-management practices. Personal Ubiquitous Computing 11, 4 (2007), 273--286.

Digital Library

[40]

Tsai, C. C., Lee, G., Raab, F., et al. Usability and feasability of PmEB: A mobile phone application for monitoring real time caloric balance. Mobile Networks and Applications 12, 2-3 (2007), 173--184.

Digital Library

[41]

Twardella, D., Merx, H., Hahmann, H., Wüsten, B., Rothenbacher, D., and Brenner, H. Long term adherence to dietary recommendations after inpatient rehabilitation: prospective follow up study of patients with coronary heart disease. Heart (British Cardiac Society) 92, 5 (2006), 635--640.

[42]

Witkiewitz, K. and Marlatt, G. A. Relapse prevention for alcohol and drug problems: that was Zen, this is Tao. The American Psychologist 59, 4 (2004), 224--235.

Cited By

van Iterson HLiang RMarkopoulos P(2025)Redesigning Fluid Tracking Probes for Elderly Lifestyle RetrofitProceedings of the Nineteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3689050.3706003(1-6)Online publication date: 4-Mar-2025
https://dl.acm.org/doi/10.1145/3689050.3706003
Milkowski MOlesk JAsfura DBarco CCervera CWachira MSisk MBalke APurushotham DWilliams RBadillo-Urquiola K(2025)Youth Vaping in the Digital Age: A Systematic Review of Technological Interventions for Prevention and CessationCompanion Proceedings of the 2025 ACM International Conference on Supporting Group Work10.1145/3688828.3699642(58-67)Online publication date: 12-Jan-2025
https://dl.acm.org/doi/10.1145/3688828.3699642
Chaudhry BSiek KConnelly K(2024)The Usability and Feasibility of a Dietary Intake Self-Monitoring Application in a Population with Varying Literacy LevelsJournal of Personalized Medicine10.3390/jpm1409100114:9(1001)Online publication date: 20-Sep-2024
https://doi.org/10.3390/jpm14091001
Show More Cited By

Index Terms

How to evaluate technologies for health behavior change in HCI research
1. Applied computing
  1. Life and medical sciences
    1. Consumer health
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Index terms have been assigned to the content through auto-classification.

Recommendations

Rethinking Technologies for Behavior Change: A View from the Inside of Human Change

Human-Computer Interaction researchers are increasingly designing technologies aimed at supporting “behavior change.” The model of change, which most of these works embrace, focuses on the idea that change occurs on the behavioral level and that it is ...
Self-Care Technologies in HCI: Trends, Tensions, and Opportunities

Many studies show that self-care technologies can support patients with chronic conditions and their carers in understanding the ill body and increasing control of their condition. However, many of these studies have largely privileged a medical ...
Exploring the Lived Experience of Behavior Change Technologies: Towards an Existential Model of Behavior Change for HCI
The majority of behavior change and persuasive technologies are exclusively addressed to modify a specific behavior. However, the focus on behavior may cloud the “existential aspects” of the process of change. To explore the lived and meaning-laden ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '11: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

May 2011

3530 pages

ISBN:9781450302289

DOI:10.1145/1978942

General Chair:
Desney Tan
Microsoft Research
,
Program Chairs:
Geraldine Fitzpatrick
Vienna University of Technology
,
Carl Gutwin
University of Saskatchewan
,
Bo Begole
PARC
,
Wendy A. Kellogg
IBM Research

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '11

Sponsor:

SIGCHI

CHI '11: CHI Conference on Human Factors in Computing Systems

May 7 - 12, 2011

BC, Vancouver, Canada

Acceptance Rates

CHI '11 Paper Acceptance Rate 410 of 1,532 submissions, 27%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

303
Total Citations
View Citations
5,169
Total Downloads

Downloads (Last 12 months)391
Downloads (Last 6 weeks)25

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

van Iterson HLiang RMarkopoulos P(2025)Redesigning Fluid Tracking Probes for Elderly Lifestyle RetrofitProceedings of the Nineteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3689050.3706003(1-6)Online publication date: 4-Mar-2025
https://dl.acm.org/doi/10.1145/3689050.3706003
Milkowski MOlesk JAsfura DBarco CCervera CWachira MSisk MBalke APurushotham DWilliams RBadillo-Urquiola K(2025)Youth Vaping in the Digital Age: A Systematic Review of Technological Interventions for Prevention and CessationCompanion Proceedings of the 2025 ACM International Conference on Supporting Group Work10.1145/3688828.3699642(58-67)Online publication date: 12-Jan-2025
https://dl.acm.org/doi/10.1145/3688828.3699642
Chaudhry BSiek KConnelly K(2024)The Usability and Feasibility of a Dietary Intake Self-Monitoring Application in a Population with Varying Literacy LevelsJournal of Personalized Medicine10.3390/jpm1409100114:9(1001)Online publication date: 20-Sep-2024
https://doi.org/10.3390/jpm14091001
Namyalo PSetekera RNakazibwe P(2024)Adopting information and communications technology in the control, prevention, and management of stroke: perspectives from patients and providers in UgandaFrontiers in Stroke10.3389/fstro.2024.14400473Online publication date: 18-Dec-2024
https://doi.org/10.3389/fstro.2024.1440047
Zeitlin AMathenjwa TZuma TWyke SMatthews PMcGrath NSeeley JShahmanesh MBlandford A(2024)Creating a usable and effective digital intervention to support men to test for HIV and link to care in a resource-constrained setting: iterative design based on a Person-Based Approach and Human Computer Interaction methods (Preprint)JMIR Formative Research10.2196/65185Online publication date: 12-Aug-2024
https://doi.org/10.2196/65185
Krämer SKottwitz GWoźniak MMeyer JKoelle M(2024)Stepping on the scale? Only in the morning, of course! - Exploring the Lived Experience of Consumer Self-Tracking Technology in the Context of Citizen ScienceProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685372(1-17)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685372
Scott AReicherts LPurohit AHossain EStefanidi EWagener NSchöning JRogers YHolzer A(2024)DIY Digital Interventions: Behaviour Change with Trigger-Action ProgrammingProceedings of the ACM on Human-Computer Interaction10.1145/36764978:MHCI(1-28)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676497
Zhao YChoudhury TKostakos VKay JHoang T(2024)Evaluate Closed-Loop, Mindless Intervention in-the-Wild: A Micro-Randomized Trial on Offset Heart Rate BiofeedbackCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678373(307-312)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3678373
Adler DChoudhury TKostakos VKay JHoang T(2024)Ubiquitous Computing in Action: Infrastructure to Support Sensing and Mental Health Research in PracticeCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678372(301-306)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3678372
Sano TTakahashi MFujimura KInoue TSato TKatagiri YKostakos VKay JHoang T(2024)Entropy-based Planning Algorithm for Personalized Interventions: Preliminary Report on Question-reduction AgentCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3677590(86-90)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3677590
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten