research-article

Guidelines for Human-AI Interaction

Authors:

Saleema Amershi,

Mihaela Vorvoreanu,

Penny Collisson,

Paul N. Bennett,

Ruth Kikin-Gil,

Eric HorvitzAuthors Info & Claims

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Paper No.: 3, Pages 1 - 13

https://doi.org/10.1145/3290605.3300233

Published: 02 May 2019 Publication History

Abstract

Advances in artificial intelligence (AI) frame opportunities and challenges for user interface design. Principles for human-AI interaction have been discussed in the human-computer interaction community for over two decades, but more study and innovation are needed in light of advances in AI and the growing uses of AI technologies in human-facing applications. We propose 18 generally applicable design guidelines for human-AI interaction. These guidelines are validated through multiple rounds of evaluation including a user study with 49 design practitioners who tested the guidelines against 20 popular AI-infused products. The results verify the relevance of the guidelines over a spectrum of interaction scenarios and reveal gaps in our knowledge, highlighting opportunities for further research. Based on the evaluations, we believe the set of design guidelines can serve as a resource to practitioners working on the design of applications and features that harness AI technologies, and to researchers interested in the further development of human-AI interaction design principles.

Supplementary Material

ZIP File (paper003.zip)

The auxiliary material consists of a single zip file containing a PDF of the paper's appendix. In this appendix, we illustrate each of our 18 human-AI interaction design guidelines with three example applications and three example violations provided by our user study participants when testing the guidelines against popular AI-infused products. For each example we indicate the product category the participant was testing, but obscure the specific product names.

Download
213.57 KB

References

[1]

Alexa. 2018. Top sites in the United States. Retrieved July, 2018 from https://www.alexa.com/topsites/countries/US

[2]

Kathy Baxter. 2017. How to Meet User Expectations for Artificial Intelligence. Medium. Retrieved September, 2018 from https://medium.com/salesforce-ux/how-to-meet-userexpectations-for-artificial-intelligence-a51d3c82af6

[3]

Victoria Bellotti and Keith Edwards. 2001. Intelligibility and Accountability: Human Considerations in Context-Aware Systems. Human-- Computer Interaction 16, 2--4 (2001), 193--212.

Digital Library

[4]

Andrea Bunt, Matthew Lount, and Catherine Lauzon. 2012. Are Explanations Always Important?: A Study of Deployed, Low-cost Intelligent Interactive Systems. In Proc. IUI '12. ACM, New York, NY, USA, 169-- 178.

Digital Library

[5]

Margaret Burnett, Simone Stumpf, Jamie Macbeth, Stephann Makri, Laura Beckwith, Irwin Kwan, Anicia Peters, and William Jernigan. 2016. GenderMag: A method for evaluating software's gender inclusiveness. Interacting with Computers 28, 6 (2016), 760--787.

[6]

comScore. 2018. Latest rankings. Retrieved July, 2018 from https: //www.comscore.com/Insights/Rankings

[7]

Maartje de Graaf, Somaya Ben Allouch, and Jan van Dijk. 2017. Why Do They Refuse to Use My Robot?: Reasons for Non-Use Derived from a Long-Term Home Study. In Proc. HRI '17. ACM, New York, NY, USA, 224--233.

Digital Library

[8]

Defy Media. 2015. Damn You Auto Correct! Retrieved September, 2018 from http://www.damnyouautocorrect.com/

[9]

T. Deuschel and T. Scully. 2016. On the Importance of Spatial Perception for the Design of Adaptive User Interfaces. In 2016 IEEE 10th International Conference on Self-Adaptive and Self-Organizing Systems (SASO). 70--79.

[10]

Anind Dey, Jennifer Mankoff, Gregory Abowd, and Scott Carter. 2002. Distributed Mediation of Ambiguous Context in Aware Environments. In Proc. UIST '02. ACM, New York, NY, USA, 121--130.

Digital Library

[11]

Leah Findlater and Joanna McGrenere. 2004. A Comparison of Static, Adaptive, and Adaptable Menus. In Proc. CHI '04. ACM, New York, NY, USA, 89--96.

Digital Library

[12]

Krzysztof Z. Gajos, Mary Czerwinski, Desney S. Tan, and Daniel S. Weld. 2006. Exploring the Design Space for Adaptive Graphical User Interfaces. In Proc. AVI '06. ACM, New York, NY, USA, 201--208.

Digital Library

[13]

Krzysztof Z Gajos, Katherine Everitt, Desney S Tan, Mary Czerwinski, and Daniel S Weld. 2008. Predictability and accuracy in adaptive user interfaces. In CHI. ACM, 1271--1274.

Digital Library

[14]

Jonathan L. Herlocker, Joseph A. Konstan, and John Riedl. 2000. Explaining Collaborative Filtering Recommendations. In Proc. CSCW '00. ACM, New York, NY, USA, 241--250.

Digital Library

[15]

Setia Hermawati and Glyn Lawson. 2016. Establishing usability heuristics for heuristics evaluation in a specifc domain: Is there a consensus? Applied Ergonomics 56 (2016), 34 -- 51.

[16]

Kristina Höök. 2000. Steps to take before intelligent user interfaces become real. Interacting with Computers 12, 4 (2000), 409--426.

[17]

Eric Horvitz. 1999. Principles of Mixed-Initiative User Interfaces. In Proc. CHI '99. ACM, New York, NY, USA, 159--166.

Digital Library

[18]

Eric Horvitz, Jack Breese, David Heckerman, David Hovel, and Koos Rommelse. 1998. The Lumière Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users. In Proc. UAI '98. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 256--265.

Digital Library

[19]

IDC Corporate USA. 2018. IDC: The premier global market intelligence firm. Retrieved September, 2018 from https://www.idc.com/

[20]

Anthony Jameson. 2008. Adaptive interfaces and agents. In The human-computer interaction handbook: Fundamentals, evolving technologies and emerging applications (2nd ed.), Andrew Sears and Julie A. Jacko (Eds.). CRC Press, Boca Raton, FL, 433--458.

Digital Library

[21]

Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015. Principles of Explanatory Debugging to Personalize Interactive Machine Learning. In Proc. IUI '15. ACM, New York, NY, USA, 126--137.

Digital Library

[22]

Chia-Jung Lee, Jaime Teevan, and Sebastian de la Chica. 2014. Characterizing Multi-click Search Behavior and the Risks and Opportunities of Changing Results During Use. In Proc. SIGIR '14. ACM, New York, NY, USA, 515--524.

Digital Library

[23]

Brian Y. Lim and Anind K. Dey. 2009. Assessing Demand for Intelligibility in Context-aware Applications. In Proc. UbiComp '09. ACM, New York, NY, USA, 195--204.

Digital Library

[24]

Josh Lovejoy. 2018. The UX of AI. Google Design. Retrieved September, 2018 from https://design.google/library/ux-ai/

[25]

Josh Lovejoy and Jess Holbrook. 2017. Human-Centered Machine Learning. 7 steps to stay focused on the user when designing with ML. Medium. Retrieved September, 2018 from https://medium.com/google-design/human-centered-machinelearning-a770d10562cd

[26]

Ewa Luger and Abigail Sellen. 2016. "Like Having a Really Bad PA": The Gulf Between User Expectation and Experience of Conversational Agents. In Proc. CHI '16. ACM, New York, NY, USA, 5286--5297.

Digital Library

[27]

Aaron Maitland and Stanley Presser. 2016. How Accurately Do Different Evaluation Methods Predict the Reliability of Survey Questions? Journal of Survey Statistics and Methodology 4, 3 (2016), 362--381.

[28]

Joseph A Maxwell. 2012. Qualitative research design: An interactive approach. Vol. 41. Sage publications.

[29]

Chelsea Myers, Anushay Furqan, Jessica Nebolsky, Karina Caro, and Jichen Zhu. 2018. Patterns for How Users Overcome Obstacles in Voice User Interfaces. In Proc. CHI '18. ACM, New York, NY, USA, Article 6, 7 pages.

Digital Library

[30]

Jakob Nielsen. 1992. Finding Usability Problems Through Heuristic Evaluation. In Proc. CHI '92. ACM, New York, NY, USA, 373--380.

Digital Library

[31]

Jakob Nielsen and Rolf Molich. 1990. Heuristic Evaluation of User Interfaces. In Proc. CHI '90. ACM, New York, NY, USA, 249--256.

Digital Library

[32]

D.A. Norman. 1988. The psychology of everyday things. Basic Books, New York.

[33]

Donald A. Norman. 1994. How Might People Interact with Agents. Commun. ACM 37, 7 (July 1994), 68--71.

Digital Library

[34]

Kristen Olson. 2010. An examination of questionnaire evaluation by expert reviewers. Field Methods 22, 4 (2010), 295--318.

[35]

Martin Porcheron, Joel E. Fischer, Stuart Reeves, and Sarah Sharples. 2018. Voice Interfaces in Everyday Life. In Proc. CHI '18. ACM, New York, NY, USA, Article 640, 12 pages.

Digital Library

[36]

Emilee Rader, Kelley Cotter, and Janghee Cho. 2018. Explanations As Mechanisms for Supporting Algorithmic Transparency. In Proc. CHI '18. ACM, New York, NY, USA, Article 103, 13 pages.

Digital Library

[37]

Ruth Ravichandran, Sang-Wha Sien, Shwetak N. Patel, Julie A. Kientz, and Laura R. Pina. 2017. Making Sense of Sleep Sensors: How Sleep Sensing Technologies Support and Undermine Sleep Health. In Proc. CHI '17. ACM, New York, NY, USA, 6864--6875.

Digital Library

[38]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. In Proc. KDD '16. ACM, New York, NY, USA, 1135--1144.

Digital Library

[39]

Katharine Schwab. 2017. 10 Principles For Design In The Age Of AI. Retrieved September, 2018 from https://www.fastcompany.com/3067632/10-principles-for-designin-the-age-of-ai

[40]

SimilarWeb. 2018. Top websites ranking. Retrieved July, 2018 from https://www.similarweb.com/top-websites/united-states

[41]

Jack Stewart. 2018. Why Tesla's Autopilot Can't See a Stopped Firetruck. Retrieved September, 2018 from https: //www.wired.com/story/tesla-autopilot-why-crash-radar/

[42]

Erica Virtue. 2017. Designing with AI.

[43]

Jolita Vveinhardt and Evelina Gulbovaite. 2016. Expert evaluation of diagnostic instrument for personal and organizational value congruence. Journal of business ethics 136, 3 (2016), 481--501.

[44]

Daniel S Weld and Gagan Bansal. 2018. Intelligible Artificial Intelligence. arXiv preprint arXiv:1803.04263 (2018).

[45]

Rayoung Yang, Eunice Shin, Mark W. Newman, and Mark S. Ackerman. 2015. When Fitness Trackers Don'T 'Fit': End-user Difficulties in the Assessment of Personal Tracking Device Accuracy. In Proc. UbiComp '15. ACM, New York, NY, USA, 623--634.

Digital Library

Cited By

Rebholz SLibbrecht PMüller W(2025)Designing Trustworthy AI in Higher EducationAI - Ethical and Legal Challenges [Working Title]10.5772/intechopen.1009134Online publication date: 27-Feb-2025
https://doi.org/10.5772/intechopen.1009134
Kucher KZohrevandi EWestin C(2025)Towards Visual Analytics for Explainable AI in Industrial ApplicationsAnalytics10.3390/analytics40100074:1(7)Online publication date: 12-Feb-2025
https://doi.org/10.3390/analytics4010007
Rajarshi Tarafdar (2025)Human-AI Collaboration in Workflow Optimization: A Framework for Hybrid Decision Systems in Automation-Heavy IndustriesInternational Journal of Scientific Research in Computer Science, Engineering and Information Technology10.32628/CSEIT25111238411:1(3594-3613)Online publication date: 25-Feb-2025
https://doi.org/10.32628/CSEIT251112384
Show More Cited By

Index Terms

Guidelines for Human-AI Interaction
1. Computing methodologies
  1. Artificial intelligence
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Assessing Human-AI Interaction Early through Factorial Surveys: A Study on the Guidelines for Human-AI Interaction
This work contributes a research protocol for evaluating human-AI interaction in the context of specific AI products. The research protocol enables UX and HCI researchers to assess different human-AI interaction solutions and validate design decisions ...
Human-AI Interaction and AI Avatars
HCI International 2023 – Late Breaking Papers
Abstract
Human-Computer Interaction has been evolving rapidly with the advancement of artificial intelligence and metaverse. Human-AI Interaction is a new area in Human-Computer Interaction. In this paper, we look at AI avatars, which are human-like ...
Participatory Evaluation of Human-Data Interaction Design Guidelines
Human-Computer Interaction – INTERACT 2019
Abstract
The design of visual analytics tools for facilitating human-data interaction (HDI) plays a key role to help people identifying useful knowledge from large masses of data. Designing data visualization based on guidelines is relevant. However, it is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

May 2019

9077 pages

ISBN:9781450359702

DOI:10.1145/3290605

General Chairs:
Stephen Brewster
University of Glasgow, Scotland, UK
,
Geraldine Fitzpatrick
TU Wien, Austria
,
Program Chairs:
Anna Cox
University College London, UK
,
Vassilis Kostakos
University of Melbourne, Australia

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article

Conference

CHI '19

Sponsor:

SIGCHI

CHI '19: CHI Conference on Human Factors in Computing Systems

May 4 - 9, 2019

Glasgow, Scotland Uk

Acceptance Rates

CHI '19 Paper Acceptance Rate 703 of 2,958 submissions, 24%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

953
Total Citations
View Citations
26,677
Total Downloads

Downloads (Last 12 months)6,220
Downloads (Last 6 weeks)821

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Rebholz SLibbrecht PMüller W(2025)Designing Trustworthy AI in Higher EducationAI - Ethical and Legal Challenges [Working Title]10.5772/intechopen.1009134Online publication date: 27-Feb-2025
https://doi.org/10.5772/intechopen.1009134
Kucher KZohrevandi EWestin C(2025)Towards Visual Analytics for Explainable AI in Industrial ApplicationsAnalytics10.3390/analytics40100074:1(7)Online publication date: 12-Feb-2025
https://doi.org/10.3390/analytics4010007
Rajarshi Tarafdar (2025)Human-AI Collaboration in Workflow Optimization: A Framework for Hybrid Decision Systems in Automation-Heavy IndustriesInternational Journal of Scientific Research in Computer Science, Engineering and Information Technology10.32628/CSEIT25111238411:1(3594-3613)Online publication date: 25-Feb-2025
https://doi.org/10.32628/CSEIT251112384
Zhu YChu J(2025)Should we express gratitude in human–AI interaction: The online public’s moral stance toward artificial intelligence assistants in ChinaPublic Understanding of Science10.1177/09636625251314337Online publication date: 14-Feb-2025
https://doi.org/10.1177/09636625251314337
Schofield AWu SBayard de Volo TKuze TGomez ASultana S(2025)"My Very Subjective Human Interpretation": Domain Expert Perspectives on Navigating the Text Analysis Loop for Topic ModelsProceedings of the ACM on Human-Computer Interaction10.1145/37012019:1(1-30)Online publication date: 10-Jan-2025
https://dl.acm.org/doi/10.1145/3701201
Dev JAkhuseyinoglu NKayas GRashidi BGarg V(2025)Building Guardrails in AI Systems with Threat ModelingDigital Government: Research and Practice10.1145/36748456:1(1-18)Online publication date: 13-Feb-2025
https://doi.org/10.1145/3674845
Wang HYu ZRen ZZhang YLiu JWang LGuo B(2025)FingHV: Efficient Sharing and Fine-Grained Scheduling of Virtualized HPU ResourcesIEEE Transactions on Cybernetics10.1109/TCYB.2024.351856955:2(600-614)Online publication date: Feb-2025
https://doi.org/10.1109/TCYB.2024.3518569
Ngo MBoone APham TChakraborty I(2025)XGridDS: An Explainable Data Analytics Toolkit for Grid Operations2025 IEEE PES Grid Edge Technologies Conference & Exposition (Grid Edge)10.1109/GridEdge61154.2025.10887493(1-4)Online publication date: 21-Jan-2025
https://doi.org/10.1109/GridEdge61154.2025.10887493
Schmager SPappas IVassilakopoulou P(2025)Understanding Human-Centred AI: a review of its defining elements and a research agendaBehaviour & Information Technology10.1080/0144929X.2024.2448719(1-40)Online publication date: 16-Feb-2025
https://doi.org/10.1080/0144929X.2024.2448719
Jiang LHwang JBhagavatula CBras RLiang JLevine SDodge JSakaguchi KForbes MHessel JBorchardt JSorensen TGabriel STsvetkov YEtzioni OSap MRini RChoi Y(2025)Investigating machine moral judgement through the Delphi experimentNature Machine Intelligence10.1038/s42256-024-00969-67:1(145-160)Online publication date: 13-Jan-2025
https://doi.org/10.1038/s42256-024-00969-6
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten