skip to main content
10.1145/3544549.3585727acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
Work in Progress

Something Borrowed: Exploring the Influence of AI-Generated Explanation Text on the Composition of Human Explanations

Published: 19 April 2023 Publication History

Abstract

Recent advances in Human-AI interaction have highlighted the possibility of employing AI in collaborative decision-making contexts, particularly in cases where the decision is subjective, without one ground truth. In these contexts, researchers argue that AI could be used not just to provide a final decision recommendation, but to surface new perspectives, rationales, and insights. In this late-breaking work, we describe the initial findings from an empirical study investigating how complementary AI input influences humans’ rationale in ambiguous decision-making. We use subtle sexism as an example of this context, and GPT-3 to create explanation-like text. We find that participants change the language, level of detail, and even the argumentative stance of their explanations after seeing the AI explanation text. They often borrow language directly from this complementary text. We discuss the implications for collaborative decision-making and the next steps in this research agenda.

Supplementary Material

MP4 File (3544549.3585727-talk-video.mp4)
Pre-recorded Video Presentation
MP4 File (3544549.3585727-video-preview.mp4)
Video Preview

References

[1]
Amina Adadi and Mohammed Berrada. 2018. Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE access 6 (2018), 52138–52160. https://doi.org/10.1109/ACCESS.2018.2870052
[2]
HLEG AI. 2019. High-level expert group on artificial intelligence. Technical Report. 6 pages.
[3]
Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, 2020. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information fusion 58 (2020), 82–115.
[4]
Gagan Bansal, Besmira Nushi, Ece Kamar, Daniel S Weld, Walter S Lasecki, and Eric Horvitz. 2019. Updates in human-ai teams: Understanding and addressing the performance/compatibility tradeoff. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 2429–2437.
[5]
Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the whole exceed its parts? the effect of ai explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16.
[6]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.
[7]
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models Are Few-Shot Learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems (Vancouver, BC, Canada) (NIPS’20). Curran Associates Inc., Red Hook, NY, USA, Article 159, 25 pages.
[8]
Erik Cambria, Lorenzo Malandri, Fabio Mercorio, Mario Mezzanzanica, and Navid Nobani. 2023. A survey on XAI and natural language explanations. Information Processing & Management 60, 1 (2023), 103111.
[9]
Samuel Carton, Qiaozhu Mei, and Paul Resnick. 2020. Feature-Based Explanations Don’t Help People Detect Misclassifications of Online Toxicity. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 14. 95–106.
[10]
Brian X Chen. 2022. How to Use ChatGPT and Still Be a Good Person. The New York Times (12 2022). https://www.nytimes.com/2022/12/21/technology/personaltech/how-to-use-chatgpt-ethically.htmln
[11]
Eric Chu, Deb Roy, and Jacob Andreas. 2020. Are visual explanations useful? a case study in model-in-the-loop prediction. arXiv preprint arXiv:2007.12248 (2020).
[12]
Berkeley J Dietvorst, Joseph P Simmons, and Cade Massey. 2015. Algorithm aversion: people erroneously avoid algorithms after seeing them err.Journal of Experimental Psychology: General 144, 1 (2015), 114.
[13]
Kate Donahue, Alexandra Chouldechova, and Krishnaram Kenthapadi. 2022. Human-Algorithm Collaboration: Achieving Complementarity and Avoiding Unfairness. In 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea) (FAccT ’22). Association for Computing Machinery, New York, NY, USA, 1639–1656. https://doi.org/10.1145/3531146.3533221
[14]
D Doran, SC Schulz, and TR Besold. 2018. What Does Explainable AI Really Mean? A New Conceptualization of Perspectives. In CEUR Workshop Proceedings, Vol. 2071.
[15]
Ben Green and Yiling Chen. 2019. The principles and limits of algorithm-in-the-loop decision making. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1–24.
[16]
Ronan Hamon, Henrik Junklewitz, Ignacio Sanchez, Gianclaudio Malgieri, and Paul De Hert. 2022. Bridging the gap between AI and explainability in the GDPR: towards trustworthiness-by-design in automated decision-making. IEEE Computational Intelligence Magazine 17, 1 (2022), 72–85.
[17]
Patrick Hemmer, Max Schemmer, Michael Vössing, and Niklas Kühl. 2021. Human-AI Complementarity in Hybrid Intelligence Systems: A Structured Literature Review.PACIS (2021), 78.
[18]
Alex Hern. 2022. AI-assisted plagiarism? ChatGPT bot says it has an answer for that. The Guardian (2022). https://www.theguardian.com/technology/2022/dec/31/ai-assisted-plagiarism-chatgpt-bot-says-it-has-an-answer-for-that
[19]
Nils Köbis and Luca D Mossink. 2021. Artificial intelligence versus Maya Angelou: Experimental evidence that people cannot differentiate AI-generated from human-written poetry. Computers in human behavior 114 (2021), 106553.
[20]
Vivian Lai, Chacha Chen, Q Vera Liao, Alison Smith-Renner, and Chenhao Tan. 2021. Towards a science of human-ai decision making: a survey of empirical studies. arXiv preprint arXiv:2112.11471 (2021).
[21]
Zachary C Lipton. 2018. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery.Queue 16, 3 (2018), 31–57.
[22]
Sean McGrath, Parth Mehta, Alexandra Zytek, Isaac Lage, and Himabindu Lakkaraju. 2020. When does uncertainty matter?: Understanding the impact of predictive uncertainty in ML assisted decision making. arXiv preprint arXiv:2011.06167 (2020).
[23]
Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence 267 (2019), 1–38.
[24]
Chelsea Mitamura, Lynnsey Erickson, and Patricia G Devine. 2017. Value-based standards guide sexism inferences for self and others. Journal of Experimental Social Psychology 72 (2017), 101–117.
[25]
Maayan Pereg, Uri Hertz, Ido Ben-Artzi, and Nitzan Shahar. 2022. Disentangling the contribution of individual and social learning processes in human advice-taking behavior. https://doi.org/10.31234/osf.io/28js3
[26]
Schaekermann, Mike. 2020. Human-AI Interaction in the Presence of Ambiguity: From Deliberation-based Labeling to Ambiguity-aware AI. PhD Thesis. UWSpace. http://hdl.handle.net/10012/16284
[27]
Giulia Vilone and Luca Longo. 2020. Explainable artificial intelligence: a systematic review. arXiv preprint arXiv:2006.00093 (2020).
[28]
Xinru Wang and Ming Yin. 2021. Are explanations helpful? a comparative study of the effects of explanations in ai-assisted decision-making. In 26th International Conference on Intelligent User Interfaces. 318–328.
[29]
Zeerak Waseem and Dirk Hovy. 2016. Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter. In Proceedings of the NAACL Student Research Workshop. Association for Computational Linguistics, San Diego, California, 88–93. https://doi.org/10.18653/v1/N16-2013
[30]
Feiyu Xu, Hans Uszkoreit, Yangzhou Du, Wei Fan, Dongyan Zhao, and Jun Zhu. 2019. Explainable AI: A brief survey on history, research areas, approaches and challenges. In CCF international conference on natural language processing and Chinese computing. Springer, 563–574.
[31]
Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 295–305.

Cited By

View all
  • (2024)Human-LLM Collaborative Annotation Through Effective Verification of LLM LabelsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641960(1-21)Online publication date: 11-May-2024
  • (2024)Using Generative Artificial Intelligence in University TeachingIntelligent Systems Design and Applications10.1007/978-3-031-64836-6_35(360-370)Online publication date: 25-Jul-2024

Index Terms

  1. Something Borrowed: Exploring the Influence of AI-Generated Explanation Text on the Composition of Human Explanations

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems
    April 2023
    3914 pages
    ISBN:9781450394222
    DOI:10.1145/3544549
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 19 April 2023

    Check for updates

    Author Tags

    1. Explainable AI
    2. GPT-3
    3. Human Explanations
    4. Human-AI Collaboration
    5. User Study

    Qualifiers

    • Work in progress
    • Research
    • Refereed limited

    Conference

    CHI '23
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)312
    • Downloads (Last 6 weeks)35
    Reflects downloads up to 20 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Human-LLM Collaborative Annotation Through Effective Verification of LLM LabelsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641960(1-21)Online publication date: 11-May-2024
    • (2024)Using Generative Artificial Intelligence in University TeachingIntelligent Systems Design and Applications10.1007/978-3-031-64836-6_35(360-370)Online publication date: 25-Jul-2024

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media