Work in Progress

Do We Trust ChatGPT as much as Google Search and Wikipedia?

Authors:

S. Shyam SundarAuthors Info & Claims

CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

Article No.: 111, Pages 1 - 9

https://doi.org/10.1145/3613905.3650862

Published: 11 May 2024 Publication History

Abstract

Although studies, audits, and anecdotal observations have shown that information generated by ChatGPT is not always accurate, many users tend to show unwarranted trust in this new source. Do they consider ChatGPT to be like any other online information source such as Google and Wikipedia, without realizing that generative AI technology creates content that is not necessarily based on facts? Why do they trust information from ChatGPT? Understanding how users perceive content from generative AI tools is crucial because it can help reduce unwarranted trust in inaccurate information and mitigate the spread of misinformation. A focus group and interview study (N=14) revealed that thankfully not all users trust ChatGPT-generated information as much as Google Search and Wikipedia. It also shed light on the primary psychological considerations when trusting an online information source, namely perceived gatekeeping, and perceived information completeness. In addition, technological affordances such as interactivity and crowdsourcing were also found to be important for trust formation. We discuss theoretical and practical implications for design of generative AI interfaces.

Supplemental Material

MP4 File

Talk Video

Transcript for: Talk Video

MP4 File

Video Figure

Download
30.11 MB

References

[1]

Weiser Benjamin and Schweber Nate. 2023. Netherlands ’will pay the price’ for blocking Turkish visit – Erdoğan. The New York Times. https://www.nytimes.com/2023/06/08/nyregion/lawyer-chatgpt-sanctions.html

[2]

Lin Carolyn A and Atkin David J. 2002. Communication technology an society: Audience adoption and uses. Creskill, NJ: Hampton Press. 153–188 pages.

[3]

Steven H Chaffee. 1982. Mass media and interpersonal channels: Competitive, convergent, or complementary. Inter/media: Interpersonal Communication in a Media World 57 (1982), 77.

[4]

Gabriel De la Calzada and Alex Dekhtyar. 2010. On measuring the quality of Wikipedia articles. In Proceedings of the 4th workshop on Information credibility. 11–18.

Digital Library

[5]

Mohan J Dutta-Bergman. 2003. Health communication on the web: The roles of web use motivation and information completeness. Communication Monographs 70, 3 (2003), 264–274.

[6]

Upol Ehsan, Q Vera Liao, Michael Muller, Mark O Riedl, and Justin D Weisz. 2021. Expanding explainability: Towards social transparency in ai systems. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–19.

Digital Library

[7]

Houda Elmimouni, Andrea Forte, and Jonathan Morgan. 2022. Why People Trust Wikipedia Articles: Credibility Assessment Strategies Used by Readers. In Proceedings of the 18th International Symposium on Open Collaboration. 1–10.

Digital Library

[8]

Gunther Eysenbach, John Powell, Oliver Kuss, and Eun-Ryoung Sa. 2002. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. Journal of American Medical Association 287, 20 (2002), 2691–2700.

[9]

Andrew J Flanagin and Miriam J Metzger. 2000. Perceptions of Internet information credibility. Journalism & Mass communication quarterly 77, 3 (2000), 515–540.

[10]

Christina Haas and Stanley T Wearden. 2003. E-credibility: Building common ground in web environments. L1-Educational Studies in Language and Literature 3 (2003), 169–184.

[11]

Vikas Hassija, Vinay Chamola, Atmesh Mahapatra, Abhinandan Singal, Divyansh Goel, Kaizhu Huang, Simone Scardapane, Indro Spinelli, Mufti Mahmud, and Amir Hussain. 2023. Interpreting black-box models: a review on explainable artificial intelligence. Cognitive Computation (2023), 1–30.

[12]

Yifeng Hu and S Shyam Sundar. 2010. Effects of online health sources on credibility and behavioral intentions. Communication Research 37, 1 (2010), 105–132.

[13]

Yan Huang and S Shyam Sundar. 2022. Do we trust the crowd? Effects of crowdsourcing on perceived credibility of online health information. Health Communication 37, 1 (2022), 93–102.

[14]

Oliver P John and Richard W Robins. 1994. Accuracy and bias in self-perception: individual differences in self-enhancement and the role of narcissism.Journal of Personality and Social Psychology 66, 1 (1994), 206.

[15]

Yoon Jeon Koh and S Shyam Sundar. 2010. Heuristic versus systematic processing of specialist versus generalist sources in online media. Human Communication Research 36, 2 (2010), 103–124.

[16]

Teun Lucassen and Jan Maarten Schraagen. 2010. Trust in wikipedia: how users trust information from an unknown source. In Proceedings of the 4th workshop on Information credibility. 19–26.

Digital Library

[17]

Miriam J Metzger and Andrew J Flanagin. 2013. Credibility and trust of information in online environments: The use of cognitive heuristics. Journal of Pragmatics 59 (2013), 210–220.

[18]

Bing Pan, Helene Hembrooke, Thorsten Joachims, Lori Lorigo, Geri Gay, and Laura Granka. 2007. In Google we trust: Users’ decisions on rank, position, and relevance. Journal of Computer-Mediated Communication 12, 3 (2007), 801–823.

[19]

Pamela J Shoemaker, Martin Eichholz, Eunyi Kim, and Brenda Wrigley. 2001. Individual and routine forces in gatekeeping. Journalism & Mass communication Quarterly 78, 2 (2001), 233–246.

[20]

Anselm Strauss and Juliet Corbin. 1998. Basics of qualitative research techniques. (1998).

[21]

S Shyam Sundar. 2008. Self as source: Agency and customization in interactive media. In Mediated interpersonal communication. Routledge, 72–88.

[22]

S Shyam Sundar, Haiyan Jia, T Franklin Waddell, and Yan Huang. 2015. Toward a theory of interactive media effects (TIME) four models for explaining how interface features affect user psychology. The Handbook of the Psychology of Communication Technology (2015), 47–86.

[23]

S Shyam Sundar and Clifford Nass. 2000. Source orientation in human-computer interaction: Programmer, networker, or independent social actor. Communication Research 27, 6 (2000), 683–703.

[24]

S Shyam Sundar and Clifford Nass. 2001. Conceptualizing sources in online news. Journal of Communication 51, 1 (2001), 52–72.

[25]

James Surowiecki. 2005. The wisdom of crowds. Anchor.

Cited By

Zhelezniak MIshchenko O(2024)Some perspectives on the application of artificial intelligence in encyclopediasEntsyklopedychnyi visnyk Ukrainy [The Encyclopedia Herald of Ukraine]10.37068/evu.16.416(1-12)Online publication date: 2024
https://doi.org/10.37068/evu.16.4
Lermann Henestrosa AKimmerle J(2024)The Effects of Assumed AI vs. Human Authorship on the Perception of a GPT-Generated TextJournalism and Media10.3390/journalmedia50300695:3(1085-1097)Online publication date: 20-Aug-2024
https://doi.org/10.3390/journalmedia5030069

Index Terms

Do We Trust ChatGPT as much as Google Search and Wikipedia?
1. Human-centered computing
  1. Collaborative and social computing
  2. Human computer interaction (HCI)
    1. HCI design and evaluation methods
    2. Interaction paradigms
      1. Web-based interaction
2. Information systems
  1. World Wide Web
    1. Web applications
    2. Web interfaces
      1. Browsers

Index terms have been assigned to the content through auto-classification.

Recommendations

Higher education students' trust and use of ChatGPT: empirical evidence

This paper combines a modified version of the unified theory of acceptance and use of technology (UATAU) with the expectancy value theory (EVT) to examine the variables that influence higher education students' trust and use of ChatGPT. The quantitative ...
Testing an integrative theoretical model of knowledge-sharing behavior in the context of Wikipedia

This study explores how and why people participate in collaborative knowledge-building practices in the context of Wikipedia. Based on a survey of 223 Wikipedians, this study examines the relationship between motivations, internal cognitive beliefs, ...
Why People Trust Wikipedia Articles: Credibility Assessment Strategies Used by Readers
OpenSym '22: Proceedings of the 18th International Symposium on Open Collaboration

We examine how a diverse global readership assigns trust to Wikipedia articles, and the strategies they use to assess Wikipedia’s credibility. Through surveys and interviews, we develop and refine a Wikipedia trust taxonomy that describes the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

May 2024

4761 pages

ISBN:9798400703317

DOI:10.1145/3613905

Editors:
Florian Floyd Mueller
Monash University
,
Penny Kyburz
The Australian National University
,
Julie R. Williamson
University of Glasgow
,
Corina Sas
Lancaster University

Copyright © 2024 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 May 2024

Check for updates

Author Tags

Qualifiers

Work in progress
Research
Refereed limited

Conference

CHI '24

Sponsor:

CHI '24: CHI Conference on Human Factors in Computing Systems

May 11 - 16, 2024

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
1,607
Total Downloads

Downloads (Last 12 months)1,607
Downloads (Last 6 weeks)192

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhelezniak MIshchenko O(2024)Some perspectives on the application of artificial intelligence in encyclopediasEntsyklopedychnyi visnyk Ukrainy [The Encyclopedia Herald of Ukraine]10.37068/evu.16.416(1-12)Online publication date: 2024
https://doi.org/10.37068/evu.16.4
Lermann Henestrosa AKimmerle J(2024)The Effects of Assumed AI vs. Human Authorship on the Perception of a GPT-Generated TextJournalism and Media10.3390/journalmedia50300695:3(1085-1097)Online publication date: 20-Aug-2024
https://doi.org/10.3390/journalmedia5030069

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View full text|Download PDF

View Table of Conten