Dana McKay
Stephann Makri
George Buchanan
ABSTRACT
In a world where misinformation is abundant, and conspiracy theorists urge others to 'do their own research’, how do people use evidence in online discussions? What types of evidence do they provide, and for what purpose? Decades of human information interaction research has focused on making it easy to share and discuss information online; and decades of information literacy research have examined how to promote critical thinking and evaluation. However, there is a lack both of systematic analyses of evidence use in online discussions, and the ways community norms affect use of evidence in those discussions. We present a mixed methods analysis of the use of three formats of external evidence (images, links, and direct quotation by using blockquotes) across three Reddit communities with very different norms. One focuses on promoting conspiracy theories, another on debunking them, and a third on personal view change. We investigate the use of these evidence formats within and between communities to understand how evidence is used in different kinds of conversation. Our findings support the design of online information tools that promote good evidentiary practice.
- Bartlett, J.C., 2020. Information Literacy and Science Misinformation. In Navigating Fake News, Alternative Facts, and Misinformation in a Post-Truth World, K. Dalkir and R. Katz Eds. IGI Global, Hershey, PA, USA, 1-17. DOI= http://dx.doi.org/10.4018/978-1-7998-2543-2.ch001.Google Scholar
Cross Ref
- Blakeslee, S., 2004. The CRAAP test. LOEX Quarterly 31, 3, 4.Google Scholar
- Bode, L. and Vraga, E.K., 2018. See Something, Say Something: Correction of Global Health Misinformation on Social Media. Health Communication 33, 9, 1131-1140. DOI= http://dx.doi.org/10.1080/10410236.2017.1331312.Google ScholarCross Ref
- Boyd, J., 2002. Public and Technical Interdependence: Regulatory Controversy, Out-Law Discourse, and the Messy Case of Olestra. Argumentation and Advocacy 39, 2, 91-109. DOI= http://dx.doi.org/10.1080/00028533.2002.11821579.Google ScholarCross Ref
- Bozdag, E. and van den Hoven, J., 2015. Breaking the filter bubble: democracy and design. Ethics and Information Technology 17, 4, 249-265. DOI= http://dx.doi.org/10.1007/s10676-015-9380-y.Google ScholarDigital Library
- Bruckman, A., Danis, C., Lampe, C., Sternberg, J., and Waldron, C., 2006. Managing deviant behavior in online communities. In Proc. CHI EA 06 (Montréal, Québec, Canada), ACM, 1125458, 21-24. DOI= http://dx.doi.org/10.1145/1125451.1125458.Google ScholarDigital Library
- Bruckman, A., Luther, K., and Fiesler, C., 2015. When should we use real names in published accounts of internet research. In Digital research confidential: The secrets of studying behavior online, 243-258.Google Scholar
- Bruns, A., 2019. It's not the technology, stupid: How the ‘Echo Chamber’ and ‘Filter Bubble’ metaphors have failed us. In Proc. IAMCR 19 (Madrid, Spain), International Association for Media and Communication Research.Google Scholar
- Carrion, M.L., 2017. “You need to do your research”: Vaccines, contestable science, and maternal epistemology. Public Understanding of Science 27, 3, 310-324. DOI= http://dx.doi.org/10.1177/0963662517728024.Google ScholarCross Ref
- Chan, M.-p.S., Jones, C.R., Hall Jamieson, K., and Albarracín, D., 2017. Debunking: A Meta-Analysis of the Psychological Efficacy of Messages Countering Misinformation. Psyc. Sci 28, 11, 1531-1546. DOI= http://dx.doi.org/10.1177/0956797617714579.Google ScholarCross Ref
- Chancellor, S., Hu, A., and Choudhury, M.D., 2018. Norms Matter: Contrasting Social Support Around Behavior Change in Online Weight Loss Communities. In Proc. CHI '18 (Montreal QC, Canada), Association for Computing Machinery, Paper 666. DOI= http://dx.doi.org/10.1145/3173574.3174240.Google ScholarDigital Library
- Chandrasekharan, E., Pavalanathan, U., Srinivasan, A., Glynn, A., Eisenstein, J., and Gilbert, E., 2017. You Can't Stay Here: The Efficacy of Reddit's 2015 Ban Examined Through Hate Speech. In Proc. CSCW 17, Article 31. DOI= http://dx.doi.org/10.1145/3134666.Google ScholarDigital Library
- Chandrasekharan, E., Samory, M., Jhaver, S., Charvat, H., Bruckman, A., Lampe, C., Eisenstein, J., and Gilbert, E., 2018. The Internet's Hidden Rules: An Empirical Study of Reddit Norm Violations at Micro, Meso, and Macro Scales. In Proc. CSCW 17, Article 32. DOI= http://dx.doi.org/10.1145/3274301.Google ScholarDigital Library
- Chen, C.-C. and Roth, C., 2012. {{Citation needed}}: the dynamics of referencing in Wikipedia. In Proc. WikiSym 12 (Linz, Austria), Association for Computing Machinery, Article 8. DOI= http://dx.doi.org/10.1145/2462932.2462943.Google ScholarDigital Library
- Chen, X., Sin, S.-C.J., Theng, Y.-L., and Lee, C.S., 2015. Why Do Social Media Users Share Misinformation? In Proc. JCDL 15 (Knoxville, Tennessee, USA), Association for Computing Machinery, 111–114. DOI= http://dx.doi.org/10.1145/2756406.2756941.Google ScholarDigital Library
- Colliander, J., 2019. “This is fake news”: Investigating the role of conformity to other users’ views when commenting on and spreading disinformation in social media. Comp. Hum. Behav. 97, 202-215. DOI= http://dx.doi.org/10.1016/j.chb.2019.03.032.Google ScholarDigital Library
- Connaway, L.S., Julien, H., Seadle, M., and Kasprak, A., 2017. Digital literacy in the era of fake news: Key roles for information professionals. ASIST Proceedings 54, 1, 554-555. DOI= http://dx.doi.org/10.1002/pra2.2017.14505401070.Google ScholarCross Ref
- de Souza, C.S. and Preece, J., 2004. A framework for analyzing and understanding online communities. Interacting with Computers 16, 3, 579-610. DOI= http://dx.doi.org/10.1016/j.intcom.2003.12.006.Google ScholarCross Ref
- Del Vicario, M., Bessi, A., Zollo, F., Petroni, F., Scala, A., Caldarelli, G., Stanley, H.E., and Quattrociocchi, W., 2016. The spreading of misinformation online. PNAS 113, 3, 554. DOI= http://dx.doi.org/10.1073/pnas.1517441113.Google ScholarCross Ref
- Draws, T., Inel, O., Tintarev, N., Baden, C., and Timmermans, B., 2022. Comprehensive Viewpoint Representations for a Deeper Understanding of User Interactions With Debated Topics. In Proc. CHIIR 22 (Regensburg, Germany), Association for Computing Machinery, 135–145. DOI= http://dx.doi.org/10.1145/3498366.3505812.Google ScholarDigital Library
- Farrell, J., McConnell, K., and Brulle, R., 2019. Evidence-based strategies to combat scientific misinformation. Nature Climate Change 9, 3, 191-195. DOI= http://dx.doi.org/10.1038/s41558-018-0368-6.Google ScholarCross Ref
- Flaxman, S., Goel, S., and Rao, J.M., 2016. Filter Bubbles, Echo Chambers, and Online News Consumption. Public Opinion Q 80, S1, 298-320. DOI= http://dx.doi.org/10.1093/poq/nfw006.Google ScholarCross Ref
- Forte, A., Andalibi, N., Gorichanaz, T., Kim, M.C., Park, T., and Halfaker, A., 2018. Information Fortification: An Online Citation Behavior. In Proc. GROUP 18 (Sanibel Island, Florida, USA), Association for Computing Machinery, 83–92. DOI= http://dx.doi.org/10.1145/3148330.3148347.Google ScholarDigital Library
- Gilbert, S.A., 2020. "I run the world's largest historical outreach project and it's on a cesspool of a website." Moderating a Public Scholarship Site on Reddit: A Case Study of r/AskHistorians. In Proc. CSCW 20, Article 19. DOI= http://dx.doi.org/10.1145/3392822.Google ScholarDigital Library
- Golbeck, J., 2012. The twitter mute button: a web filtering challenge. In Proc. CHI 2012 (Austin, Texas, USA), Association for Computing Machinery, 2755–2758. DOI= http://dx.doi.org/10.1145/2207676.2208673.Google ScholarDigital Library
- Goodnight, G.T., 1982. The Personal, Technical, and Public Spheres of Argument: A Speculative Inquiry into the Art of Public Deliberation. J. Amer. Forensic Assn 18, 4, 214-227. DOI= http://dx.doi.org/10.1080/00028533.1982.11951221.Google ScholarCross Ref
- Halpern, S., 2019. Why The UK Condemned Facebook for Fuelling Fake News. In New Yorker.Google Scholar
- He, L. and He, C., 2022. Help Me #DebunkThis: Unpacking Individual and Community's Collaborative Work in Information Credibility Assessment. In Proc. CSCW 22 (Taipei, Taiwan), ACM. DOI= http://dx.doi.org/10.1145/3555138.Google ScholarDigital Library
- Helberger, N., Karppinen, K., and D'Acunto, L., 2018. Exposure diversity as a design principle for recommender systems. Inf. Comm & Soc. 21, 2, 191-207. DOI= http://dx.doi.org/10.1080/1369118X.2016.1271900.Google ScholarCross Ref
- Jones-Jang, S.M., Mortensen, T., and Liu, J., 2021. Does Media Literacy Help Identification of Fake News? Information Literacy Helps, but Other Literacies Don't. Amer. Behav. Sci 65, 2, 371-388. DOI= http://dx.doi.org/10.1177/0002764219869406.Google ScholarCross Ref
- Khan, M.L. and Idris, I.K., 2019. Recognise misinformation and verify before sharing: a reasoned action and information literacy perspective. Behav. & IT 38, 12, 1194-1212. DOI= http://dx.doi.org/10.1080/0144929X.2019.1578828.Google ScholarCross Ref
- Klein, C., Clutton, P., and Dunn, A.G., 2019. Pathways to conspiracy: The social and linguistic precursors of involvement in Reddit's conspiracy theory forum. PLOS One 14, 11, e0225098. DOI= http://dx.doi.org/10.1371/journal.pone.0225098.Google ScholarCross Ref
- Lampe, C., Zube, P., Lee, J., Park, C.H., and Johnston, E., 2014. Crowdsourcing civility: A natural experiment examining the effects of distributed moderation in online forums. Gov Info Q. 31, 2, 317-326. DOI= http://dx.doi.org/https://doi.org/10.1016/j.giq.2013.11.005.Google ScholarCross Ref
- Leverich, D., 2018. Evidence in Online Political Discourse: How Everyday Citizens Argue about Politics on Social Media. University of Dayton, Dayton, OH.Google Scholar
- Levy, N., 2022. Do your own research! Synthese 200, 5, 356. DOI= http://dx.doi.org/10.1007/s11229-022-03793-w.Google ScholarCross Ref
- Lewandowsky, S., Ecker, U.K.H., Seifert, C.M., Schwarz, N., and Cook, J., 2012. Misinformation and Its Correction: Continued Influence and Successful Debiasing. Psyc Sci in Public Interest 13, 3, 106-131. DOI= http://dx.doi.org/10.1177/1529100612451018.Google ScholarCross Ref
- Lu, J., Chiu, M.M., and Law, N.W., 2011. Collaborative argumentation and justifications: A statistical discourse analysis of online discussions. Comput. Hum. Behav. 27, 2, 946-955. DOI= http://dx.doi.org/https://doi.org/10.1016/j.chb.2010.11.021.Google ScholarDigital Library
- Matias, J.N., 2019. Preventing harassment and increasing group participation through social norms in 2,190 online science discussions. PNAS 116, 20, 9785-9789. DOI= http://dx.doi.org/10.1073/pnas.1813486116.Google ScholarCross Ref
- Mckay, D., Makri, S., Gutierrez-Lopez, M., MacFarlane, A., Missaoui, S., Porlezza, C., and Cooper, G., 2020. We are the Change that we Seek: Information Interactions During a Change of Viewpoint. In Proc. CHIIR 20 (Vancouver BC, Canada), Association for Computing Machinery, 173–182. DOI= http://dx.doi.org/10.1145/3343413.3377975.Google ScholarDigital Library
- McKay, D., Owyong, K., Makri, S., and Lopez, M.G., 2022. Turn and Face the Strange: Investigating Filter Bubble Bursting Information Interactions. In Proc. CHIIR 22 (Regensburg, Germany), Association for Computing Machinery, 233–242. DOI= http://dx.doi.org/10.1145/3498366.3505822.Google ScholarDigital Library
- Medvedev, A.N., Lambiotte, R., and Delvenne, J.-C., 2019. The Anatomy of Reddit: An Overview of Academic Research. In Proc. DOOCN 17 (Indianapolis, IN), Springer International Publishing, 183-204. DOI= http://dx.doi.org/10.1007/978-3-030-14683-2_9.Google ScholarCross Ref
- Mercier, H. and Sperber, D., 2011. Why do humans reason? Arguments for an argumentative theory. Behav. Brain Scie 34, 2, 57-74. DOI= http://dx.doi.org/https://doi.org/10.1017/S0140525X10000968.Google ScholarCross Ref
- Nonnecke, B. and Preece, J., 2000. Lurker demographics: counting the silent. In Proc. CHI 00 (The Hague, The Netherlands), Association for Computing Machinery, 73–80. DOI= http://dx.doi.org/10.1145/332040.332409.Google ScholarDigital Library
- Öcal, A., Xiao, L., and Park, J., 2021. Reasoning in social media: insights from Reddit “Change My View” submissions. Online Info. Rev. 45, 7, 1208-1226. DOI= http://dx.doi.org/10.1108/OIR-08-2020-0330.Google ScholarCross Ref
- Oxford English Dictionary, "evidence, n.". Oxford University Press.Google Scholar
- Petty, E., S.C., W., and Tormala, Z., 2003. Persuasion and Attitude Change. In Handbook of Psychology Wiley, 353-382. DOI= http://dx.doi.org/10.1002/0471264385.wei0515.Google ScholarCross Ref
- Preece, J., 2000. Online communities: Designing usability and supporting socialbilty. John Wiley & Sons, Inc.Google ScholarDigital Library
- Preece, J., 2001. Online Communities: Usability, Sociabilty, Theory and Methods. In Frontiers of Human-Centered Computing, Online Communities and Virtual Environments, R.A. Earnshaw, R.A. Guedj, A.v. Dam and J.A. Vince Eds. Springer London, London, 263-277. DOI= http://dx.doi.org/10.1007/978-1-4471-0259-5_18.Google ScholarCross Ref
- Priniski, J.H. and Horne, Z., 2018. Attitude Change on Reddit's Change My View. In Proc. CogSci18 (Madison, WS), Cognitive Science Society.Google Scholar
- Proferes, N., Jones, N., Gilbert, S., Fiesler, C., and Zimmer, M., 2021. Studying Reddit: A Systematic Overview of Disciplines, Approaches, Methods, and Ethics. Social Media + Society 7, 2, 20563051211019004. DOI= http://dx.doi.org/10.1177/20563051211019004.Google ScholarCross Ref
- Rajadesingan, A., Resnick, P., and Budak, C., 2020. Quick, Community-Specific Learning: How Distinctive Toxicity Norms Are Maintained in Political Subreddits. In Proc. ICWSM '20, 557-568. DOI= http://dx.doi.org/10.1609/icwsm.v14i1.7323.Google ScholarCross Ref
- Rieger, A., Draws, T., Theune, M., and Tintarev, N., 2021. This Item Might Reinforce Your Opinion: Obfuscation and Labeling of Search Results to Mitigate Confirmation Bias. In Proc. HT21 (Virtual Event, USA), Association for Computing Machinery, 189–199. DOI= http://dx.doi.org/10.1145/3465336.3475101.Google ScholarDigital Library
- Rubin Victoria, L., 2019. Disinformation and misinformation triangle: A conceptual model for “fake news” epidemic, causal factors and interventions. J Doc 75, 5, 1013-1034. DOI= http://dx.doi.org/10.1108/JD-12-2018-0209.Google ScholarCross Ref
- Saling, L.L., Mallal, D., Scholer, F., Skelton, R., and Spina, D., 2021. No one is immune to misinformation: An investigation of misinformation sharing by subscribers to a fact-checking newsletter. PLOS ONE 16, 8, e0255702. DOI= http://dx.doi.org/10.1371/journal.pone.0255702.Google ScholarCross Ref
- Samory, M. and Mitra, T., 2018. Conspiracies Online: User Discussions in a Conspiracy Community Following Dramatic Events. In Proc. AAAI 18.Google Scholar
- Scolyer-Gray, P., 2022. Artistic Works of Fiction and Falsehood: An Analysis of the Production and Consumption of Knowledge on 4chan. Patrick Scolyer-Gray.Google Scholar
- Seargeant, P. and Tagg, C., 2019. Social media and the future of open debate: A user-oriented approach to Facebook's filter bubble conundrum. Doiscourse Context and Media 27, 41-48. DOI= http://dx.doi.org/10.1016/j.dcm.2018.03.005.Google ScholarCross Ref
- Snavely, L. and Cooper, N., 1997. The information literacy debate. J. Acad. Libr. 23, 1, 9-14. DOI= http://dx.doi.org/https://doi.org/10.1016/S0099-1333(97)90066-5.Google ScholarCross Ref
- Srinivasan, K.B., Danescu-Niculescu-Mizil, C., Lee, L., and Tan, C., 2019. Content Removal as a Moderation Strategy: Compliance and Other Outcomes in the ChangeMyView Community. In Proc. CSCW 19, Article 163. DOI= http://dx.doi.org/10.1145/3359265.Google ScholarDigital Library
- Stefanone, M.A., Vollmer, M., and Covert, J.M., 2019. In News We Trust? Examining Credibility and Sharing Behaviors of Fake News. In Proc. SMS19 (Toronto, ON, Canada), Association for Computing Machinery, 136–147. DOI= http://dx.doi.org/10.1145/3328529.3328554.Google ScholarDigital Library
- Suh, A. and Wagner, C., 2013. Factors Affecting Individual Flaming in Virtual Communities. In Proc. HICSS '13, 3282-3291. DOI= http://dx.doi.org/10.1109/HICSS.2013.230.Google ScholarDigital Library
- Tambuscio, M., Ruffo, G., Flammini, A., and Menczer, F., 2015. Fact-checking Effect on Viral Hoaxes: A Model of Misinformation Spread in Social Networks. In Proc. WWW 15 (Florence, Italy), Association for Computing Machinery, 977–982. DOI= http://dx.doi.org/10.1145/2740908.2742572.Google ScholarDigital Library
- Terveen, L., Konstan, J., and Lampe, C., 2014. Study, Build, Repeat: Using Online Communities as a Research Platform. In Ways of Knowing in HCI, J.S. Olson and W.A. Kellogg Eds. Springer New York, 95-117. DOI= http://dx.doi.org/10.1007/978-1-4939-0378-8_5.Google ScholarCross Ref
- Tewell, E., 2015. A decade of critical information literacy: A review of the literature. Comm. Info. Lit. 9, 1, 2. DOI= http://dx.doi.org/10.15760/comminfolit.2015.9.1.174.Google ScholarCross Ref
- Vranic, A., Hromatko, I., and Tonković, M., 2022. "I Did My Own Research": Overconfidence, (Dis)trust in Science, and Endorsement of Conspiracy Theories. Front. Psychol. 13, 931865. DOI= http://dx.doi.org/10.3389/fpsyg.2022.931865.Google ScholarCross Ref
- Weeks, B.E., Ksiazek, T.B., and Holbert, R.L., 2016. Partisan Enclaves or Shared Media Experiences? A Network Approach to Understanding Citizens’ Political News Environments. J Broadcasting Elec Media 60, 2, 248-268. DOI= http://dx.doi.org/10.1080/08838151.2016.1164170.Google ScholarCross Ref
- Wu, L., Morstatter, F., Carley, K.M., and Liu, H., 2019. Misinformation in Social Media: Definition, Manipulation, and Detection. SIGKDD Explor. Newsl. 21, 2, 80–90. DOI= http://dx.doi.org/10.1145/3373464.3373475.Google ScholarDigital Library
- Xiao, L. and Khazaei, T., 2019. Changing Others' Beliefs Online: Online Comments' Persuasiveness. In Proc. SMS18 (Toronto, ON, Canada), Association for Computing Machinery, 92–101. DOI= http://dx.doi.org/10.1145/3328529.3328549.Google ScholarDigital Library
- Yom-Tov, E., Dumais, S., and Guo, Q., 2013. Promoting Civil Discourse Through Search Engine Diversity. Soc Sci Comp Rev 32, 2, 145-154. DOI= http://dx.doi.org/10.1177/0894439313506838.Google ScholarDigital Library
- Zannettou, S., Sirivianos, M., Blackburn, J., and Kourtellis, N., 2019. The Web of False Information: Rumors, Fake News, Hoaxes, Clickbait, and Various Other Shenanigans. J. Data and Inf. Quality 11, 3, Article 10. DOI= http://dx.doi.org/10.1145/3309699.Google ScholarDigital Library
- Zhang, H., Alim, M.A., Li, X., Thai, M.T., and Nguyen, H.T., 2016. Misinformation in Online Social Networks: Detect Them All with a Limited Budget. ToIS 34, 3, Article 18. DOI= http://dx.doi.org/10.1145/2885494.Google ScholarDigital Library
- Zhou, Y. and Farzan, R., 2021. Designing to Stop Live Streaming Cyberbullying: A case study of Twitch Live Streaming Platform. In Proc. C&T 21 (Seattle, WA, USA), Association for Computing Machinery, 138–150. DOI= http://dx.doi.org/10.1145/3461564.3461574.Google ScholarDigital Library
Index Terms
- [citation needed]: An Examination of Types and Purpose of Evidence Provided in Three Online Discussions on Reddit
Recommendations
A Characterization of Political Communities on Reddit
HT '19: Proceedings of the 30th ACM Conference on Hypertext and Social MediaThe social news aggregator Reddit is among the most popular websites on the internet. Many online users use the platform to anonymously share and discuss (mostly US-centric) political content. In this ongoing work, we perform a comparative large-scale ...
Characterizing Conversation Patterns in Reddit: From the Perspectives of Content Properties and User Participation Behaviors
COSN '15: Proceedings of the 2015 ACM on Conference on Online Social NetworksIt becomes the norm for people to communicate with one another through various online social channels, where different conversation structures are formed depending on platforms. One of the common online communication patterns is a threaded conversation ...
Disinformation Warfare: Understanding State-Sponsored Trolls on Twitter and Their Influence on the Web
WWW '19: Companion Proceedings of The 2019 World Wide Web ConferenceOver the past couple of years, anecdotal evidence has emerged linking coordinated campaigns by state-sponsored actors with efforts to manipulate public opinion on the Web, often around major political events, through dedicated accounts, or “trolls.” ...
Comments