Skip to main content
SpringerLink
Log in

Testing Users’ Ability to Recognize Fake News in Three Countries. An Experimental Perspective

  • Conference paper
  • First Online:
Social Computing and Social Media: Experience Design and Social Network Analysis (HCII 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12774))

Included in the following conference series:

  • 2611 Accesses

Abstract

Fake news dissemination online can negatively affect public deliberation and opinion formation as well as contribute to conflicts in society. In recent years, the spread of misinformation has attracted ample attention of scholars who examine why people believe fake news. Nonetheless, existing research focusing on this phenomenon often lacks comparative perspective. This article is devoted to the methodological aspects of designing a cross-national online experiment on fake news perception. Based on our recent study, which tested the influence of news frames, sources and thinking styles on the ability to recognize fake news about foreign countries in Russia, Ukraine, and Kazakhstan, we provide a set of methodological steps that could be taken to design such an experiment. In particular, we demonstrate the necessity to create unique sets of stimulus material adjusted to specific national media systems. We also discuss the operationalization of complex variables (i.e. frames) and format of message presentation. Furthermore, we report the results of recruitment of participants through the ad campaigns on social network sites (Facebook and VKontakte) and suggest the approaches to creating samples representative of these platforms’ populations. This article is aimed to be a methodological toolkit, which points at the challenges that may occur while designing a cross-national experiment on media consumption—and provides practical recommendations to overcome them.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Huang, H.: A War of (mis)information: the political effects of rumors and rumor rebuttals in an authoritarian country. Br. J. Polit. Sci. 47(2), 283–311 (2017). https://doi.org/10.1017/S0007123415000253

    Article  Google Scholar 

  2. Larson, H.: The biggest pandemic risk? viral misinformation. Nature 562, 309 (2018). https://doi.org/10.1038/d41586-018-07034-4

    Article  Google Scholar 

  3. Van Duyn, E., Collier, J.: Priming and fake news: the effects of elite discourse on evaluations of news media. Mass Commun. Soc. 22(1), 29–48 (2019). https://doi.org/10.1080/15205436.2018.1511807

    Article  Google Scholar 

  4. Tucker, J.A., et al.: Social media, political polarization, and political disinformation: a review of the scientific literature, pp. 1–95 (2018). https://doi.org/10.2139/ssrn.3144139

  5. Nyhan, B.: Why the “death panel” myth wouldn’t die: misinformation in the health care reform debate. The Forum 8(1) (2010). https://doi.org/10.2202/1540-8884.1354

  6. Lee, T., Hosam, C.: Fake news is real: the significance and sources of disbelief in mainstream media in Trump’s America. Sociol. Forum 35(S1), 996–1018 (2020). https://doi.org/10.1111/socf.12603

    Article  Google Scholar 

  7. Martens, B., Aguiar, L., Gomez, E., Mueller-Langer, F.: The digital transformation of news media and the rise of disinformation and fake news - An economic perspective. Digital Economy Working Paper 2018–02; JRC Technical Reports (2018). https://doi.org/10.2139/ssrn.3164170

  8. Lazer, D., et al.: The science of fake news. Science 359(6380), 1094–1096 (2018). https://doi.org/10.1126/science.aao2998

  9. Allcott, H., Gentzkow, M.: Social media and fake news in the 2016 election. J. Econ. Perspect. 31(2), 211–236 (2017). https://doi.org/10.1257/jep.31.2.211

    Google Scholar 

  10. Benkler, Y., Faris, R., Roberts, H.: Network Propaganda: Manipulation, disinformation, and radicalization in American politics. Oxford University Press, New York (2018). https://doi.org/10.1093/oso/9780190923624.001.0001

  11. Brewer, P.R., Graf, J., Willnat, L.: Priming or framing: media influence on attitudes toward foreign countries. Int. Commun. Gaz. 65(6), 493–508 (2003). https://doi.org/10.1177/0016549203065006005

    Article  Google Scholar 

  12. Wanta, W., Golan, G., Lee, C.: Agenda setting and international news: media influence on public perceptions of foreign nations. Journalism Mass Commun. Quart. 81(2), 364–377 (2004). https://doi.org/10.1177/107769900408100209

    Article  Google Scholar 

  13. Lewandowsky, S., Stritzke, W.G.K., Freund, A.M., Oberauer, K., Krueger, J.I.: Misinformation, disinformation, and violent conflict: from Iraq and the “War on Terror” to future threats to peace. Am. Psychol. 68(7), 487–501 (2013). https://doi.org/10.1037/a0034515

    Article  Google Scholar 

  14. Colvin, R.M., Witt, G.B., Lacey, J., McCrea, R.: The role of conflict framing and social identity in public opinion about land use change: an experimental test in the Australian context. Environ. Policy Gov. 30(2), 84–98 (2020). https://doi.org/10.1002/eet.1879

    Article  Google Scholar 

  15. Martin, J.D., Hassan, F.: News media credibility ratings and perceptions of online fake news exposure in five countries. J. Stud. 21(16), 2215–2233 (2020). https://doi.org/10.1080/1461670X.2020.1827970

    Article  Google Scholar 

  16. Humprecht, E.: Where ‘fake news’ flourishes: a comparison across four Western democracies. Inf. Commun. Soc. 22(13), 1973–1988 (2019). https://doi.org/10.1080/1369118X.2018.1474241

    Article  Google Scholar 

  17. Humprecht, E.: How do they debunk “fake news”? a cross-national comparison of transparency in fact checks. Digit. Journal. 8(3), 310–327 (2020). https://doi.org/10.1080/21670811.2019.1691031

    Article  Google Scholar 

  18. Chandler, J., Rosenzweig, C., Moss, A.J., Robinson, J., Litman, L.: Online panels in social science research: expanding sampling methods beyond Mechanical Turk. Behav. Res. Methods 51(5), 2022–2038 (2019). https://doi.org/10.3758/s13428-019-01273-7

    Article  Google Scholar 

  19. Porshnev, A., Miltsov, A., Lokot, T., Koltsova, O.: Effects of conspiracy thinking style, framing and political interest on accuracy of fake news recognition by social media users: evidence from Russia, Kazakhstan and Ukraine. In: Social Computing and Social Media. Design, Ethics, User Behavior, and Social Network Analysis. Springer, Cham (in press)

    Google Scholar 

  20. Kohn, M.: Cross National Research in Sociology. SAGE Publications. (1989)

    Google Scholar 

  21. Charness, G., Gneezy, U., Kuhn, M.A.: Experimental methods: between-subject and within-subject design. J. Econ. Behav. Organ. 81(1), 1–8 (2012). https://doi.org/10.1016/j.jebo.2011.08.009

    Article  Google Scholar 

  22. Reeves, B., Geiger, S.: Designing experiments that assess psychological responses to media messages. In: Lang A. (ed.) Measuring Psychological Responses to Media Messages, 1st edn., pp. 165–180. Routledge, New York (1994). https://doi.org/10.4324/9780203812853

  23. Tao, C.-C., Bucy, E.P.: Conceptualizing media stimuli in experimental research: psychological versus attribute-based definitions. Human Commun. Rese. 33(4), 397–426 (2007). https://doi.org/10.1111/j.1468-2958.2007.00305.x

    Article  Google Scholar 

  24. Thorson, E., Wicks, R., Leshner, G.: Experimental methodology in journalism and mass communication research. Journalism Mass Commun. Quart. 89(1), 112–124 (2012). https://doi.org/10.1177/1077699011430066

    Article  Google Scholar 

  25. Slater, M.D.: Use of message stimuli in mass communication experiments: a methodological assessment and discussion. Journalism Mass Commun. Q. 68(3), 412–421 (1991). https://doi.org/10.1177/107769909106800312

    Article  Google Scholar 

  26. Entman, R. M.: Projections of Power: Framing News, Public Opinion, and U.S. Foreign Policy. University of Chicago Press (2004)

    Google Scholar 

  27. Iyengar, S.: Speaking of values: the framing of American politics. The Forum 3(3) (2005). https://doi.org/10.2202/1540-8884.1093

  28. Druckman, J.N.: On the limits of framing effects: who can frame? J. Polit. 63(4), 1041–1066 (2001). https://www.jstor.org/stable/2691806

  29. Cacciatore, M.A., Scheufele, D.A., Iyengar, S.: The end of framing as we know it … and the future of media effects. Mass Commun. Soc. 19(1), 7–23 (2016). https://doi.org/10.1080/15205436.2015.1068811

    Article  Google Scholar 

  30. D’Angelo, P.: Framing: media frames. In: Roessler, P., Hoffner, C. A., Van Zoonen, L. (eds.) The International Encyclopedia of Media Effects, pp. 1–10. Wiley, New York (2017). https://doi.org/10.1002/9781118783764.wbieme0048

  31. Pennycook, G., Rand, D.G.: Lazy, not biased: susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. Cognition 188, 39–50 (2019). https://doi.org/10.1016/j.cognition.2018.06.011

    Article  Google Scholar 

  32. Hameleers, M., Powell, T.E., Van Der Meer, T.G.L.A., Bos, L.: A picture paints a thousand lies? the effects and mechanisms of multimodal disinformation and rebuttals disseminated via social media. Polit. Commun. 37(2), 281–301 (2020). https://doi.org/10.1080/10584609.2019.1674979

    Article  Google Scholar 

  33. Pennycook, G., Cannon, T.D., Rand, D.G.: Prior exposure increases perceived accuracy of fake news. J. Exp. Psychol. Gen. 147(12), 1865–1880 (2018). https://doi.org/10.1037/xge0000465

    Article  Google Scholar 

  34. Koltsova, O., Pashakhin, S.: Agenda divergence in a developing conflict: quantitative evidence from Ukrainian and Russian TV newsfeeds. Media, War & Conflict 13(3), 1–21 (2019). https://doi.org/10.1177/1750635219829876

  35. Fierman, W.: Language and education in Post-Soviet Kazakhstan: Kazakh-medium instruction in urban schools. Russian Rev. 65(1), 98–116 (2006). https://www.jstor.org/stable/3664037

  36. Bilaniuk, L., Melnyk, S.: A Tense and shifting balance: bilingualism and education in Ukraine. Int. J. Biling. Educ. Biling. 11(3–4), 340–372 (2008). https://doi.org/10.1080/13670050802148731

    Article  Google Scholar 

  37. Kulyk, V.: Language identity, linguistic diversity and political cleavages: evidence from Ukraine. Nations Nationalism 17(3), 627–648 (2011). https://doi.org/10.1111/j.1469-8129.2011.00493.x

    Article  Google Scholar 

  38. Pennycook, G., Rand, D.G.: Who falls for fake news? the roles of bullshit receptivity, overclaiming, familiarity, and analytic thinking. J. Pers. 88, 185–200 (2020). https://doi.org/10.1111/jopy.12476

    Article  Google Scholar 

  39. Gabielkov, M., Ramachandran, A., Chaintreau, A., Legout, A.: Social clicks: what and who gets read on Twitter ? In: Proceedings of ACM SIGMETRICS/IFIP Performance 2016. (2016). https://hal.inria.fr/hal-01281190

  40. Andrew, B.C.: Media-generated shortcuts: do newspaper headlines present another roadblock for low-information rationality ? Int. J. Press/Politics 12(2), 24–43 (2007). https://doi.org/10.1177/1081180X07299795

    Article  Google Scholar 

  41. Ecker, U.K.H., Lewandowsky, S., Chang, E.P., Pillai, R.: The effects of subtle misinformation in news headlines. J. Exp. Psychol. Appl. 20(4), 323–335 (2014). https://doi.org/10.1037/xap0000028

    Article  Google Scholar 

  42. Molyneux, L., Coddington, M.: Aggregation, clickbait and their effect on perceptions of journalistic credibility and quality. J. Pract. 14(4), 429–446 (2020). https://doi.org/10.1080/17512786.2019.1628658

    Article  Google Scholar 

  43. Kosinski, M., Matz, S.C., Gosling, S.D., Popov, V., Stillwell, D.: Facebook as a research tool for the social sciences: opportunities, challenges, ethical considerations, and practical guidelines. Am. Psychol. 70(6), 543–556 (2015). https://doi.org/10.1037/a0039210

    Article  Google Scholar 

  44. Nelson, E.J., Loux, T., Arnold, L.D., Siddiqui, S.T., Schootman, M.: Obtaining contextually relevant geographic data using Facebook recruitment in public health studies. Health Place 55, 37–42 (2019). https://doi.org/10.1016/j.healthplace.2018.11.002

    Article  Google Scholar 

  45. Pew Research Internet Project-2019. Social Networking Fact Sheet. http://www.pewinternet.org/fact-sheets/social-networking-fact-sheet/. Assessed 15 Jan 2021

  46. Gosling, S.D., Vazire, S., Srivastava, S., John, O.P.: Should we trust web-based studies? a comparative analysis of six preconceptions about internet questionnaires. Am. Psychol. 59(2), 93–104 (2004). https://doi.org/10.1037/0003-066X.59.2.93

    Article  Google Scholar 

  47. Iannelli, L., Giglietto, F., Rossi, L., Zurovac, E.: Facebook digital traces for survey research: assessing the efficiency and effectiveness of a Facebook ad–based procedure for recruiting online survey respondents in niche and difficult-to-reach populations. Soc. Sci. Comput. Rev. 38(4), 462–476 (2020). https://doi.org/10.1177/0894439318816638

    Article  Google Scholar 

  48. Thornton, L., Batterham, P.J., Fassnacht, D.B., Kay-Lambkin, F., Calear, A.L., Hunt, S.: Recruiting for health, medical or psychosocial research using Facebook: systematic review. Internet Interv. 4, 72–81 (2016). https://doi.org/10.1016/j.invent.2016.02.001

    Article  Google Scholar 

  49. Ramo, D.E., Prochaska, J.J.: Broad reach and targeted recruitment using Facebook for an online survey of young adult substance use. J. Med. Internet Res. 14(1), e28 (2012). https://doi.org/10.2196/jmir.1878

    Article  Google Scholar 

  50. Gilligan, C., Kypri, K., Bourke, J.: Social networking versus facebook advertising to recruit survey respondents: a quasi-experimental study. JMIR Res. Protocols 3(3), e48 (2014). https://doi.org/10.2196/resprot.3317

    Article  Google Scholar 

  51. Whitaker, C., Stevelink, S., Fear, N.: The use of Facebook in recruiting participants for health research purposes: a systematic review. J. Med. Internet Res. 19(8), e290 (2017). https://doi.org/10.2196/jmir.7071

    Article  Google Scholar 

  52. Youn, S.J., et al.: Using online social media, Facebook, in screening for major depressive disorder among college students. Int. J. Clin. Health Psychol. 13(1), 74–80 (2013). https://doi.org/10.1016/S1697-2600(13)70010-3

    Article  MathSciNet  Google Scholar 

  53. King, D.B., O’Rourke, N., DeLongis, A.: Social media recruitment and online data collection: a beginner’s guide and best practices for accessing low-prevalence and hard-to-reach populations. Can. Psychol. 55(4), 240–249 (2014). https://doi.org/10.1037/a0038087

    Article  Google Scholar 

  54. Arcia, A.: Facebook advertisements for inexpensive participant recruitment among women in early pregnancy. Health Educ. Behav. 41(3), 237–241 (2014). https://doi.org/10.1177/1090198113504414

    Article  Google Scholar 

  55. Kapp, J.M., Peters, C., Oliver, D.P.: Research recruitment using Facebook advertising: big potential, big challenges. J. Cancer Educ. 28(1), 134–137 (2013). https://doi.org/10.1007/s13187-012-0443-z

    Article  Google Scholar 

  56. Zhang, B., Mildenberger, M., Howe, P.D., Marlon, J., Rosenthal, S.A., Leiserowitz, A.: Quota sampling using Facebook advertisements. Polit. Sci. Res. Methods 8(3), 558–564 (2020). https://doi.org/10.1017/psrm.2018.49

    Article  Google Scholar 

  57. Richiardi, L., Pivetta, E., Merletti, F.: Recruiting study participants through Facebook. Epidemiology 23(1), 175 (2012). https://doi.org/10.1097/EDE.0b013e31823b5ee4

  58. Smith, T.M.: Post-stratification. J. Roy. Stat. Soc. Ser. D (The Statistician) 40(3), 315–323 (1991)

    Google Scholar 

  59. Little, R.J.A.: Post-stratification: a modeler’s perspective. J. Am. Stat. Assoc. 88(423), 1001–1012 (1993)

    Article  Google Scholar 

  60. Heeringa, S.G., West, B.T., Berglund, P.A.: Applied Survey Data Analysis, 2nd edn. CRC Press, Boca Raton (2017)

    Google Scholar 

  61. Tillé, Y., Wilhelm, M.: Probability sampling designs: principles for choice of design and balancing. Stat. Sci. 32(2), 176–189 (2017). https://doi.org/10.1214/16-STS606

  62. Livingstone, S.: On the challenges of cross-national comparative media research. Eur. J. Commun. 18(4), 477–500 (2003). https://doi.org/10.1177/0267323103184003

    Article  Google Scholar 

  63. Lewandowsky, S., Ecker, U.K.H., Seifert, C.M., Schwarz, N., Cook, J.: Misinformation and its correction: continued influence and successful debiasing. Psychol. Sci. Public Interest 13(3), 106–131 (2012). https://doi.org/10.1177/1529100612451018

    Article  Google Scholar 

Download references

Acknowledgements

The research was implemented in the framework of the Russian Scientific Fund Grant №19–18-00206 at the National Research University Higher School of Economics (HSE) in 2021.

Author information

Authors and Affiliations

  1. Laboratory for Social and Cognitive Informatics, National Research University Higher School of Economics, Saint-Petersburg, Russia

    Victoria Vziatysheva, Yadviga Sinyavskaya, Alexander Porshnev, Maxim Terpilovskii, Sergey Koltcov & Kirill Bryanov

Authors
  1. Victoria Vziatysheva
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yadviga Sinyavskaya
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander Porshnev
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Maxim Terpilovskii
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sergey Koltcov
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kirill Bryanov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Victoria Vziatysheva .

Editor information

Editors and Affiliations

  1. Department of Computer Science, Towson University, Towson, MD, USA

    Gabriele Meiselwitz

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Vziatysheva, V., Sinyavskaya, Y., Porshnev, A., Terpilovskii, M., Koltcov, S., Bryanov, K. (2021). Testing Users’ Ability to Recognize Fake News in Three Countries. An Experimental Perspective. In: Meiselwitz, G. (eds) Social Computing and Social Media: Experience Design and Social Network Analysis . HCII 2021. Lecture Notes in Computer Science(), vol 12774. Springer, Cham. https://doi.org/10.1007/978-3-030-77626-8_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-77626-8_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-77625-1

  • Online ISBN: 978-3-030-77626-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation