Abstract
This paper compares quantitatively the spread of Ukraine-related disinformation and its corresponding debunks, first by considering re-tweets, replies, and favourites, which demonstrate that despite platform efforts Ukraine-related disinformation is still spreading wider than its debunks. Next, bidirectional post-hoc analysis is carried out using Granger causality tests, impulse response analysis and forecast error variance decomposition, which demonstrate that the spread of debunks has a positive impact on reducing Ukraine-related disinformation eventually, albeit not instantly. Lastly, the paper investigates the dominant themes in Ukraine-related disinformation and their spatiotemporal distribution. With respect to debunks, we also establish that around 18% of fact-checks are debunking claims which have already been fact-checked in another language. The latter finding highlights an opportunity for better collaboration between fact-checkers, so they can benefit from and amplify each other’s debunks through translation, citation, and early publication online.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
Where debunked claims were in languages other than English, these were translated automatically with Google Translate first, prior to filtering with the keywords listed here: https://gist.github.com/greenwoodma/430d9443920a589b6802070f2ca54134.
- 8.
- 9.
The dataset used for analysis received ethical approval from the University of Sheffield Ethics Board. This paper only discusses analysis and results in aggregate data, without providing examples or information about individual users.
- 10.
- 11.
https://developer.twitter.com/en/docs/twitter-api/data-dictionary/object-model/user. Where needed, Geopy Python library (Ref. https://pypi.org/project/geopy/) is used to extract the country name from the information provided by the API.
- 12.
- 13.
The multilingual model available at https://huggingface.co/sentence-transformers/paraphrase-multilingual-mpnet-base-v2, since it performs best according to the leaderboard (Ref. https://www.sbert.net/docs/pretrained_models.html).
- 14.
The Statsmodel Python library is used to perform the Granger causality test. Ref. https://www.statsmodels.org/.
- 15.
Cholesky decomposition is used for orthogonalisation.
- 16.
We use the BERTTopic [11] Python library for clustering and MPNet [25] as the transformer model. Ref. https://huggingface.co/sentence-transformers/all-mpnet-base-v2.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
References
Aguerri, J., Santisteban, M., Miró-Llinares, F.: The fight against disinformation and its consequences: measuring the impact of “Russia state-affiliated media” on Twitter. SocArXiv (2022)
Allcott, H., Gentzkow, M.: Social media and fake news in the 2016 election. J. Econom. Perspect. 31(2), 211–36 (2017)
Barrera, O., Guriev, S., Henry, E., Zhuravskaya, E.: Facts, alternative facts, and fact checking in times of post-truth politics. J. Public Econ. 182, 104123 (2020)
Burel, G., Farrell, T., Alani, H.: Demographics and topics impact on the co-spread of COVID-19 misinformation and fact-checks on Twitter. Inform. Process. Manage. 58(6), 102732 (2021)
Burel, G., Farrell, T., Mensio, M., Khare, P., Alani, H.: Co-spread of misinformation and fact-checking content during the COVID-19 pandemic. In: International Conference on Social Informatics, pp. 28–42, Springer (2020). https://doi.org/10.1007/978-3-030-60975-7_3
Chen, Q., Zhang, Y., Evans, R., Min, C.: Why do citizens share COVID-19 fact-checks posted by Chinese government social media accounts? The elaboration likelihood model. Int. J. Environ. Res. Public Health 18(19), 10058 (2021)
Erlich, A., Garner, C.: Is pro-kremlin disinformation effective? Evidence from Ukraine. Int. J. Press/Polit. 19401612211045221 (2021)
Gerber, T.P., Zavisca, J.: Does Russian propaganda work? Wash. Q. 39(2), 79–98 (2016)
Granger, C.W.: Investigating causal relations by econometric models and cross-spectral methods. Econometrica: J. Econom. Soc. 424–438 (1969)
Grinberg, N., Joseph, K., Friedland, L., Swire-Thompson, B., Lazer, D.: Fake news on twitter during the 2016 us presidential election. Science 363(6425), 374–378 (2019)
Grootendorst, M.: BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:2203.05794 (2022)
Hiemstra, C., Jones, J.D.: Testing for linear and nonlinear granger causality in the stock price-volume relation. J. Financ. 49(5), 1639–1664 (1994)
Jiang, M., Gao, Q., Zhuang, J.: Reciprocal spreading and debunking processes of online misinformation: a new rumor spreading-debunking model with a case study. Physica A 565, 125572 (2021)
Lange-Ionatamishvili, E., Svetoka, S., Geers, K.: Strategic communications and social media in the Russia Ukraine conflict. Cyber war in perspective: Russian aggression against Ukraine, pp. 103–111 (2015)
Mejias, U.A., Vokuev, N.E.: Disinformation and the media: the case of Russia and Ukraine. Media Culture Soc. 39(7), 1027–1042 (2017)
Miller, C., Inskip, C., Marsh, O., Arcostanzo, F., Weir, D.: Message-based Community Detection on Twitter (2022). https://glavcom.ua/pub/pdf/49/4935/message-based-community-detection-on-twitter.pdf
Nyhan, B., Reifler, J.: Estimating fact-checking’s effects. American Press Institute, Arlington, VA (2015)
Park, C.Y., Mendelsohn, J., Field, A., Tsvetkov, Y.: VoynaSlov: a data set of Russian social media activity during the 2022 Ukraine-Russia War. arXiv preprint arXiv:2205.12382 (2022)
Park, S., Park, J.Y., Chin, H., Kang, J.h., Cha, M.: An experimental study to understand user experience and perception bias occurred by fact-checking messages. In: Proceedings of the Web Conference 2021, pp. 2769–2780 (2021)
Recuero, R., Soares, F.B., Vinhas, O., Volcan, T., Hüttner, L.R.G., Silva, V.: Bolsonaro and the far right: how disinformation about COVID-19 circulates on Facebook in Brazil. Int. J. Commun. 16, 24 (2022)
Shao, C., Ciampaglia, G.L., Varol, O., Yang, K.C., Flammini, A., Menczer, F.: The spread of low-credibility content by social bots. Nat. Commun. 9(1), 1–9 (2018)
Sims, C.A.: Macroeconomics and reality. Econometrica: J. Econom. Soc. 1–48 (1980)
Singh, I., Bontcheva, K., Scarton, C.: The false COVID-19 narratives that keep being debunked: a spatiotemporal analysis. arXiv preprint arXiv:2107.12303 (2021)
Siwakoti, S., Yadav, K., Bariletto, N., Zanotti, L., Erdogdu, U., Shapiro, J.N.: How COVID drove the evolution of fact-checking. Harvard Kennedy School Misinformation Review (2021)
Song, K., Tan, X., Qin, T., Lu, J., Liu, T.Y.: MPNet: masked and permuted pre-training for language understanding. Adv. Neural Inf. Process. Syst. 16857–16867 (2020)
Swire, B., Berinsky, A.J., Lewandowsky, S., Ecker, U.K.: Processing political misinformation: comprehending the Trump phenomenon. Royal Soc. Open Sci. 4(3), 160802 (2017)
Tsonis, A.A., Deyle, E.R., Ye, H., Sugihara, G.: Convergent cross mapping: theory and an example. Adv. Nonlinear Geosci. 587–600 (2018)
Vosoughi, S., Roy, D., Aral, S.: The spread of true and false news online. Science 359(6380), 1146–1151 (2018)
Yablokov, I.: Russian disinformation finds fertile ground in the West. Nature Hum. Behav. 1–2 (2022)
Zhang, Y., et al.: Investigation of the determinants for misinformation correction effectiveness on social media during COVID-19 pandemic. Inform. Process. Manage. 59(3), 102935 (2022)
Acknowledgements
This research has been partially supported by a European Union – Horizon 2020 Program, grant no. 825297 (WeVerify), the European Union – Horizon 2020 Program under the scheme “INFRAIA-01-2018-2019 – Integrating Activities for Advanced Communities” and Grant Agreement n.871042 (“SoBigData++: European Integrated Infrastructure for Social Mining and Big Data Analytics” (http://www.sobigdata.eu)).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
A Appendix
A Appendix
1.1 A.1 Data Collection
As described in Sect. 3, we collect Ukraine-related debunks from EUvsDsinfo and ClaimReview. In order to collect the disinformation links, 1) the debunks indexed in ClaimReview schema has the itemReviewedFootnote 23 object which includes disinformation links that are being debunked by fact-checking organisation and debunked claim statement is present in claimReviewed object; 2) the debunks on EUvsDsinfo explicitly mention disinformation links on their website. Figure 8 shows the screenshot of one of the EUvsDsinfo debunks. The section enclosed in the red box contains disinformation links and the blue box represents the debunked claim statement.
Screenshot of one of the EUvsDsinfo debunks. Section enclosed in the 
box contains disinformation links and the 
box represents the debunked claim statement. (Color figure online)
1.2 A.2 Heatmap
Figure 9 illustrates the heatmap of cluster similarity. The results show that except clusters one and two, most of the clusters are distinct in terms of the topics they cover. This indicates reasonable separation between the clusters found in Sect. 6.
Heatmap for topic cluster similarity. The description of clusters can be found in Sect. 6.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Singh, I., Bontcheva, K., Song, X., Scarton, C. (2022). Comparative Analysis of Engagement, Themes, and Causality of Ukraine-Related Debunks and Disinformation. In: Hopfgartner, F., Jaidka, K., Mayr, P., Jose, J., Breitsohl, J. (eds) Social Informatics. SocInfo 2022. Lecture Notes in Computer Science, vol 13618. Springer, Cham. https://doi.org/10.1007/978-3-031-19097-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-19097-1_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-19096-4
Online ISBN: 978-3-031-19097-1
eBook Packages: Computer ScienceComputer Science (R0)