Abstract
Social media have an enormous impact on modern life but are prone to the dissemination of false information. In several domains, such as crisis management or political communication, it is of utmost importance to detect false and to promote credible information. Although educational measures might help individuals to detect false information, the sheer volume of social big data, which sometimes need to be analysed under time-critical constraints, calls for automated and (near) real-time assessment methods. Hence, this paper reviews existing approaches before designing and evaluating three deep learning models (MLP, RNN, BERT) for real-time credibility assessment using the example of Twitter posts. While our BERT implementation achieved best results with an accuracy of up to 87.07% and an F1 score of 0.8764 when using metadata, text, and user features, MLP and RNN showed lower classification quality but better performance for real-time application. Furthermore, the paper contributes with a novel dataset for credibility assessment.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Allcott, H., Gentzkow, M.: Social media and fake news in the 2016 election. J. Econ. Perspect. 31(2), 211–236 (2017)
Buntain, C., Golbeck, J.: Automatically identifying fake news in popular twitter threads. In: IEEE Proceedings of (SmartCloud), pp. 208–215 (2017)
Castillo, C., Mendoza, M., Poblete, B.: Information credibility on twitter. In: Proceedings of WWW, p. 675 (2011)
Castillo, C., Mendoza, M., Poblete, B.: Predicting information credibility in time-sensitive social media. Internet Res. 23(5), 560–588 (2013)
Conroy, N.K., Rubin, V.L., Chen, Y.: Automatic deception detection: methods for finding fake news. Proc. ASIS&T 52(1), 1–4 (2015)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT, pp. 4171–4186 (2019)
Duan, X., Naghizade, E., Spina, D., Zhang, X.: RMIT at PAN-CLEF 2020: proling fake news spreaders on twitter. In: CLEF 2020 (2020)
Flanagin, A.J., Metzger, M.J.: Digital media and youth: unparalled opportunity and unprecedented responsibility. In: Flanagin, A.J., Metzger, M.J. (eds.) Digital Media, Youth, and Credibility, pp. 5–28 (2008)
Floria, S.A., Leon, F., Logofătu, D.: A credibility-based analysis of information diffusion in social networks. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds.) Proceedings of ICANN, pp. 828–838 (2018)
Fogg, B.J., Tseng, H.: The elements of computer credibility. In: Proceedings of CHI, pp. 80–87 (1999)
Gupta, A., Kumaraguru, P., Castillo, C., Meier, P.: TweetCred: real-time credibility assessment of content on twitter. In: Aiello, L.M., McFarland, D. (eds.) Social Informatics, vol. 8851, pp. 228–243 (2014)
Hassan, D.: A text mining approach for evaluating event credibility on twitter. In: Proceedings of WETICE, pp. 171–174 (2018)
Helmstetter, S., Paulheim, H.: Weakly supervised learning for fake news detection on twitter. In: Proceedings of ASONAM, pp. 274–277. IEEE (2018)
Hutto, C., Gilbert, E.: VADER: a parsimonious rule-based model for sentiment analysis of social media text. In: Proceedings of ICWSM (2015)
Iftene, A., Gifu, D., Miron, A.R., Dudu, M.S.: A real-time system for credibility on twitter. In: Proceedings of LREC, pp. 6166–6173 (2020)
Kaufhold, M.A., Bayer, M., Hartung, D., Reuter, C.: Paper Appendix (2021). https://github.com/mkx89-sci/KaufholdBayerHartungReuter2021_ICANN
Kaufhold, M.A., Reuter, C.: Cultural violence and peace in social media. In: Reuter, C. (ed.) Information Technology for Peace and Security - IT-Applications and Infrastructures in Conflicts, Crises, War, and Peace, pp. 361–381 (2019)
Kaufhold, M.A.: Information Refinement Technologies for Crisis Informatics: User Expectations and Design Principles for Social Media and Mobile Apps (2021)
Kaufhold, M.A., Bayer, M., Reuter, C.: Rapid relevance classification of social media posts in disasters and emergencies: a system and evaluation featuring active, incremental and online learning. IP&M 57(1), 102132 (2020)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2017)
Lazer, D.M.J., et al.: The science of fake news. Science 359(6380), 1094–1096 (2018)
Liu, X., Nourbakhsh, A., Li, Q., Fang, R., Shah, S.: Real-time rumor debunking on twitter. In: Proceedings of CIKM, pp. 1867–1870 (2015)
Liu, Y., Wu, Y.F.B.: Early detection of fake news on social media through propagation path classification with recurrent and convolutional networks. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)
Ma, J., et al.: Detecting rumors from microblogs with recurrent neural networks. In: IJCAI International Joint Conference on Artificial Intelligence, pp. 3818–3824 (2016)
Mitra, T., Gilbert, E.: CREDBANK: a large-scale social media corpus with associated credibility annotations. In: Proceedings of ICWSM (2015)
Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of EMNLP, pp. 1532–1543 (2014)
Pierri, F., Ceri, S.: False news on social media: a data-driven Survey. ACM SIGMOD Record 48(2), 18–27 (2019)
Qazvinian, V., Rosengren, E., Radev, D.R., Mei, Q.: Rumor has it: identifying misinformation in microblogs. In: Proceedings of EMNLP, pp. 1589–1599 (2011)
Rangel, F., Giachanou, A., Ghanem, B., Rosso, P.: Overview of the 8th Author Proling Task at PAN 2020: Proling Fake News Spreaders on Twitter (2020)
Ratkiewicz, J., et al.: Truthy: mapping the spread of astroturf in microblog streams. In: Proceedings of WWW, p. 249 (2011)
Ruchansky, N., Seo, S., Liu, Y.: CSI: a hybrid deep model for fake news detection. In: Proceedings of CIKM, pp. 797–806 (2017)
Sanh, V., Debut, L., Chaumond, J., Wolf, T.: DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter (2020)
Shu, K., Mahudeswaran, D., Wang, S., Lee, D., Liu, H.: FakeNewsNet: a data repository with news content, social context and dynamic information for studying fake news on social media. Big Data 8(3) (2018)
Shu, K., Sliva, A., Wang, S., Tang, J., Liu, H.: Fake news detection on social media: a data mining perspective. ACM SIGKDD Explor. Newsl. 19(1), 22–36 (2017)
Tandoc, E.C., Lim, Z.W., Ling, R.: Defining “fake news’’: a typology of scholarly definitions. Digit. Journal. 6(2), 137–153 (2018)
Tian, L., Zhang, X., Peng, M.: FakeFinder: twitter fake news detection on mobile. In: Companion Proceedings of the Web Conference, vol. 2020, pp. 79–80 (2020)
Tian, L., Zhang, X., Wang, Y., Liu, H.: Early detection of rumours on twitter via stance transfer learning. In: Jose, J.M., et al. (eds.) Advances in Information Retrieval, vol. 12035, pp. 575–588 (2020)
Viviani, M., Pasi, G.: Credibility in social media: opinions, news, and health information-a survey: credibility in social media. Wiley Interdiscip. Rev.: Data Min. Knowl. Discov. 7(5), e1209 (2017)
Wu, L., Rao, Y., Yu, H., Wang, Y., Nazir, A.: False information detection on social media via a hybrid deep model. In: Staab, S., Koltsova, O., Ignatov, D.I. (eds.) SocInfo 2018. LNCS, vol. 11186, pp. 323–333. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01159-8_31
Xia, X., Yang, X., Wu, C., Li, S., Bao, L.: Information credibility on twitter in emergency situation. In: Chau, M., Wang, G.A., Yue, W.T., Chen, H. (eds.) PAISI 2012. LNCS, vol. 7299, pp. 45–59. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-30428-6_4
Zubiaga, A., Liakata, M., Procter, R., Wong Sak Hoi, G., Tolmie, P.: Analysing how people orient to and spread rumours in social media by looking at conversational threads. PLoS ONE 11(3), e0150989 (2016)
Acknowledgements
This work has been co-funded by the German Federal Ministry of Education and Research (BMBF) in the project CYWARN (No. 13N15407) and by the BMBF and the Hessen State Ministry for Higher Education, Research and Arts (HMKW) within the SecUrban mission of the National Research Center for Applied Cybersecurity ATHENE.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Kaufhold, MA., Bayer, M., Hartung, D., Reuter, C. (2021). Design and Evaluation of Deep Learning Models for Real-Time Credibility Assessment in Twitter. In: Farkaš, I., Masulli, P., Otte, S., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2021. ICANN 2021. Lecture Notes in Computer Science(), vol 12895. Springer, Cham. https://doi.org/10.1007/978-3-030-86383-8_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-86383-8_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86382-1
Online ISBN: 978-3-030-86383-8
eBook Packages: Computer ScienceComputer Science (R0)