Abstract
In today’s interconnected world, where individuals can create and receive information freely, the proliferation of fake news has become a significant issue. This type of false information frequently appears in areas such as business or politics, and its widespread dissemination on the internet can disrupt the normal social order and create a biased net- work atmosphere, ultimately leading to the destruction of the normal network environment. The evolution of fake news, from early plain text to complex images and texts, has made its detection more difficult. To address this, we propose an Albert ResNet50 hybrid deep neural net- work model that combines implicit features of both text and images for detecting multimodal fake news. We tested our model on three fake news datasets, and the results showed an accuracy rate of 90.51%, 79.87%, and 92.93%, respectively. Compared to traditional models that only use text data, our multimodal model can better identify fake news.
Similar content being viewed by others
Data availability
Not applicable.
Code availability
Not applicable.
References
Boididou C, Papadopoulos S, Dang-Nguyen DT, Boato G, Kompatsiaris Y. (2015) The certh-unitn participation @ verifying multimedia use 2015. In: MediaEval 2015 Workshop
Christina B, Andreadou K, Papadopoulos S, Dang-Nguyen DT, Kompatsiaris Y (2015) Verifying multimedia use at mediaeval 2015 in mediaeval benchmarking initiative for multimedia evaluation. In: MediaEval Benchmark 2015
Devlin J, Chang MW, Lee K, Toutanova K. (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. https://doi.org/10.48550/arXiv.1810.04805
Dhawan M, Sharma S, Kadam A, Sharma R, Kumaraguru P. (2022) Game-on: Graph attention network based multimodal fusion for fake news detection. https://doi.org/10.48550/arXiv.2202.12478
Guo B, Ding Y, Yao L, Liang Y, Yu Z. (2019) The future of misinformation detection: New perspectives and trends. https://doi.org/10.48550/arXiv.1909.03654
Hanselowski A, Pvs A, Schiller B, Caspelherr F, Gurevych I. (2018) A retrospective analysis of the fake news challenge stance detection task. https://doi.org/10.48550/arXiv.1806.05180
He H, Sun G (2020) Fake news content detection model based on feature aggregation. Computer Application v40. 360(08):25–29. https://doi.org/10.11772/j.issn.1001-9081.2019122114
Jin Z, Cao J, Han, G, Zhang Y, Luo J (2017) Multimodal fusion with recurrent neural networks for rumor detection on microblogs. the 2017 ACM.ACM, 2017. https://doi.org/10.1145/3123266.3123454
Jin Z, Cao J, Zhang Y, Luo J (2016) News verification by exploiting conflicting social viewpoints in microblogs. Proceedings of the AAAI Conference on Artificial Intelligence. 30. https://doi.org/10.1609/aaai.v30i1.10382
Jin Z, Cao J, Zhang Y, Zhou J, Qi T (2017) Novel visual and statistical image features for microblogs news verification. IEEE Trans Multimed 19(3):598–608
Khattar D, Singh J, Gupta M, Varma V (2019) MVAE: Multimodal Variational Autoencoder for Fake News Detection. WWW '19: The World Wide Web Conference, pp 2915–2921. https://doi.org/10.1145/3308558.3313552
Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R (2019) ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Li Q, Zhang Q, Si L (2019) Rumor Detection by Exploiting User Credibility Information, Attention and Multi-task Learning, pp 1173–1179. https://doi.org/10.18653/v1/P19-1113
Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B. (2021) Swin transformer: Hierarchical vision transformer using shifted windows
Mayank M, Sharma S, Sharma R (2021) Deap-faked: Knowledge graph based approach for fake news detection. https://doi.org/10.48550/arXiv.2107.10648
Mccloskey S, Albright M (2018) Detecting gan-generated imagery using color cues. https://doi.org/10.48550/arXiv.1812.08247
Qi P, Cao J, Yang T, Guo J, Li J. (2019) Exploiting multi-domain visual information for fake news detection. IEEE. https://doi.org/10.1109/ICDM.2019.00062
Ruixiang T, Yu-Neng Chuang XH (2023) The science of detecting llm- generated texts. https://doi.org/10.48550/arXiv.2303.07205
Shu K, Sliva A, Wang S, Tang J, Liu H (2017) Fake news detection on social media: a data mining perspective. ACM SIGKDD Explorations Newsletter 19(1). https://doi.org/10.1145/3137597.3137600
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large- scale image recognition. Computer Science. https://doi.org/10.48550/arXiv.1409.1556
Singhal S, Kabra A, Sharma M, Shah RR, Kumaraguru P (2020) SpotFake+: A Multimodal Framework for Fake News Detection via Transfer Learning (Student Abstract). The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20)
Sun S, Liu H, He J, Du X (2013) Detecting Event Rumors on Sina Weibo Automatically. 7808:120–131. https://doi.org/10.1007/978-3-642-37401-2_14
Tang XWX, Huang C. (2020) Annual report on the development of new media in china no.11
Uyangodage L, Ranasinghe T, Hettiarachchi H (2021) Transformers to fight the covid-19 infodemic. https://doi.org/10.48550/arXiv.2104.12201
Wang Y, Ma F, Jin Z, Ye Y, Jha K. (2018) Eann: Event adversarial neural networks for multi-modal fake news detection. In: Acm Sigkdd International Conference
Zaremba W, Sutskever IVO (2014) Recurrent neural network regularization. Eprint Arxiv 2014. https://doi.org/10.48550/arXiv.1409.2329
Zhang C (2019) Algorithmic governance of fake news on social platforms: Logic, limitation and collaborative governance model. Press 000(011):19–2899
Zhou X, Wu J, Zafarani R. (2020) Safe: Similarity-aware multi-modal fake news detection. arXiv. https://doi.org/10.1007/978-3-030-47436-2_27
Zhou X, Zafarani R. (2018) Fake news: A survey of research, detection methods, and opportunities. ArXiv abs/1812.00315
Zhuo TY, Huang CCZXY (2023) Exploring ai ethics of chatgpt: A diagnostic analysis. arXiv Preprint, arXiv:2301.12867
Funding
The paper has been supported by various sources, including the Guangzhou Science and Technology Plan Project (No. 201903010103), the 2021 Guangdong Province Undergraduate College Teaching Quality and Teaching Reform Project Construction Project (Guangdong Higher Education [2021] 29 No.154), the 2021 South China Normal University Quality Engineering Construction Project (Teaching (2021) 72 No. 136), the 16th Batch of General Education Curriculum Construction Projects of South China Normal University (Teaching (2021) 74 No. 10), the Guangzhou Philosophy and Social Science Planning 2022 Annual Project (2022GZYB66), the 2022 South China Normal University Quality Engineering Construction project (Teaching [2022] 41 No.109 & 143), the Science and Technology Plan Project of Guangdong Provincial Department of Communications (NO.
2015–02-064). The project also received support from the 2022”Challenge Cup” Gold Seed Cultivation Project of South China Normal University, as well as two general topics of students’ extracurricular scientific research projects:”Auxiliary Diagnosis and Drug Recommendation of Diabetes based on Medical Knowledge Graph”.
Author information
Authors and Affiliations
Contributions
All authors contributed to the research, experiment and manuscript. Mingyue Jiang,Chang Jing and Shouqiang Liu were responsible for the design and the preparation of the experiment. The expriment and related discussion were performed by Mingyue Jiang, Chang Jing, Liming Chen, Yang Wang and Shouqiang Liu.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare they have no conflict of interest.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Jiang, M., Jing, C., Chen, L. et al. An application study on multimodal fake news detection based on Albert-ResNet50 Model. Multimed Tools Appl 83, 8689–8706 (2024). https://doi.org/10.1007/s11042-023-15741-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-023-15741-y