Skip to main content

Advertisement

Springer Nature Link
Log in

Fake news detection based on multi-modal domain adaptation

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The rapid development of social media has led to the unguaranteed authenticity of news, and the role of fake news detection in cybersecurity governance has become increasingly prominent. In this paper, we mine information from multiple modalities, such as the text and images of news, and propose a multi-modal fake news detection model based on the multi-stage domain adaptation for the differences existing between source and task domains and between different modalities. The multi-modal feature extraction network of BERT combined with EfficientNet is used to deeply analyze the features of social media data, and the multi-modal domain adaptation network is used to reduce the domain shift of different domains and different modalities of news data and to capture the correlation between events by adversarial ideas. Experimental results on public datasets of Weibo and Twitter show that the model significantly improves the effectiveness of the fake news detection task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The fake news datasets supporting Table 1 [37, 38] are publicly available in the gitee repository, as part of this record: https://gitee.com/w-xiaopei/fake-news-detection-based-on-multi-modal-domain-adaptation-f.git.

References

  1. Xue J, Wang Y, Tian Y, Li Y, Shi L, Wei L (2021) Detecting fake news by exploring the consistency of multimodal data. Inf Process Manage 58(5):102610–102610

    Article  MATH  Google Scholar 

  2. Zhou H, Ma T, Rong H, Qian Y, Tian Y, Al-Nabhan N (2022) MDMN: Multi-task and Domain Adaptation based Multi-modal Network for early rumor detection. Exp Syst Appl 195:116517

    Article  MATH  Google Scholar 

  3. Allcott H, Gentzkow M (2017) Social media and fake news in the 2016 election. J Economic Perspectives 31(2):211–36

    Article  MATH  Google Scholar 

  4. Raj C, Meel P (2021) Convnet frameworks for multi-modal fake news detection. Appl Intell 51(11):8132–8148

    Article  MATH  Google Scholar 

  5. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  6. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778

  7. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708

  8. Zhang H, Qian S, Fang Q, Xu C (2020) Multimodal disentangled domain adaption for social media event rumor detection. IEEE Trans Multimedia 23:4441–4454

    Article  MATH  Google Scholar 

  9. Castillo C, Mendoza M, Poblete B (2011) Information credibility on twitter. In: Proceedings of the 20th International Conference on World Wide Web, pp. 675–684

  10. Feng S, Banerjee R, Choi Y (2012) Syntactic stylometry for deception detection. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 171–175

  11. Elyassami S, Alseiari S, ALZaabi M, Hashem A, Aljahoori N. Fake news detection using ensemble learning and machine learning algorithms. Combating fake news with computational intelligence techniques. 2022:149-62.

  12. Hakak S, Alazab M, Khan S, Gadekallu TR, Maddikunta PKR, Khan WZ (2021) An ensemble machine learning approach through effective feature extraction to classify fake news. Futur Gener Comput Syst 117:47–58

    Article  Google Scholar 

  13. Ma J, Gao W, Mitra P, Kwon S, Jansen BJ, Wong K-F, Cha M (2016) Detecting rumors from microblogs with recurrent neural networks, 3818–3824

  14. Chen T, Li X, Yin H, Zhang J (2018) Call attention to rumors: Deep attention based recurrent neural networks for early rumor detection. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 40–52. Springer

  15. Ali AM, Ghaleb FA, Al-Rimy BAS, Alsolami FJ, Khan AI (2022) Deep ensemble fake news detection model using sequential deep learning technique. Sensors 22(18):6970

    Article  Google Scholar 

  16. Aslam N, Ullah Khan I, Alotaibi FS, Aldaej LA, Aldubaikil AK (2021) Fake detect: A deep learning ensemble model for fake news detection. Complexity 2021:1–8

    Article  Google Scholar 

  17. Kumar A, Singh JP, Singh AK (2022) Covid-19 fake news detection using ensemble-based deep learning model. IT Professional 24(2):32–37

    Article  MATH  Google Scholar 

  18. Jin Z, Cao J, Guo H, Zhang Y, Luo J (2017) Multimodal fusion with recurrent neural networks for rumor detection on microblogs. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 795–816

  19. Wang Y, Ma F, Jin Z, Yuan Y, Xun G, Jha K, Su L, Gao J (2018) Eann: Event adversarial neural networks for multi-modal fake news detection. In: Proceedings of the 24th Acm Sigkdd International Conference on Knowledge Discovery & Data Mining, pp. 849–857

  20. Khattar D, Goud JS, Gupta M, Varma V (2019) Mvae: Multimodal variational autoencoder for fake news detection. In: The World Wide Web Conference

  21. Zhang T, Wang D, Chen H, Zeng Z, Guo W, Miao C, Cui L (2020) Bdann: Bert-based domain adaptation neural network for multi-modal fake news detection. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE

  22. Zhou X, Wu J, Zafarani R (2020) Safe: similarity-aware multi-modal fake news detection (2020). Preprint. arXiv:2003.04981

  23. Chen Y, Li D, Zhang P, Sui J, Lv Q, Tun L, Shang L (2022) Cross-modal ambiguity learning for multimodal fake news detection. In: Proceedings of the ACM Web Conference 2022, pp. 2897–2905

  24. Al Obaid A, Khotanlou H, Mansoorizadeh M, Zabihzadeh D (2022) Multimodal fake-news recognition using ensemble of deep learners. Entropy 24(9):1242

    Article  Google Scholar 

  25. Ganin Y, Ustinova E, Ajakan H, Germain P, Larochelle H, Laviolette F, March M, Lempitsky V (2016) Domain-adversarial training of neural networks. J Machine Learn Res 17(59):1–35

    MathSciNet  MATH  Google Scholar 

  26. Qi F, Yang X, Xu C (2018) A unified framework for multimodal domain adaptation. In: Proceedings of the 26th ACM International Conference on Multimedia, pp. 429–437

  27. Munro J, Damen D (2020) Multi-modal domain adaptation for fine-grained action recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 122–132

  28. Kenton JDM-WC, Toutanova LK (2019) Bert: Pre-training of deep bidirectional transformers for language understanding, 4171–4186

  29. Xu B, Wang N, Chen T, Li M (2015) Empirical evaluation of rectified activations in convolutional network. arXiv e-prints, 1505

  30. Tan M, Le Q (2019) Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114. PMLR

  31. Sifre L, Mallat S (2014) Rigid-motion scattering for texture classification. arXiv preprint arXiv:1403.1687

  32. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141

  33. Ramachandran P, Zoph B, Le QV (2017) Searching for activation functions. arXiv preprint arXiv:1710.05941

  34. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Machine Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  35. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456. PMLR

  36. Munro J, Damen D (2020) Multi-modal Domain Adaptation for Fine-grained Action Recognition. In: Computer Vision and Pattern Recognition (CVPR)

  37. Jin Z, Cao J, Guo H, Zhang Y, Luo J (2017) Multimodal fusion with recurrent neural networks for rumor detection on microblogs. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 795–816

  38. Maigrot C, Claveau V, Kijak E, Sicre R (2016) Mediaeval 2016: A multimodal system for the verifying multimedia use task. In: MediaEval 2016:" Verfiying Multimedia Use" Task

  39. Antol S, Agrawal A, Lu J, Mitchell M, Batra D, Zitnick CL, Parikh D (2015) Vqa: Visual question answering. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2425–2433

  40. Vinyals O, Toshev A, Bengio S, Erhan D (2015) Show and tell: A neural image caption generator. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3156–3164

  41. Zhang H, Fang Q, Qian S, Xu C (2019) Multi-modal knowledge-aware event memory network for social media rumor detection. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 1942–1951

  42. Guo Y (2023) A mutual attention based multimodal fusion for fake news detection on social network. Appl Intell 53(12):15311–20

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the research of Liaoning Provincial Social Science Association's 2025 economic and social development research project 2025 lslqnwzzkt-041.

Author information

Authors and Affiliations

  1. Computer Science and Engineering, Dalian Minzu University, No. 18, Liaohe West Road, Dalian, 116600, Liaoning, China

    Xiaopei Wang, Jiana Meng, Di Zhao, Xuan Meng & Hewen Sun

  2. Information Engineering College, Binzhou Polytechnic, No. 919, Huanghe 12 Road, Bincheng District, Binzhou, 256600, Shandong, China

    Xiaopei Wang

Authors
  1. Xiaopei Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jiana Meng
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Di Zhao
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Xuan Meng
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Hewen Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Di Zhao.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Meng, J., Zhao, D. et al. Fake news detection based on multi-modal domain adaptation. Neural Comput & Applic 37, 5781–5793 (2025). https://doi.org/10.1007/s00521-024-10896-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-024-10896-7

Keywords