Skip to main content
Springer Nature Link
Log in

CCGN: consistency contrastive-learning graph network for multi-modal fake news detection

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Fake news can mislead the public and cause great harm to society. As social media contains more and more multimodal information, multimodal fake news detection has received widespread attention. However, existing methods face difficulties in dealing with the consistency of text and images. Considering the consistent relationship between text and images, this paper proposes a multimodal fake news detection model based on consistent contrastive learning graph. Specifically, the network first uses vision GNN to treat the image as a grid structure to suppress irrelevant information. Then, consistency contrast learning is used to calculate the semantic distance between the extracted text features and image features to improve the consistency between the text and the image. Finally, multimodal cross-attention is used to fuse text and image features interactively. The experimental results on Weibo and Twitter datasets demonstrate the effectiveness of the proposed model in the fake news detection task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

All data included are freely available through the following repository: https://github.com/wangzhuang1911/Weibo-dataset.

References

  1. Bondielli, A., Marcelloni, F.: A survey on fake news and rumour detection techniques. Inf. Sci. 497, 38–55 (2019). https://doi.org/10.1016/j.ins.2019.05.035

    Article  MATH  Google Scholar 

  2. Singhal, S., Shah, RR., Chakraborty, T., Kumaraguru, P., Satoh, S.: Spotfake: a multi-modal framework for fake news detection. In: 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM), pp. 39–47 (2019) https://doi.org/10.1109/BigMM.2019.00-44

  3. Wang, Y., Ma, F., Jin, Z., Yuan, Y., Xun, G., Jha, K., Su, L., Gao, J.: EANN: event adversarial neural networks for multi-modal fake news detection. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Association for Computing Machinery, New York, NY, USA, KDD ’18, pp. 849–857 (2018) https://doi.org/10.1145/3219819.3219903

  4. Zhang, H., Fang, Q., Qian, S., Xu, C.: Multi-modal knowledge-aware event memory network for social media rumor detection. In: Proceedings of the 27th ACM International Conference on Multimedia, Association for Computing Machinery, New York, NY, USA, MM ’19, pp. 1942–1951 (2019) https://doi.org/10.1145/3343031.3350850,

  5. Jin, Z., Cao, J., Guo, H., Zhang, Y., Luo, J.: Multimodal fusion with recurrent neural networks for rumor detection on microblogs. In: Proceedings of the 25th ACM international conference on Multimedia, pp 795–816 (2017)

  6. Zhou, X., Wu, J., Zafarani, R.: SAFE: similarity-aware multi-modal fake news detection. CoRR https://arxiv.org/abs/2003.04981 (2020)

  7. Bahad, P., Saxena, P., Kamal, R.: Fake news detection using bi-directional LSTM-recurrent neural network. Procedia Comput. Sci. 165, 74–82 (2019). https://doi.org/10.1016/j.procs.2020.01.072

    Article  Google Scholar 

  8. Nasir, J.A., Khan, O.S., Varlamis, I.: Fake news detection: a hybrid CNN-RNN based deep learning approach. Int. J. Inf. Manag. Data Insights 1(1), 100007 (2021). https://doi.org/10.1016/j.jjimei.2020.100007

    Article  MATH  Google Scholar 

  9. Ni, S., Li, J., Kao, H.Y.: Mvan: multi-view attention networks for fake news detection on social media. IEEE Access 9, 106907–106917 (2021). https://doi.org/10.1109/ACCESS.2021.3100245

    Article  MATH  Google Scholar 

  10. Trueman, T.E., Ashok Kumar, J., Narayanasamy, P., Vidya, J.: Attention-based C-BILSTM for fake news detection. Appl. Soft Comput. 110, 107600 (2021). https://doi.org/10.1016/j.asoc.2021.107600

    Article  MATH  Google Scholar 

  11. Raza, S., Ding, C.: Fake news detection based on news content and social contexts: a transformer-based approach. Int. J. Data Sci. Anal. 13(4), 335–362 (2022)

    Article  MATH  Google Scholar 

  12. Gundapu, S., Mamidi, R.: Transformer based automatic covid-19 fake news detection system. arXiv preprint arXiv:2101.00180 (2021)

  13. Guo, Z., Zhang, Q., Ding, F., Zhu, X., Yu, K.: A novel fake news detection model for context of mixed languages through multiscale transformer. IEEE Trans. Comput. Soc. Syst. 11, 5079–5089 (2023). https://doi.org/10.1109/TCSS.2023.3298480

    Article  MATH  Google Scholar 

  14. Li, T., Sun, Y., Hsu, Sl., Li, Y., Wong, R. C. W.: Fake news detection with heterogeneous transformer. arXiv preprint arXiv:2205.03100 (2022)

  15. Mahmud, FB., Rayhan, MMS., Shuvo, MH., Sadia, I., Morol, M.: A comparative analysis of graph neural networks and commonly used machine learning algorithms on fake news detection. In: 2022 7th International Conference on Data Science and Machine Learning Applications (CDMA), pp. 97–102 (2022) https://doi.org/10.1109/CDMA54072.2022.00021

  16. Xu, W., Wu, J., Liu, Q., Wu, S., Wang, L.: Evidence-aware fake news detection with graph neural networks. In: Proceedings of the ACM Web Conference 2022, Association for Computing Machinery, New York, NY, USA, WWW ’22, pp. 2501–2510 (2022) https://doi.org/10.1145/3485447.3512122

  17. Hu, L., Yang, T., Zhang, L., Zhong, W., Tang, D., Shi, C., Duan, N., Zhou, M.: Compare to the knowledge: Graph neural fake news detection with external knowledge. In: Zong C, Xia F, Li W, Navigli R (eds) Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Association for Computational Linguistics, Online, pp 754–763 (2021) https://doi.org/10.18653/v1/2021.acl-long.62,

  18. Bazmi, P., Asadpour, M., Shakery, A., Maazallahi, A.: Entity-centric multi-domain transformer for improving generalization in fake news detection. Inf. Process. Manag. 61(5), 103807 (2024)

    Article  Google Scholar 

  19. Khattar, D., Goud, JS., Gupta, M., Varma, V.: Mvae: Multimodal variational autoencoder for fake news detection. In: The World Wide Web Conference, Association for Computing Machinery, New York, NY, USA, WWW ’19, pp. 2915–2921 (2019) https://doi.org/10.1145/3308558.3313552

  20. Xue, J., Wang, Y., Tian, Y., Li, Y., Shi, L., Wei, L.: Detecting fake news by exploring the consistency of multimodal data. Inf. Process. Manag. 58(5), 102610 (2021). https://doi.org/10.1016/j.ipm.2021.102610

    Article  MATH  Google Scholar 

  21. Hu, L., Chen, Z., Zhao, Z., Yin, J., Nie, L.: Causal inference for leveraging image-text matching bias in multi-modal fake news detection. IEEE Trans. Knowl. Data Eng. 35(11), 11141–11152 (2022)

    Article  Google Scholar 

  22. Hu, L., Zhao, Z., Qi, W., Song, X., Nie, L.: Multimodal matching-aware co-attention networks with mutual knowledge distillation for fake news detection. Inf. Sci. 664, 120310 (2024). https://doi.org/10.1016/j.ins.2024.120310

    Article  Google Scholar 

  23. Chen, Y., Li, D., Zhang, P., Sui, J., Lv, Q., Tun, L., Shang, L.: Cross-modal ambiguity learning for multimodal fake news detection. In: Proceedings of the ACM Web Conference 2022, Association for Computing Machinery, New York, NY, USA, WWW ’22, p 2897–2905 (2022) https://doi.org/10.1145/3485447.3511968,

  24. Li, J., Bin, Y., Zou, J., Wei, J., Wang, G., Yang, Y.: Cross-modal consistency learning with fine-grained fusion network for multimodal fake news detection. In: Proceedings of the 5th ACM International Conference on Multimedia in Asia, Association for Computing Machinery, New York, NY, USA, MMAsia ’23 (2024) https://doi.org/10.1145/3595916.3626397,

  25. Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Philip, S.Y.: A comprehensive survey on graph neural networks. IEEE Trans. Neural Netw. Learn. Syst. 32(1), 4–24 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  26. Kipf, TN., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)

  27. Han, K., Wang, Y., Guo, J., Tang, Y., Wu, E.: Vision GNN: an image is worth graph of nodes. Adv. Neural Inf. Process. Syst. 35, 8291–8303 (2022)

    Google Scholar 

  28. Wu, Z., Xiong, Y., Yu, SX., Lin, D.: Unsupervised feature learning via non-parametric instance discrimination. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3733–3742 (2018)

  29. Jin, Z., Cao, J., Guo, H., Zhang, Y., Luo, J.: Multimodal fusion with recurrent neural networks for rumor detection on microblogs. In: Proceedings of the 25th ACM International Conference on Multimedia, Association for Computing Machinery, New York, NY, USA, MM ’17, pp. 795–816 (2017) https://doi.org/10.1145/3123266.3123454,

  30. Boididou, C., Andreadou, K., Papadopoulos, S., Dang Nguyen, DT., Boato, G., Riegler, M., Kompatsiaris, Y., et al.: Verifying multimedia use at mediaeval 2015. In: MediaEval 2015, vol 1436, CEUR-WS (2015)

  31. Devlin, J.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

  32. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 10012–10022 (2021)

  33. Kumari, R., Ekbal, A.: AMFB: attention based multimodal factorized bilinear pooling for multimodal fake news detection. Expert Syst. Appl. 184, 115412 (2021). https://doi.org/10.1016/j.eswa.2021.115412

    Article  Google Scholar 

  34. Singh, P., Srivastava, R., Rana, K., Kumar, V.: Semi-FND: stacked ensemble based multimodal inferencing framework for faster fake news detection. Expert Syst. Appl. 215, 119302 (2023). https://doi.org/10.1016/j.eswa.2022.119302

    Article  MATH  Google Scholar 

  35. Qian, S., Wang, J., Hu, J., Fang, Q., Xu, C.: Hierarchical multi-modal contextual attention network for fake news detection. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Association for Computing Machinery, New York, NY, USA, SIGIR ’21, pp. 153–162 (2021) https://doi.org/10.1145/3404835.3462871

  36. Wu, Y., Zhan, P., Zhang, Y., Wang, L., Xu, Z.: Multimodal fusion with co-attention networks for fake news detection. In: Zong C, Xia F, Li W, Navigli R (eds) Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, Association for Computational Linguistics, Online, pp. 2560–2569 (2021) https://doi.org/10.18653/v1/2021.findings-acl.226

  37. Bai, Y., Cao, M., Gao, D., Cao, Z., Chen, C., Fan, Z., Nie, L., Zhang, M.: Rasa: relation and sensitivity aware representation learning for text-based person search. In: Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI ’23 (2023) https://doi.org/10.24963/ijcai.2023/62,

  38. Hu, L., Chen, Z., Zhao, Z., Yin, J., Nie, L.: Causal inference for leveraging image-text matching bias in multi-modal fake news detection. IEEE Trans. Knowl. Data Eng. 35(11), 11141–11152 (2023). https://doi.org/10.1109/TKDE.2022.3231338

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Engineering, Ibaraki University, Hitachi, Ibaraki, Japan

    ShaoDong Cui, Kaibo Duan, Wen Ma & Hiroyuki Shinnou

Authors
  1. ShaoDong Cui
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Kaibo Duan
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Wen Ma
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Hiroyuki Shinnou
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

ShaoDong Cui and Kaibo Duan contributed equally to the conceptualization and methodology design of the study. ShaoDong Cui implemented the core algorithms, performed experiments, and analyzed the results. Kaibo Duan assisted with experimental validation and data analysis. Wen Ma contributed to the model development and provided critical revisions. Hiroyuki Shinnou supervised the research, contributed to the methodology, and provided overall guidance. All authors contributed to the manuscript writing and approved the final version.

Corresponding author

Correspondence to ShaoDong Cui.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethics approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by Bing-kun Bao.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cui, S., Duan, K., Ma, W. et al. CCGN: consistency contrastive-learning graph network for multi-modal fake news detection. Multimedia Systems 31, 119 (2025). https://doi.org/10.1007/s00530-025-01715-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00530-025-01715-7

Keywords