Abstract
With the rapid growth of the number of social media users, a variety of unverified information inevitably spreads on the social platform, which leads to the diffusion of rumors. Although some methods are explored on multi-modal data, they seldom take into account the hidden knowledge behind the text and image, and ignore the widely dispersed structure on multi-modal data in the rumor detection field. To solve the above issues, we propose a novel Multi-Modal Rumor detection model via Knowledge-aware Heterogeneous Graph Convolutional Networks, i.e., M\(^3\)KHG, which can model a post as a propagation graph, capture the interactive semantic information of image and text at the cross-modal level, and highlight suspicious signals according to the correlation between text-image knowledge in a unified framework. Finally, the “knowledgeable” feature generated by the propagation graph is assigned to debunk rumors. Experimental results on three popular datasets show that our model M\(^3\)KHG is superior to the state-of-the-art baselines.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
The image knowledge consists of the entities with brief introduction and is extracted by an object recognition tool (https://ai.baidu.com/tech/imagerecognition/general).
References
Bian, T., et al.: Rumor detection on social media with bi-directional graph convolutional networks. In: AAAI, pp. 549–556. AAAI Press (2020)
Chen, W., Zhang, Y., Yeo, C.K., Lau, C.T., Lee, B.: Unsupervised rumor detection based on users’ behaviors using neural networks. Pattern Recogn. Lett. 105, 226–233 (2018)
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT (1), pp. 4171–4186. Association for Computational Linguistics (2019)
Jin, Z., Cao, J., Guo, H., Zhang, Y., Luo, J.: Multimodal fusion with recurrent neural networks for rumor detection on microblogs. In: ACM Multimedia, pp. 795–816. ACM (2017)
Khoo, L.M.S., Chieu, H.L., Qian, Z., Jiang, J.: Interpretable rumor detection in microblogs by attending to user interactions. In: AAAI, pp. 8783–8790. AAAI Press (2020)
Kochkina, E., Liakata, M., Zubiaga, A.: All-in-one: multi-task learning for rumour verification. In: COLING, pp. 3402–3413. Association for Computational Linguistics (2018)
Li, Q., Zhang, Q., Si, L.: Rumor detection by exploiting user credibility information, attention and multi-task learning. In: ACL (1), pp. 1173–1179. Association for Computational Linguistics (2019)
Liu, X., Nourbakhsh, A., Li, Q., Fang, R., Shah, S.: Real-time rumor debunking on Twitter. In: CIKM, pp. 1867–1870. ACM (2015)
Lu, Y., Li, C.: GCAN: graph-aware co-attention networks for explainable fake news detection on social media. In: ACL, pp. 505–514. Association for Computational Linguistics (2020)
Ma, J., et al.: Detecting rumors from microblogs with recurrent neural networks. In: IJCAI, pp. 3818–3824. IJCAI/AAAI Press (2016)
Ma, J., Gao, W., Wong, K.: Detect rumor and stance jointly by neural multi-task learning. In: WWW (Companion Volume), pp. 585–593. ACM (2018)
Ma, J., Gao, W., Wong, K.: Rumor detection on Twitter with tree-structured recursive neural networks. In: ACL (1), pp. 1980–1989. Association for Computational Linguistics (2018)
Sujana, Y., Li, J., Kao, H.: Rumor detection on Twitter using multiloss hierarchical BiLSTM with an attenuation factor. In: AACL/IJCNLP, pp. 18–26. Association for Computational Linguistics (2020)
Sun, M., Zhang, X., Ma, J., Liu, Y.: Inconsistency matters: a knowledge-guided dual-inconsistency network for multi-modal rumor detection. In: EMNLP (Findings), pp. 1412–1423. Association for Computational Linguistics (2021)
Wang, Y., et al.: EANN: event adversarial neural networks for multi-modal fake news detection. In: KDD, pp. 849–857. ACM (2018)
Wei, L., Hu, D., Zhou, W., Yue, Z., Hu, S.: Towards propagation uncertainty: edge-enhanced Bayesian graph convolutional networks for rumor detection. In: ACL/IJCNLP (1), pp. 3845–3854. Association for Computational Linguistics (2021)
Wu, B., et al.: Visual transformers: token-based image representation and processing for computer vision. CoRR abs/2006.03677 (2020)
Wu, L., Rao, Y., Zhao, Y., Liang, H., Nazir, A.: DTCA: decision tree-based co-attention networks for explainable claim verification. In: ACL, pp. 1024–1035. Association for Computational Linguistics (2020)
Zhang, H., Fang, Q., Qian, S., Xu, C.: Multi-modal knowledge-aware event memory network for social media rumor detection. In: ACM Multimedia, pp. 1942–1951. ACM (2019)
Acknowledgments
The authors would like to thank the three anonymous reviewers for their comments on this paper. This research was supported by the National Natural Science Foundation of China (Nos. 61836007, 62276177 and 62006167), and Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Li, B., Qian, Z., Li, P., Zhu, Q. (2023). Multi-modal Rumor Detection via Knowledge-Aware Heterogeneous Graph Convolutional Networks. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Lecture Notes in Computer Science, vol 13624. Springer, Cham. https://doi.org/10.1007/978-3-031-30108-7_21
Download citation
DOI: https://doi.org/10.1007/978-3-031-30108-7_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-30107-0
Online ISBN: 978-3-031-30108-7
eBook Packages: Computer ScienceComputer Science (R0)