Abstract
Due to the rapid spread of rumors on social media and their negative impact on society, real-time rumor detection is of utmost importance. Although some rumor detection methods have applied the structure of temporal or graphic information, they do not consider multiple structures to obtain better representation. Besides, since the authors maybe only post texts in real-world social scenarios, image modalities become inaccessible in multi-modal rumor detection. To solve the above issues, we propose a Multi-Modal rumor detection model on Modality Alignment and multi-Perspective Structures (M3APS). The model uses the image generation method to fill the inaccessible image modalities in the multi-modal heterogeneous node pair and fuses the node pair to obtain multi-modal features. Then, the “debunkers” which are extracted from the perspective of temporal structure and graphic structure query the events described in the source tweet, respectively. Experimental results on three popular datasets show that our model M3APS is superior to the state-of-the-art baselines.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Albalawi, R.M., Jamal, A.T., Khadidos, A.O., Alhothali, A.M.: Multimodal arabic rumors detection. IEEE Access 11, 9716–9730 (2023)
Bian, T., et al.: Rumor detection on social media with bi-directional graph convolutional networks. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence, pp. 549–556 (2020)
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 4171–4186 (2019)
Han, X., Huang, Z., Lu, M., Li, D., Qiu, J.: Rumor verification on social media with stance-aware recursive tree. In: Proceedings of the 14th Knowledge Science, Engineering and Management International Conference, pp. 149–161 (2021)
Huang, Z., Lv, Z., Han, X., Li, B., Lu, M., Li, D.: Social bot-aware graph neural network for early rumor detection. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 6680–6690 (2022)
Jin, Z., Cao, J., Guo, H., Zhang, Y., Luo, J.: Multimodal fusion with recurrent neural networks for rumor detection on microblogs. In: Proceedings of the 2017 ACM on Multimedia Conference, pp. 795–816 (2017)
Khoo, L.M.S., Chieu, H.L., Qian, Z., Jiang, J.: Interpretable rumor detection in microblogs by attending to user interactions. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence, pp. 8783–8790 (2020)
Kumar, S., Carley, K.M.: Tree LSTMs with convolution units to predict stance and rumor veracity in social media conversations. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, pp. 5047–5058 (2019)
Li, Q., Zhang, Q., Si, L.: Rumor detection by exploiting user credibility information, attention and multi-task learning. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, pp. 1173–1179 (2019)
Ma, J., et al.: Detecting rumors from microblogs with recurrent neural networks. In: Proceedings of the 25th International Joint Conference on Artificial Intelligence, pp. 3818–3824 (2016)
Ma, J., Gao, W., Wong, K.: Rumor detection on twitter with tree-structured recursive neural networks. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 1980–1989 (2018)
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., Chen, M.: Hierarchical text conditional image generation with CLIP latents. CoRR abs/2204.06125 (2022)
Sujana, Y., Li, J., Kao, H.: Rumor detection on twitter using multi-loss hierarchical BiLSTM with an attenuation factor. In: Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pp. 18–26 (2020)
Sun, M., Zhang, X., Ma, J., Liu, Y.: Inconsistency matters: a knowledge-guided dual-inconsistency network for multi-modal rumor detection. In: Findings of the Association for Computational Linguistics, pp. 1412–1423 (2021)
Sun, M., Zhang, X., Zheng, J., Ma, G.: DDGCN: dual dynamic graph convolutional networks for rumor detection on social media. In: Proceedings of the 36th AAAI Conference on Artificial Intelligence, pp. 4611–4619 (2022)
Wang, B., Wei, H., Li, R., Liu, S., Wang, K.: Rumor detection model fused with static spatiotemporal information. J. Intell. Fuzzy Syst. 44(2), 2847–2862 (2023)
Wang, Y., et al.: EANN: event adversarial neural networks for multi-modal fake news detection. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 849–857 (2018)
Wu, B., et al.: Visual transformers: Token-based image representation and processing for computer vision. CoRR abs/2006.03677 (2020)
Xia, R., Xuan, K., Yu, J.: A state-independent and time-evolving network for early rumor detection in social media. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pp. 9042–9051 (2020)
Xu, K., et al.: Show, attend and tell: neural image caption generation with visual attention. In: Proceedings of the 32nd International Conference on Machine Learning, pp. 2048–2057 (2015)
Acknowledgements
The authors would like to thank the three anonymous reviewers for their comments on this paper. This research was supported by the National Natural Science Foundation of China (Nos. 62006167, 62276177 and 61836007), and Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Li, B., Qian, Z., Li, P., Zhu, Q. (2023). Multi-modal Rumor Detection on Modality Alignment and Multi-perspective Structures. In: Huang, DS., Premaratne, P., Jin, B., Qu, B., Jo, KH., Hussain, A. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2023. Lecture Notes in Computer Science(), vol 14089. Springer, Singapore. https://doi.org/10.1007/978-981-99-4752-2_39
Download citation
DOI: https://doi.org/10.1007/978-981-99-4752-2_39
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-4751-5
Online ISBN: 978-981-99-4752-2
eBook Packages: Computer ScienceComputer Science (R0)