Loading [MathJax]/extensions/MathMenu.js
PVCG: Prompt-Based Vision-Aware Classification and Generation for Multi-Modal Rumor Detection | IEEE Conference Publication | IEEE Xplore

PVCG: Prompt-Based Vision-Aware Classification and Generation for Multi-Modal Rumor Detection


Abstract:

Multi-modal Rumor Detection (MRD) has emerged as a crucial research hotpot due to the continuous rise in the spread of multi-modal information on the Internet. Existing s...Show More

Abstract:

Multi-modal Rumor Detection (MRD) has emerged as a crucial research hotpot due to the continuous rise in the spread of multi-modal information on the Internet. Existing studies frequently employ traditional single-classifier models, which cannot accurately classify challenging positive samples. Moreover, the interaction of multiple modalities typically involves an additional fusion module, which results in a trade-off between the granularity of modality interaction and the complexity of the fusion modules. To address these issues, we present a model called Prompt-based Visionaware Classification and Generation (PVCG), where we use a generator module for the MRD. Notably, the encoder independently handles modality fusion more finely by including image as a soft prompt in text embeddings. Our evaluations on Fakeddit and Pheme corpus demonstrate that our PVCG outperforms the state-of-the-art baselines, showcasing its superior performance on the MRD task.
Date of Conference: 14-19 April 2024
Date Added to IEEE Xplore: 18 March 2024
ISBN Information:

ISSN Information:

Conference Location: Seoul, Korea, Republic of

References

References is not available for this document.