Multi-granularity Feature Attention Fusion Network for Image-Text Sentiment Analysis

Abstract

Multi-modal sentiment analysis of images and texts in social media has surpassed traditional text-based analysis and attracted more and more attention from researchers. Existing studies on multi-modal sentiment analysis of texts and images focus on learning each modal feature independently, which ignores the correlation between images and texts. In the field of social media, such correlation is often multi-granularity, that is, image areas are often associated with text (words, phrases, sentences) with multiple granularity. In this paper, a multi-granularity feature attention fusion network is proposed to model the correlation between image content and multi-granularity text content for multi-modal sentiment analysis. Specifically, the model proposed in this paper includes feature learning layer, interactive information fusion layer and classification layer. Image features and text features of multi-granularity can be learned in feature learning layer. In the interactive information fusion layer, multi-granularity text features and image features are interacted and fused, and the last classification layer uses the features learned last time to complete classification. The proposed model is validated on two public multimodal data sets of graphs and texts, and the experimental results show that the model is effective.