Do Photos Help Express Our Feelings: Incorporating Multimodal Features into Microblog Sentiment Analysis

Abstract

Because of the interest in discovering public moods and opinions in both industrial and academic researches, sentiment analysis of microblogs has become one of the major concerns in Web data mining and natural language processing studies. Although a large part of the microblog posts contain non-textual components such as images, emoticons and location information, most existing works rely on textual information only to generate sentiment analysis results. Different from these efforts, we focus on the influence of other sources of information in sentiment analysis, especially the images from social media, which are commonly posted by users along with texts. Having noticed that images reinforce sentiment expression along with text in microblog environment, we propose a unified model to extract the features of text and image together. Learning based approaches are then adopted to finish sentiment analysis tasks such as subjectivity classification. Experimental results based on practical microblog data show that features extracted from images help gain better sentiment analysis results.