A novel unsupervised ensemble framework using concept-based linguistic methods and machine learning for twitter sentiment analysis

Highlights

• A Novel ensemble/cooperative framework based on concept-based and clustering is proposed to perform Twitter sentiment Analysis.

• It employs majority voting, tie breaker criteria, and linguistic rules in concept-based module.

• Comparative analysis between clustering and classification is presented when integrated with concept based methods.

• It presents the performance of feature representation methods (Boolean and TF-IDF).

• Experimental results on Twitter Datasets revealed better performance of proposed framework.

Abstract

Concept-based sentiment analysis (CBSA) methods have gained prominence in natural language processing in recent years. These methods consider the underlying semantic meanings of text to perform different tasks such as Twitter sentiment analysis (assigning positive, negative, or neutral sentiment to Tweets). CBSA is superior to traditional statistical methods for accurately discovering sentiment labels. Due to a limited knowledge base, these methods are unable to identify the sentiment polarity of all kinds of text. Therefore, supervised learning techniques are mostly ensembled with CBSA methods to classify the whole text. These techniques require labeled data. It is a tedious and time-consuming task due to the manually labeling of large datasets (Such as Twitter datasets). Therefore, an unsupervised learning mechanism can be a better alternative to solve this problem. In this paper, a novel unsupervised learning framework based on Concept-based and hierarchical clustering is proposed for Twitter sentiment analysis. Popular hierarchical clustering methods including single linkage, complete linkage, and average linkage algorithms are ensembled serially. Two different feature representation methods including Boolean and Term frequency-inverse document frequency (TF-IDF) are investigated. We have also experimented with Well-known classifiers (Naive Bayes, Neural Network) for a fair comparison. Accuracy measure (proportion of correct predictions) is used to evaluate the performance of understudied techniques. It is empirically shown that the performance of unsupervised learning techniques is comparable with supervised learning techniques.

Introduction

The explosion of user-generated content (UGC) led to the opportunity to automatically discover associated sentiments. The term “sentiment” represents a positive/negative opinion, emotion, feeling, or thought expressed by a sentiment holder (user). Generally, sentiment analysis aims to automatically extract these sentiments from the text. Sentiment analysis aims to examine textual features to automatically seek a sentiment at the word, sentence, or document level. Sentiment analysis is popular nowadays in diverse fields including public-health monitoring [1], election trends [2], prediction of terrorism activities [3], and social network analysis [4].

Social networks provide online platforms to emulate social relationships between people. Twitter is one of the famous microblogging platforms that allows users to post real-time short messages (limited to 280 characters) called Tweets relevant to personal and social issues. On Twitter, more than 1 billion new Tweets have been posted every three days [5]. Twitter data has widely been explored by researchers to address diverse research issues e.g., sentiment analysis [4], [6], [7]. Sentiment analysis of Twitter data is a challenging problem in human computing. However, due to the restriction of 280 characters limit in a tweet, the informal language used by people poses a significant challenge to uncover the underlying sentiment of Tweets [6]. Therefore, it is crucial to use automatic intelligent techniques to perform Twitter sentiment analysis. Twitter sentiment analysis is important for many reasons such as identifying highly valued customers’ opinions for different products and services. Also, a broader range of diseases such as pandemics, election trends including potential candidates, and negative campaigning can be highlighted through Twitter sentiment analysis. Similarly, it can be useful to improve education policies by monitoring students’ performance.

Bag of words (BOWs) is a popular method in natural language processing for feature extraction in different domains, e.g. sentiment analysis [8], disease surveillance system [9], etc. However, the literature identified the limited capabilities of BOWs for extracting underlying semantics associated with text and dictates the use of Bag of Concepts (BOCs) [10]. The BOCs representation is a major drift from the BOWs approach. It intends to perform Concept-based sentiment analysis (CBSA) by utilizing semantic meanings of natural language opinions/text [10]. Concept-based sentiment analysis methods are unsupervised in the sense that pre-labeled data is not mandatory. SenticNet [11], [12] and Linguistic rules [13] are developed as a part of these methods. Relevant studies have revealed that these approaches cannot assign a sentiment polarity to all kinds of text due to the lack of richness of its knowledge base [10], [13]. Therefore, researchers ensembled other techniques along with CBSA methods. Among ML techniques, different classifiers have been integrated with unsupervised Concept-based sentiment methods for the sentiment prediction [10], [14].

The challenge faced in using classifiers is the requirement of pre-labeled data for the training process. It is a cumbersome task to label manually a large amount of unlabeled data. The labeling process may also be prolonged due to the time constraints of domain experts. Whereas, pre-labeled data is not a mandatory requirement for unsupervised (clustering) approaches. These methods accept unlabeled data and generate clusters of similar data instances.

In this paper, we have proposed a novel unsupervised ensemble framework based on Concept-based sentiment analysis methods and hierarchical clustering to perform Twitter sentiment analysis as shown in Fig. 1. In the proposed framework, both methods work in an unsupervised fashion for sentiment analysis. Hierarchical clustering has not been integrated earlier with concept-based methods. In this framework, initially, the concept-based analysis module, classifies Tweets using a) majority voting mechanism b) tie-breakers based on intensity ranking c) Linguistic Patterns. To the best of our knowledge, concept-based sentiment analysis has not been investigated earlier in this manner. Those Tweets, which are not classified by this module are then delegated to three popular agglomerative hierarchical clustering algorithms including single-linkage (SL), complete-linkage (CL), and average-linkage (AL). These methods have already been employed in some recent relevant research studies [15], [16], [17]. We have also performed a comparative analysis with earlier investigated classifiers i.e. Naive Bayes and Neural Network. An empirical study is performed on four English language-based Twitter datasets. Accuracy measure has been used to evaluate the performance of the proposed unsupervised framework in terms of polarity prediction of Tweets. Unigrams are considered for feature extraction and boolean and TF-IDF methods are used to represent features for delegated Tweets.

The main contributions of this research work are as follows:

• It proposes an unsupervised ensemble/cooperative framework built on concept-based and agglomerative hierarchical clustering for Twitter sentiment analysis.

• It presents a performance-based comparative analysis of clustering and classification when integrated with concept-based methods.

• It shows performance analysis of individual understudied techniques.

• It employs majority voting, tie-breakers criteria, and Linguistic rules in the concept-based sentiment analysis module.

• It also presents the performance of feature representation methods (Boolean and TF-IDF).