S${}^3$ map: Semisupervised aspect-based sentiment analysis with masked aspect prediction

Abstract

Aspect-based sentiment analysis (ABSA) refers to a fine-grained task of detecting the sentiment polarities of sentences at the aspect level. To resolve this task, ABSA training samples must be annotated with aspect words and the corresponding sentiment polarities. However, collecting such fine-grained training samples is expensive and time-consuming. Therefore the available ABSA training samples are often scarce. To break the data scarcity challenge of ABSA, we investigate semi-supervised aspect-based sentiment analysis (SemiABSA), which trains ABSA models using a limited amount of expensive labeled sentences and more unlabeled-yet-cheaper sentences. We propose a novel SemiABSA framework, namely semi-supervised aspect-based sentiment analysis with masked aspect prediction (S${}^3$ map), built on the self-training paradigm. We form pseudo-aspect words and pseudo-sentiment polarities for unlabeled sentences and improve model training. Specifically, a BERT-encoder-based masked aspect prediction (MAP) task achieves the pseudo-aspect words generation. Based on S${}^3$ map, we thoroughly investigate the potential of SemiABSA from various perspectives. The empirical results show that S${}^3$ map can consistently improve performance by leveraging unlabeled sentences, even those from different domains.

Introduction

Sentiment analysis (SA) is a fundamental research topic in the information retrieval, and natural language processing communities [1], [2], [3]. Generally, SA aims to automatically detect the sentiment polarity, {e.g.,Positive, Neutral, Negative} from sentences such as those in reviews on restaurants, movies, and products. For example, given a restaurant review,“ Nice restaurant whose service needs to be improved”, SA aims to accurately detect its actual sentiment polarity Positive. Naturally, SA is potentially in demand in many real-world applications.

Unfortunately, the traditional SA task only concentrates on the sentiment polarity of the full text. We expect to analyze fine-grained aspect-based sentiment in many real scenarios to explore more valuable information. Retaking the example above, we may be more concerned with the sentiment target “service” whose polarity is negative rather than the polarity of the full text. In response to this demand, more attention has been recently paid to the emerging topic of Aspect-Based Sentiment Analysis (ABSA), whose aim is to automatically detect the sentiment polarity of certain aspects [4], [5], [6], [7], [8], [9], [10], [11].

Generally speaking, the first step is to collect fine-grained training datasets to resolve ABSA with machine learning techniques. Several human annotators should tag aspect words and their sentiment polarities for training sentences, as illustrated in Table 1. Collecting such training datasets is costly and much more expensive than traditional SA training data. Therefore the available ABSA training datasets are scarce and often contain very few training sentences, e.g., the volumes of the prevalent ABSA datasets Restaurant and Laptop from SemiEval 14 [12] are only 2282 and 3608, respectively. Such scarce ABSA training sentences contain limited supervised signals, limiting the performance upper bound of ABSA models.

To meet this demand, we take inspiration from the spirit of semi-Supervised learning (SSL) and accordingly attempt to train ABSA models by simultaneously leveraging a limited amount of expensive labeled sentences and more unlabeled-yet-cheaper sentences, yielding a topic of semi-supervised aspect-based sentiment analysis (SemiABSA). Formally, we are given a training dataset, which consists of a subset of $N_l$ labeled triplets ${\Omega }_l={\left\{({\mathbf{s}}_i,{\mathbf{a}}_i,{\mathbf{y}}_i)\right\}}_{i=1}^{N_l}$ and a subset of $N_u$ unlabeled sentences ${\Omega }_u={\left\{{\mathbf{s}}_i\right\}}_{i=N_l+1}^{N_l+N_u}$. Specifically, ${\mathbf{s}}_i=\{s_{i1},s_{i2},\dots ,s_{i{N}_i}\}$, ${\mathbf{a}}_i\in {\{0,1\}}^{N_i}$, and ${\mathbf{y}}_i\in {\{0,1\}}^M$ denote the raw sentence, aspect word indicator, and one-hot sentiment polarity indicator, respectively, where $N_i$ represents its number of word tokens and $M$ is the number of sentiment polarities. We consider the inductive learning paradigm, where the aim is to train a predictive model from ${\Omega }_l\cup {\Omega }_u$ and apply it to predict any unseen sentence-aspect tuple $(\mathbf{s},\mathbf{a})$. To our knowledge, very few studies have addressed this topic.

In this paper, we propose a novel self-training SemiABSA framework, namely Semi-Supervised aspect-based Sentiment analysis with Masked Aspect Prediction (S³map). To fully use unlabeled sentences, our basic idea is to form pseudo-aspect-specific sentence embeddings and pseudo-sentiment polarities and train the sentiment classifier with them in a self-training manner. Specifically, S${}^3$ map consists of 4 key modules: BERT-Encoder, Aspect-Discriminator, Aspect-Sentence-Encoder(AS-Encoder), and Sentiment-Predicter. First, the BERT-Encoder can be treated as the basic feature encoder for both sentences and tokens. Second, the Aspect-Discriminator is a BERT-Encoder-based masked aspect prediction task trained over labeled sentences, and it is used to identify the aspect words for unlabeled sentences. Third, with the pseudo-aspect in hand, we utilize the AS-Encoder to update the pseudo-aspect embedding and sentence embedding. Various structures such as GCN or attention networks can instantiate AS-Encoder. Then, we combine pseudo-aspect embedding and sentence embedding to form the pseudo-aspect-specific sentence embeddings. Finally, S${}^3$ map is jointly trained with labeled and pseudo-labeled sentences.

Empirically, we thoroughly investigate the potential of SemiABSA from various perspectives based on S${}^3$ map. First, we employ two prevalent ABSA collections of reviews on restaurants and laptops and four collections of unlabeled sentences from multiple domains, including reviews, daily social media posts, and encyclopedias. We generate synthetic SemiABSA datasets using pairwise combinations of labeled and unlabeled datasets. Accordingly, a total of 8 synthetic SemiABSA datasets are generated. We evaluate S${}^3$ map on these datasets. The empirical results demonstrate that S${}^3$ map can consistently improve performance by leveraging unlabeled sentences in various scenarios, even when labeled and unlabeled sentences are from different domains, and unlabeled sentences, i.e., encyclopedias, tend to be without any sentiment polarities. In addition, S${}^3$ map significantly outperforms the existing SemiABSA methods.

In summary, the major contributions of this paper are outlined below:

• We investigate the problem of SemiABSA and propose a novel framework named S³map.

• We propose a novel BERT-based MAP task to infer the aspect words of sentences.

• We conduct extensive experiments to indicate the effectiveness of S³map.