Aspect-based sentiment analysis via multitask learning for online reviews

Abstract

Aspect based sentiment analysis(ABSA) aims to identify aspect terms in online reviews and predict their corresponding sentiment polarity. Sentiment analysis poses a challenging fine-grained task. Two typical subtasks are involved: Aspect Term Extraction (ATE) and Aspect Polarity Classification (APC). These two subtasks are usually trained discretely, which ignores the connection between ATE and APC. Concretely, we can relate ATE to APC through aspects and train them concurrently. We mainly use the ATE task as an auxiliary task, allowing the APC to focus more on relevant aspects to facilitate aspect polarity classification. In addition, previous studies have shown that utilizing dependency syntax information with a graph neural network (GNN) also contributes to the performance of the APC task. However, most studies directly input sentence dependency relations into graph neural networks without considering the influence of aspects, which do not emphasize the important dependency relationships. To address these issues, we propose a multitask learning model combining APC and ATE tasks that can extract aspect terms as well as classify aspect polarity simultaneously. Moreover, we exploit multihead attention(MHA) to associate dependency sequences with aspect extraction, which not only combines both ATE and APC tasks but also stresses the significant dependency relations, enabling the model to focus more on words closely related to aspects. According to our experiments on three benchmark datasets, we demonstrate that the connection between ATE and APC can be better established by our model, which enhances aspect polarity classification performance significantly. The source code has been released on GitHub https://github.com/winder-source/MTABSA.

Introduction

Aspect based sentiment analysis (ABSA) aims to mine sentiment information toward a given sentence, but is fine-grained. Specifically, its goal is to identify aspect terms in a comment and predict their corresponding sentiment polarity. In the example, “I like the service in the restaurant, but the environment is not very good”, the aspect terms are “service” and “environment”. The output emotional polarity of the two aspects is positive and negative. The sentiments corresponding to these two aspects are quite opposite, so it is not appropriate to conduct a sentiment analysis of the whole sentence but to conduct a more fine-grained analysis. The main research line of ABSA focuses on two subtasks, namely, ATE and APC.

The APC task is usually considered a classification task, or sentiment classification of a given aspect in a sentence. The approach to solving APC tasks has evolved from feature engineering to deep learning-based methods. The most common deep neural network architectures used in APC tasks are convolutional neural networks (CNNs) and recurrent neural networks (RNNs) [1], [2]. Moreover, the application of attention mechanisms in neural networks is becoming increasingly extensive. The attention mechanism [3], [4] is also suitable for ABSA tasks. In recent years, SOTA results have been obtained in numerous NLP tasks with proposed pretraining models. Therefore, many studies are based on the pretraining model, such as the AEN model [5] and BERT-PT [6]. In addition, a sentence contains not only semantic information but also syntactic structure information, such as dependency tree structure. Intuitively, it is helpful to integrate syntactic structure information into the APC task because the syntactic structure can better capture sentiment words related to aspect. Recently, many methods have regarded the dependency tree as an adjacency matrix and utilized GNNs to encode the entire adjacency matrix, such as graph attention networks [7] (GAT) and graph convolutional networks [8], [9] (GCN).

In most of these studies, the ATE task was studied independently. The ATE task is regarded as a NER task aimed at extracting aspects of the sentences as a sequence labeling task [10], [11], [12]. The advancement of deep learning has shown its usefulness in tasks. Recent methods use deep neural networks to assist aspect extraction [13], [14], [15]. Furthermore, there are many models based on BERT to perform sequence labeling tasks due to the success of the model.

Other approaches for sentiment analysis [16], [17] have emerged in recent years, with meta-based self-training sentiment analysis [18] and prompt-based sentiment analysis [19], [20] being proposed to perform sentiment analysis more efficiently, and new trends in neurosymbolic AI for explainable sentiment analysis [21], [22], [23], [24] have also emerged.

In addition, there are also some works that mainly consider multitask learning in ABSA [25], [26], [27] to achieve better performance with the help of interactions between tasks. Based on the inspiration of multitask learning, we also propose a multitask learning model that combines APC and ATE tasks. In our model, the APC task’s performance is further boosted by using the feature of the ATE task. Inspired by the relational graph attention network (RGAT) presented by Wang et al. [28], we also use a series of RGAT processes to encode the reshaped and pruned dependency tree. Although previous studies have shown that graph neural networks contribute to the performance of APC tasks, most previous studies have fed sentence dependencies directly into graph neural networks. Such dependencies do not consider the influence of aspects. Therefore, to address the challenge of dependency relations, we apply MHA to associate dependency sequences with aspect extraction, enabling our model to focus on the dependency sequences that are more closely related to the aspects. To validate the effectiveness of the proposed model, extensive experiments are conducted on three public datasets, and according to the experimental results, our model has obvious improvements and achieves superior performance.

Our main contributions are as follows:

• We propose a multitask learning model that integrates BERT and RGAT models for APC and ATE tasks. The two tasks are conducted simultaneously in a joint training manner.

• We propose to associate dependency sequences with aspect extraction via MHA, which can enhance the connection between aspects and their associated dependency sequences.

• Three public datasets were used to confirm the validity of the model. As seen from the experimental results, the proposed model outperforms recent state-of-the-art models. We further conduct domain-adaptation experiments, achieving appealing results.