EK-BERT: An Enhanced K-BERT Model for Chinese Sentiment Analysis

Abstract

Pre-trained language models (PLMs), such as BERT, have achieved good results on many natural language processing (NLP) tasks. Recently, some studies have attempted to integrate factual knowledge into PLMs for adapting to various downstream tasks. For sentiment analysis task, sentiment knowledge largely helps determine the sentiment tendencies of texts, such as sentiment words. For Chinese sentiment analysis, historical stories and fables give richer connotations and more complex emotions to words, which makes sentiment knowledge injection more necessary. But clearly, this knowledge has not been fully considered. In this paper, we propose EK-BERT, an Enhanced K-BERT model for Chinese sentiment analysis, which is based on the K-BERT model and utilizes sentiment knowledge graph to achieve better results on sentiment analysis task. In order to construct a high-quality sentiment knowledge graph, we collect a large number of emotional words by combining several existing emotional dictionaries. Moreover, in order to understand texts better, we enhance local attention through syntactic analysis to make EK-BERT pay more attention to syntactically relevant words. EK-BERT is compatible with BERT and existing structural knowledge. Experimental results show that our proposed EK-BERT achieves better performance on Chinese sentiment analysis task.