Elsevier

Neurocomputing

Volume 409, 7 October 2020, Pages 329-340
Neurocomputing

Joint multi-level attentional model for emotion detection and emotion-cause pair extraction

https://doi.org/10.1016/j.neucom.2020.03.105Get rights and content

Abstract

Emotion detection (ED) and emotion-cause pair extraction (ECPE) have drawn extensive research interests due to their wide applications in real-world scenarios. However, existing work fails to capture the implicit connection between two tasks. This limits the performances of these tasks. To address this issue, we propose a novel joint framework to take full advantage of the clause-level and word-level information about ED and ECPE tasks. Specifically, we explore a multi-level attentional module to model the relationship between two clauses in an emotion-cause pair. Results on two benchmark datasets show that our proposed model achieves the current best performance, outperforming the previous methods and strong neural baselines by a large margin. Our code is available at https://github.com/tomsonsgs/LVE-joint-MANN-master.

Introduction

Emotion cause extraction (ECE) aims at identifying the corresponding clauses that induce a certain emotion expression in current context, and become a popular research topic in recent years [1], [2]. This task is first treated as a word-level sequence labeling problem [3]. Considering the limitation of word-level labeling, which is not appropriate for the document-level emotion analysis, Gui et al. (2016) re-designed the ECE task for cause clause detection, and modeled the task as a clause-level classification problem [4]. Furthermore, Xia et al. (2019) thought that there exists an implicit connection between the cause clauses and their corresponding emotion clauses in the document-level representation [5]. They expanded the original task into a new one called emotion-cause pair extraction (ECPE), which aims to predict emotion-cause pairs, and meanwhile introduced a new dataset for the task.

Emotion detection (ED) task [6] aims to predict the clause-level emotion category. As shown in Fig. 1, we can find that ED is closely related to ECPE in two aspects: (1) All emotion clauses in ECPE must carry certain kinds of emotions, which means that these clauses can not be non-emotion clauses in ED; (2) Cause clause is also related to the given emotion, and different semantic fragments can result in different emotions. For example, ‘became better’ usually indicates the positive emotions such as ‘happiness’. The information of the specific emotion, emotion clause and cause clause can affect each other, so we propose a joint task (called ED+ECPE) to improve the performance of both ECPE and ED tasks.

As shown in Fig. 1, we can see that emotion clause and cause clause are implicitly connected in a fine-grained manner, and they both contribute to the formation of emotions. For example, ‘cried excitedly’ reflects the emotion ‘happiness’, and ‘became better’ actually induces the action ‘cried excitedly’. To get a better emotion-cause pair representation and take advantage of the emotion formation process, a coherent method is required to model the inner relationship between emotion clause and cause clause with the guidance of the emotion information. To this end, we can use self attention to describe the connection between the specific emotion and emotion clause, and mutual inner attention to describe the connection between emotion clause and cause clause. Furthermore, these two attentional forms can be integrated coherently by a multi-level attentional module.

In this paper, we devise a hierarchical structure enhanced by a biaffine module to address the joint task of ED+ECPE. This structure consists of a contextual encoder and multiple clause encoders, which are implemented by long short-term memory networks (LSTMs). The outputs of the contextual encoder are further fed into a biaffine module to extract emotion-cause pairs. To capture the fine-grained emotion-sensitive relationship between emotion clause and cause clause, we introduce a multi-level attentional module, which requires the information of self attention and the mutual inner attention, for the hierarchical structure. Specifically, self attention is generated in clause encoders for ED and reflects the word-level importance to formulate the emotion in the emotion clause. Mutual inner attention is gathered from the alignment matrix between two clauses in an emotion-cause pair, and its value represents the connection strength. Then a multi-level attentional module utilizes these attention values to generate pair representations. Experimental results show our proposed architecture can bring consistent performance gains in comparison with several existing methods, achieving the state-of-the-art performance on two benchmark datasets.

Section snippets

Related work

In recent years, emotion cause extraction (ECE) has received extensive research attention in NLP community. Existing work can mainly be divided into three categories: word-level emotion cause extraction, clause-level emotion cause extraction and emotion-cause pair extraction.

Word-level emotion cause extraction: In early work, emotion cause extraction is defined as extracting the word-level causes that lead to the given emotions in the text [3]. Traditional methods such as machine learning and

Preliminaries

In this section, we first introduce the basic procedure and original hierarchical structure for handling ECE task, then we give a brief illumination of our ED+ECPE task.

Model

The overall structure of the proposed model is illustrated in Fig. 2. Specifically, we enhance the encoder with BERT and propose a joint model for ED+ECPE task. With the guidance of self attention mechanism, a multi-level attentional module is designed for modeling the word-level interaction between emotion clauses and cause clauses, to obtain a fused pair representation. Note that all possible emotion-cause pairs are evaluated by the score function in the biaffine module.

Datasets

We conduct our experiments on two benchmark datasets: the ECE corpus and ECPE corpus. The ECE corpus is the mostly used corpus for emotion cause extraction [4] and the ECPE corpus is a newly released dataset for emotion-cause pair extraction [5]. The two datasets are split by the same proportion of 8:1:1. For the ECE, there are 21,050 annotated documents, including 16,840 documents for training, 2110 for development, and 2100 for testing. Correspondingly, there are 24,354 clauses for training,

Conclusion

We investigated a joint structure for the emotion detection and emotion-cause pair extraction tasks by constructing a multi-task learning problem. We introduce a multi-level attentional module, which requires the information of self attention and the mutual inner attention, to capture the relationship between the emotion clause and cause clause. To utilize the hierarchical structure information, latent variable was introduced to model the emotion distribution over the clauses in a document.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

We wish to draw the attention of the Editor to the following facts which may be considered as potential conflicts of interest and to significant financial contributions to this work. We confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for

CRediT authorship contribution statement

Hao Tang: . Donghong Ji: Writing - review & editing, Investigation. Qiji Zhou: .

Acknowledgments

We thank the reviewers for their valuable comments. This work is supported by the National Natural Science Foundation of China (Grant No.61702121, 61772378).

Hao Tang is currently pursuing Ph.D. at Wuhan University, China. His current research interests are machine learning and natural language process.

References (37)

  • K. Gao et al.

    Emotion cause detection for chinese micro-blogs based on ECOCC model

  • K. Gao et al.

    A rule-based approach to emotion cause detection for chinese micro-blogs

    Expert Syst. Appl.

    (2015)
  • L. Gui et al.

    Emotion cause detection with linguistic construction in chinese weibo text

  • J.D. Lafferty et al.

    Conditional random fields: Probabilistic models for segmenting and labeling sequence data

  • A. Neviarouskaya et al.

    Extracting causes of emotions from text

    Proceedings of the Sixth International Joint Conference on Natural Language Processing, IJCNLP 2013, Nagoya, Japan

    (2013)
  • D. Ghazi et al.

    Detecting emotion stimuli in emotion-bearing sentences

  • S. Song et al.

    Detecting concept-level emotion cause in microblogging

  • S. Yada et al.

    A bootstrap method for automatic rule acquisition on emotion cause extraction

  • Cited by (49)

    View all citing articles on Scopus

    Hao Tang is currently pursuing Ph.D. at Wuhan University, China. His current research interests are machine learning and natural language process.

    DongHong Ji is Professor of School of Cyber Science and Engineering at Wuhan University. His research interests are machine learning, logics and reasoning, natural language processing and their applications in text analysis. He is coinvestigator of several other externally funded projects.

    QiJi Zhou is currently pursuing Ph.D. at Wuhan University, China. His current research interests are semantic parsing and natural language process.

    View full text