Interactive double states emotion cell model for textual dialogue emotion prediction

Abstract

Daily dialogues are full of emotions that control the trends of dialogues and influence the attitudes of interlocutors toward each other, and understanding the human emotions in dialogues is of great significance in emotional comfort, human–computer interaction and intelligent question-answering. This paper defines a new task called emotion prediction in textual dialogue. Different from the text emotion recognition task, which derives the current emotional state of interlocutor from the utterance, emotion prediction aims at predicting the future emotional state of interlocutor before the interlocutor utters something. Moreover, this paper summarizes and explains three notable characteristics of emotional propagation in text dialogue: context dependence, persistence and contagiousness. By considering these characteristics, a fully data-driven interactive double states emotion cell model (IDS-ECM) is proposed. The model has two layers. The first layer automatically extracts the emotional information of historical dialogue and is used to describe the contextual dependence of the textual dialogue emotion. The second layer models the change process of interlocutors’ emotional states during the dialogue and depicts the persistence and contagiousness of emotions. Experimental results on two manually annotated datasets show that the proposed model is superior to the baseline in the macro-averaged F1 evaluation metric and that the proposed model can simulate the emotional changes in the process of dialogue so as to predict the emotions with high accuracy. The experimental results also reveal the communication differences between different emotional categories in dialogue, which is of guiding significance for future research.

Introduction

Emotion plays an important role in dialogue systems by controlling the trends of dialogues, guiding the decision-making of interlocutors, and influencing our attitudes towards each other. It was shown in [1] that the average number of turns in an emotional dialogue (8.7) is longer than that in a non-emotional dialogue (7.0), suggesting that emotions contribute to the continuity of dialogue. Therefore, understanding the human emotions in dialogues is of great significance. Dialogue emotion analysis can also guide human–human interactions [2], such as daily chats and communications between users and customer service, doctors and patients, etc. For example, in the communications between psychologists and patients with depression, understanding the emotions of depressive patients is the key to giving patients emotional comfort and treating them. Dialogue emotion analysis is also the essential factor for a human–computer interaction system to pass the Turing test [2], [3], since emotions can personify robots and enhance the user experience in human–machine dialogue systems, intelligent question answering systems, personal assistant robots, etc.

Most of the existing researches [4], [5], [6], [7], [8], [9], [10] on dialogue emotion analysis recognize emotions through tonal features and facial expressions in video, audio and multi-modal datasets. In many practical applications, however, we express emotions mainly through text, such as social media, instant messaging, on-line customer service, e-mail, etc., where audio and video dialogue datasets are usually difficult to obtain. In addition, the Turing test [3] is designed to communicate only via text and sensory expression (e.g., voice intonations and facial expressions) does not play a role [2]. Thus, due to the lack of audio and visual features, it is important to fully mine the effective information in textual dialogues for emotion analysis.

Recently, great progress has been made in text sentiment and emotion analysis [11], [12], [13], [14], [15], [16]. However, it is not appropriate to apply text emotion analysis to textual dialogue emotion analysis, because the emotion analysis of textual dialogue has its own distinctive characteristics compared with general text emotion analysis:

• Context dependence. The emotion analysis of textual dialogue relies on contexts, whereas the general text emotional analysis mainly focuses on a single sentence. As the example in Table 1 shows, for those marked utterances such as ‘I am going to invite other guys. See you that day.’, the emotion in the sentence is not obvious, but it may carry emotion in a specific context.

• Persistence. The emotional states in the textual dialogue are continuous and may be consistently affected by their own moods, whereas the emotional state in general text emotion analysis is independent. We call it emotional persistence. As the example in Table 2 shows, $A$ and $B$ maintain their emotions in the process of dialogue without being influenced by each other. In the example in Table 3, although the emotions of $A$ are not shown in the middle of the dialogue, they are kept, as verified by the subsequent emotions. These examples show that the emotional state of the interlocutor is consistent.

• Contagiousness. Textual dialogue emotion analysis assesses the emotions of two or more interlocutors in a dialogue, whereas general text emotion analysis usually involves only one interlocutor. In the textual dialogue, the emotional state of a interlocutor is interactive, influential, and contagious to others. We call it the emotional contagiousness. As seen from the example in Table 4, $A$ is not emotional at the beginning of the conversation, but was subsequently infected by the ‘happiness’ of $B$.

In addition, traditional text dialogue emotion recognition task derives the current $m$th emotional state $E_m$ of an interlocutor from the utterance $U_m$ by modelling the probability $P\left(E_m\right|U_m)$. For some practical applications, it is also important to predict the trend of the interlocutor’s emotional changes. Therefore, in this paper, we define a new task of emotion analysis in text dialogue: dialogue emotion prediction. Different from dialogue emotion recognition, the dialogue emotion prediction task models the probability $P\left(E_{m+1}\right|U_{\le m})$, predicting the future emotional state $E_{m+1}$ of an interlocutor at time $m+1$ through the existing dialogue history $U_{\le m}$ without knowing any future utterance information $U_{m+1}$ of the interlocutor. The example in Fig. 1 shows the difference between emotion recognition and emotion prediction. The dialogue prediction may help to understand the emotional trends of users or patients in real time, so as to better guide customer services and doctors to make optimal decisions in advance.

By taking into account the characteristics of textual dialogue emotion analysis, this paper proposes a fully data-driven interactive double states emotion cell model (IDS-ECM) for emotion prediction. The IDS-ECM is composed of two layers. The first layer, called the emotion feature extraction layer, depicts the contextual dependence of the textual dialogue emotion and is used to automatically extract the emotional information from historical dialogues. The second layer, called the emotion propagation layer, retains the historical emotional states and enhances the persistence of the emotion. The IDS-ECM embodies the characteristics of textual dialogue emotion analysis and is more suitable for emotion prediction tasks.

The main contributions of this paper can be summarized as follows:

• Three characteristics of textual dialogue are summarized: context dependence, persistence and contagiousness;

• A new dialogue emotion analysis task is proposed: dialogue emotion prediction;

• A data-driven model (IDS-ECM) is proposed;

• Experiments on two datasets show that IDS-ECM can correctly predict the emotions in dialogues and reveal the communication difference between different emotion categories in dialogue.

The rest of the paper is organized as follows. Section 2 presents some related works. We introduce and verify the proposed model IDS-ECM in Sections 3 IDS-ECM: interactive double states emotion cell model, 4 Experiments respectively. Section 5 presents some future plans.