Dual-View Conditional Variational Auto-Encoder for Emotional Dialogue Generation

The VAE [7, 15] is a directed graphical model with continuous latent variables, and it is widely used in the generation task of image and natural language. Different from traditional autoencoder, the VAE encodes an input x into a probability distribution, then reconstructs the original input with a decoder network by sampling a continuous latent variable z from this probability distribution, as illustrated in Figure 2. A formal description of the problem is as follows. Let x be an observation of random variable, taking values in . We assume that the generation of x involves a continuous latent variable z, taking values in , by means of a point density , parametrized by θ. Given a set of observed data points , the goal of maximum likelihood estimation is to eatimate the parameters θ that maximize the marginal log-likelihood :Due to the integration over the latent variables, it is intractable to directly compute or differentiate the marginal log-likelihood. A common approach is to maximize a variational lower bound on the marginal log-likelihood by introducing an approximate posterior :where KL denotes the Kullback–Leibler divergence. The evidence lower bound can be also rewritten as a minimum description length loss function:where the neural network with parameters φ, called “recognition” model, is introduced to approximate the true posterior . Another neural network with parameters θ, which is represented as , is aim to reconstruct the data. In general, we assume that is a multivariate diagonal Gaussian distribution:For particularly simple parametric forms of , one can backpropagate through the sampling process by applying the reparametrization trick, which first samples , and then computes . As a result, the VAE can be trained efficiently using stochastic gradient descent. This is essential in VAE training.

The CVAE is a modification of VAE based on certain attributes, e.g., generating different human faces given skin color, gender, age and so on [42], or generating different sentences given sentiment topic and so on. The formula is as follows where the c in this formula is condition:

Variational encoder-decoders have shown promising results in text generation [4, 34, 43]. Straightforwardly optimizing with Equation (4) results in the KL-vanishing problem where the Recurrent Neural Network (RNN) part ends up explaining all the structures without making use of the latent representation. Much meaningful work has been done to alleviate this problem [11, 14, 45]. When dealing with text generation, the CVAE model can generate more diverse sentences than the Seq2Seq model. However, in the emotional dialogue generation task, the general CVAE model is not powerful enough to be consistent with the corresponding emotion. Our proposed method employs CVAE as the baseline to accommodate the fine-grained emotions.

The content relevance module uses the Long Short-Term Memory (LSTM) model with a multi-head attention mechanism [32] to model the semantics of query–response pairs and predict their relatedness. This attention mechanism is designed to focus on the phrases of a response that are connected to the query content. This mechanism allows the model to learn pair-specific representations that are more effective at predicting responses, which better contributes to the predictions of key information in the inference process. Instead of performing a single attention function, Vaswani et al. [32] found it is beneficial to capture different context with multiple individual attention functions. More specifically, multi-head attention model first transforms Q, K, and V into H subspaces, with different, learnable linear projections.

Aiming at building pairwise relevance directly, multi-head attention calculates attention weights between each pair of elements in all input. Figure 4 demonstrates an overview of our content relevance model. Query interacts with response, resulting in a weighted vector representation (the green vector) for generating content-related latent vectors(z_r). To contain sentence-level information, we first encode the query and response into fixed-size vectors using Bi-LSTM. The input contains the last hidden state of query q and all hidden states of response that are expressed as . Formally, for an H heads model, the hth head can be computed aswhere means the hth trainable parameter matrices and is the attention weight between the q and jth response elements. It can be calculated asandin which are parameter matrices and d_k represents the dimensionality of its subspace. Finally, we get the concatenation of all weighted response,

This part mainly introduces the emotional consistency module, which is the green part on the left side of Figure 3. Figure 5 demonstrates a detail structure of our emotion consistency model. Our goal is to incorporate the emotion features into the basic CVAE model, and we first introduce self-attention described in Xu et al. [40] into CVAE. Generally speaking, the attention score partly reflects the sentiment contribution of each word in a well-trained sentiment classifier. Emotional words tend to get a higher score in sentences. Therefore we use the attention scores to denote the emotion representation in response. We encode this emotional representation into a probability distribution that is supervised by emoji tag, and drawing samples of z_e from the learned emotional distribution with a recognition network in the training time. During inference, we use the emoji vector to predict the emotion-related representation directly. In this part, we did not use the multi-head attention mechanism, because the multi-head attention mechanism has more parameters, and the redundant attention weights are a kind of noise for extracting emotional information. The experimental results show that the self-attention mechanism is better than the multi-head attention mechanism. The details of sentiment classifier are described as follows.

This module assumes that the last hidden state of encoder represents the information of the whole sentence, and then uses this last hidden state to compute the attention weights with all previous hidden states. This is why it is called self-attention. Finally, the weighted vectors are mapped to the softmax layer to calculate the final classification distribution. The final sentence representation before softmax layer iswhere is the hidden state of the Bi-LSTM in the ith time step and is the attention weight computed aswhere is an alignment module and is the final hidden state of sentences that contains all information of an input sentence.

Zhou and Wang [49] constructed a dataset by collecting data from the Twitter website, annotated as MojiTalk dataset in this article. The MojiTalk dataset contains 64 common emojis as labels. The corpus consists of 596,959/32,600/32,600 conversation pairs for train /validation/test set. However, the emoji category of this dataset is extremely unbalanced, and the top-1 category accounts for more than 30% of the total data. Moreover, this dataset is an English dataset and no Chinese dataset is available. To test the model in more scenarios than Twitter’s emoji dataset, we collected a large-scale Chinese emoji conversation data from the Weibo.

To construct a large-scale emoji-enriched post-comment conversation dataset, we first crawl hundreds of millions of post-reply conversation pairs on Sina Weibo. Weibo is a popular Twitter-like microblogging service in China on which a user can post short messages and comments on others’ posts. The most popular emoji expressions in Weibo are in the form of pictures. Each picture has a tag like “[haha]” or “[doge],” so we use the emoji tag instead of emoji Unicode. During data pre-processing, we remove those data pairs whose responses do not contain any emoji. Then we filter out potential advertisements and a large number of repetitive high-frequency responses such as “[heart][heart][heart].” We finally select 40 most frequent emoji labels for our experiments. Figure 6 shows some statistics of the dataset used in this work. Similarly to the extraction method in Zhou and Wang [49], we select the emoji in the response as the tag. If there are multiple types of emoji in a response, then we use the emoji with most occurrences inside the response. If there are multiple emoji with the same frequency, then we choose emoji with lower frequency in the whole corpus. Finally, we select 698,167 training data, while the validation and test data are 9,966 and 5,200 respectively.

We also use a high-quality coarse-grained dataset NLPCC2017 Emotional Conversation Generation dataset.³ This dataset has 1,119,207 post-response pairs and each post and response are annotated with 6 emotion labels (happy, angry, sad, like, disgust, Other).

Automatic evaluation. Automatic evaluation of the open domain generation dialogue model is still an open research challenge [21]. We use both quantitative metrics and human judgment to evaluate the proposed models. The quantitative metrics include not only PPL (perplexity) but also the DIST-n, which is a kind of metric to evaluate the degree of diversity of generated responses. PPL explicitly measures the model’s ability to interpret the syntactic structure of dialogue and each utterance. Lower PPL is indicative of a better model. Distinct N-grams (DIST-n) calculates the percentage of the distinct n-grams in all the n-grams of the generated responses [18]. We calculated the DIST-1 and DIST-2 scores that measure the degree of the unigram and bigram diversity.

Human Evaluation. In the human judgment, we ask three judges to evaluate 100 random items, each of which consists of a query, a given emoji and two generated responses from different models. We evaluate three aspects including relevance, consistency and overall evaluation respectively. We present our results from these three aspects. In the overall aspect, we present the original query, emoji, and generated responses, and the judges are asked to decide which one is a better reply to the original query and emoji. In the relevance aspect, we present the original query and generated responses, and the judges are asked to choose the one that is more relevant to the query. In the consistency setting, the emoji label is provided, and the judges are asked to select the better one that fits this emoji. Finally, we treat the 300 results as 300 cases and average the results.

Seq2Seq. The sequence to sequence model has been successfully applied to a variety of tasks ranging from machine translation to speech recognition and dialogue generation. Here we use the traditional attention-enhanced Seq2Seq model [22] as our baseline. To model the emotion, the emoji tag is generated as the same way of the word embedding and is further reduced to smaller size through a dense layer. The small dense emotion vector together with the encoder output are fed to a decoder in the initial time. Response is generated step by step from the decoder.

Transformer. Sequence to sequence model solely on attention mechanisms. The emotion tags are encoded embeddings and add to the inputs of the decoder.

Emotional Chatting Machine (ECM) a sequence-to-sequence model with the emotion category embeddings and internal and external memory mechanisms [46]. The external memory mechanisms of ECM needs an external manual dictionary of emotions. As far as we know, there is no fine-grained dictionary of emotions. So we compare with ECM model only on coarse-grained dataset and our mainly experiments is on fine-grained dataset.

CVAE. As described in Section 3.1, the CVAE uses the emotion tag as a condition in the generation, which is similar to the Seq2Seq model. To get z, we use another same structure of encoder to encode the response into a dense vector and use it to generate z by using MLP. Based on the emotion latent vector, the encoder output of query and the emotion tag dense embedding, the decoder generates the response step by step.

RL-CVAE. RL-CVAE [49] is a combination of the above CVAE model and reinforcement learning using the classifier’s result as rewards. The formula is as follows:where α is a variant coefficient related to the rank of emoji label probability, the higher R ranks in all types of emoji label, the closer α to 0, the value of α ∈ [0,1]. r is the baseline reward and R is the generation reward. λ is a balancing coefficient. This two-stage method is based on the pre-trained CVAE model.

To verify the effectiveness of our approach, we conduct several experiments on two fine-grained datasets and one coarse-grained dataset. Prior studies have shown that pretraining can achieve better performance [49]. Therefore, we pre-train Seq2Seq together with self-attention emoji classifier. We perform training with the following hyperparameters: word embedding has size 128, bi-directional LSTM is used for the encoder, and the dimension of the LSTM hidden units set to 128. To be fair, the ECM’s hidden units and embedding size are also set 128. The latent variable z in baselines CVAE and RL-CVAE has a size of 268. The relevance latent variable size is 64, and the latent variable size in the consistency module is 128. We use Adam learning [13] to update the gradient and clip the gradient to 5.0. Finally, we use the BOW loss for both emotional consistency module and content relevance module, along with KL annealing [4] to achieve the best performance. For the transformer model, we use a smaller Transformer than the base configuration [32]: three layers, 278-dimensional embedding, 507-dimensional inner layers, two heads, and the dropout is 0.1.

Automatic Evaluation. The quantitative evaluation results are shown in Table 1, Table 2, and Table 3. We use the PPL to test the fluency of generated sentences. The DIST-n metric indicates the diversity of generated sentences. Intuitively a good model should achieve low perplexity and high DIST-n scores. From the results, we can see that our Dual-View CVAE outperforms baselines in terms of PPL and distinct measures. Our model achieves lower PPL on both fine-grained datasets and coarsed-grained dataset. These results demonstrate that our model maintains good fluency while generating texts with emotions. In terms of DIST-1 metric, our model performs best on Weibo dataset. Regarding to DIST-2, our model works best on MojiTalk dataset. This indicates that the response diversity generated by our model is at least on par with the baselines, but the fluency is much better than the baselines. In addition, we found that RL-CVAE is very prone to model collapse and generate meaningless continuous non-repetitive words. This result greatly increases the value of DIST-n, but the fluency of generated sentences is affected. The RL-CVAE model is trained on the basis of pre-trained CVAE model, while CVAE model is trained on the pre-trained Seq2Seq model. The results of CVAE model pre-training greatly affect the results of RL model. When the CVAE model converges to a certain extent, the RL model cannot be improved at all. CVAE is relatively stable whose effect is better than ECM model, and our model is superior to CVAE and does not require additional pre-training beyond Seq2Seq.

Considering the consistency between the emotion category of the generated sentence and the given emoji label, we also performed an experiment on the classification of the generated sentences on fine-grained datasets. Since the meaning of different emojis may overlap with only a subtle difference, there are potentially multiple types of emotion in reaction to an utterance. Some automatic categorization is considered wrong, but the results of human assessment are indeed acceptable. This makes the difference of automatic and human evaluations. Specially, in the Weibo dataset the “[heart]” and “[love],” which are the second- and third-largest categories in the dataset, are similar. This afftects the top-1 accuracy of the classifier to some extent. So we also use top-5 accuracy as an automatic evaluation metric. As the results are shown in Table 4, our model is also superior to baselines in emotional categories of generated sentences in the MojiTalk Dataset, which further indicates that the emotional consistency module we display is effective. In the Weibo dataset, our model performs better than other models on top-5 accuracy metric. In Figure 7, we drew a comparison of the accuracy of CVAE, RL-CVAE, and DV-CVAE models among the top 16 emotions in the English MojiTalk dataset. In most cases, the accuracy of DV-CVAE is higher than that of the other two models.

Human Evaluation. At present, there is no unified and authoritative criterion to test the correlation between sentences in dialogues. Most papers use human evaluation, so we use human evaluation to test the relevance and consistency. And we calculated the Fleiss’ kappa [9] to measure the consistency among annotators. Fleiss’ Kappa values are mostly between 0.511 and 0.613, which indicates that most of them are “Moderate agreement.”

As shown in Table 5 and Table 6, the results of human evaluation of our model are obviously better than those of baselines. Compared with the RL-CVAE model, the percentage of “Win” in the Chinese dataset is as high as 46% in relevance and consistency. In the English dataset, the percentage of “Win” is also more than 35%. In the Weibo dataset, the percentage of “Win” in our model exceeds that of “Lose” by more than 15%. At the same time, our model has obvious advantages in the English MojiTalk dataset. The percentage of “Win” in our model exceeds that of “Lose” by more than 5%. From the results of Table 5 and Table 6, we observe that our Dual-view CVAE model significantly outperforms the baseline methods. Notably, our model obtains particularly high scores in both relevance and consistency aspects, which indicates that our model is capable of capturing the content and emotion information. That is, our model does not treat the words in the sentence uniformly but can better capture the key information.

To further verify the effect of the relevance and consistency part, we use a case study to visualize the attention weights. As shown in Figure 8, the affective consistency module is more inclined to notice some words related to the expression of mood and emotion, such as “ang” in the example, which is a common modal particle. The correlation module pays more attention to the words related to the content, such as “you already so thin” in the example. The word “thin” is associated with both content and emotion, so the weight of attention in each is similar. We also show some examples of the response generation in Table 7. In these examples, the assigned emoji is [doge], which represents a lonely bachelor/bachelorette or expresses pride, silence, or disdain. We can see that the Seq2Seq model tends to generate safe responses and the CVAE model can generate content-related responses but cannot consistent with the emotion. The RL-CVAE model seems better, generating more emotion-related responses. However, it is hard for RL-CVAE to cover content and emotion two aspects, which is just the reason why we propose the dual-view CVAE model. Our model is more prone to generate content related to the query, such as “fitness” and “sports” and shows more consistent response in terms of emotions. In addition, we also provide the results of giving different emoji conditions for the same query, as shown in Table 8. From the table, we can see that emoji tags are effective in changing the emotional expression of response in our DV-CVAE model.