Sirui Zhao
Enhong Chen
Skip Abstract Section
Abstract
As a vivid and linguistic symbol, Emojis have become a prevailing medium interspersed in text-based communication (e.g., social media and chit-chat) to express emotions, attitudes, and situations. Generally speaking, a social-oriented chatbot that can generate appropriate Emoji-embedded responses would be much more competitive, making communications more fun, engaging, and human-like. However, the current Emoji-related research is still in its infancy, leading to an awkward situation of data deficiency. How to develop an Emoji-embedded dialogue system while addressing the lack of data will be interesting and meaningful for the application of future AI. To bridge this gap, we propose a multi-task learning method for persona-aware Emoji-embedded dialogue generation in this article. Specifically, as the benchmark of model training and evaluation, which includes 1.2 million Emoji-embedded tweets and 1.1 million post-response pairs, we first construct a dataset named EmojiTweet to handle the data deficiency problem. Then, a Seq2Seq-based model with multi-task learning is designed to simultaneously learn response generation and Emoji embedding from the constructed non-Emoji dialogue and Emoji-embedded monologue data. Afterward, we incorporate persona factors into our model by adopting persona fusion and personalized bias methods to deliver personalized dialogues with more accurately selected Emojis. Finally, we conduct extensive experiments, where the experimental results and evaluations demonstrate that our model has three key benefits: improved dialogue quality, higher user engagement, and not relying on large-scale Emoji-embedded dialogue data representing specific personas. EmojiTweet will be published publicly via https://mea-lab-421.github.io/EmojiTweet/.
- [1] . 2016. Tensorflow: A system for large-scale machine learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI’16). 265–283.Google Scholar
- [2] . 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).Google Scholar
- [3] . 2018. Interpretable emoji prediction via label-wise attention LSTMs. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 4766–4771.Google ScholarCross Ref
- [4] . 2018. Semeval 2018 task 2: Multilingual emoji prediction. In Proceedings of the 12th International Workshop on Semantic Evaluation. 24–33.Google ScholarCross Ref
- [5] . 2018. Interpretable emoji prediction via label-wise attention LSTMs. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 4766–4771.
DOI: Google ScholarCross Ref - [6] . 2018. Exploring emoji usage and prediction through a temporal variation lens. arXiv preprint arXiv:1805.00731 (2018).Google Scholar
- [7] . 2000. A model of inductive bias learning. Journal of Artificial Intelligence Research 12 (2000), 149–198.Google ScholarCross Ref
- [8] . 2003. A neural probabilistic language model. Journal of Machine Learning Research 3 (2003), 1137–1155.Google ScholarDigital Library
- [9] . 2017. Differences in emoji sentiment perception between readers and writers. In 2017 IEEE International Conference on Big Data (Big Data’17). IEEE, 4321–4328.Google ScholarCross Ref
- [10] . 2018. Twemoji Dataset. Data retrieved from University of Amsterdam /Amsterdam University of Applied Sciences. https://doi.org/10.21942/uva.5822100.v3Google Scholar
- [11] . 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014).Google Scholar
- [12] . 2019. Bam! Born-again multi-task networks for natural language understanding. arXiv preprint arXiv:1907.04829 (2019).Google Scholar
- [13] . 2018. Are emoji a poor substitute for words? Sentence processing with emoji substitutions. In CogSci.Google Scholar
- [14] Jan Deriu, Alvaro Rodrigo, Arantxa Otegi, Guillermo Echegoyen, Sophie Rosset, Eneko Agirre, and Mark Cieliebak. 2021. Survey on evaluation methods for dialogue systems. Artificial Intelligence Review 54, 1 (2021), 755–810.Google Scholar
- [15] . 2018. Neural approaches to conversational AI. The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 1371–1374.Google ScholarDigital Library
- [16] . 2020. Are GRU cells more specific and LSTM cells more sensitive in motive classification of text. J. Front. Artif. Intell 3 (2020), 1–6.Google Scholar
- [17] . 2017. WeChat: Social and political development of China’s dominant messaging app. Chinese Journal of Communication 10, 3 (2017), 312–327.Google ScholarCross Ref
- [18] . 2019. Analyzing incorporation of emotion in emoji prediction. In Proceedings of the 10th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis. 91–99.Google ScholarCross Ref
- [19] . 2020. Multimodal dialogue systems via capturing context-aware dependencies of semantic elements. In Proceedings of the 28th ACM International Conference on Multimedia. 2755–2764.Google ScholarDigital Library
- [20] . 2008. Generating dialogues for virtual agents using nested textual coherence relations. In International Workshop on Intelligent Virtual Agents. Springer, 139–145.Google Scholar
- [21] . 1997. Long short-term memory. Neural Computation 9, 8 (1997), 1735–1780.Google ScholarDigital Library
- [22] . 2020. Challenges in building intelligent open-domain dialog systems. ACM Transactions on Information Systems (TOIS) 38, 3 (2020), 1–32.Google ScholarDigital Library
- [23] . 2020. Automatic prediction and insertion of multiple emojis in social media text. In IEEE International Conference on Cyber Physical and Social Computing (CPSCom’20). IEEE, 505–512.Google ScholarCross Ref
- [24] . 2020. Genre-based emoji usage analysis and presiction in video comments. In International Conference on Cyber Science and Technology Congress (CyberSciTech’20). IEEE, 296–305.Google Scholar
- [25] . 2021. How emotional are emoji?: Exploring the effect of emotional valence on the processing of emoji stimuli. Computers in Human Behavior 116 (2021), 106648.Google ScholarCross Ref
- [26] . 2016. “Turn that frown upside-down”: A contextual account of emoticon usage on different virtual platforms. Computers in Human Behavior 60 (2016), 463–467.Google ScholarDigital Library
- [27] . 2020. No more one liners: Bringing context into emoji recommendations. ACM Transactions on Social Computing 3, 2 (2020), 1–25.Google ScholarDigital Library
- [28] . 2017. Exploring personalized neural conversational models. In IJCAI. 3728–3734.Google Scholar
- [29] . 2015. A diversity-promoting objective function for neural conversation models. arXiv preprint arXiv:1510.03055 (2015).Google Scholar
- [30] . 2016. A diversity-promoting objective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 110–119.Google ScholarCross Ref
- [31] . 2016. A persona-based neural conversation model. arXiv preprint arXiv:1603.06155 (2016).Google Scholar
- [32] . 2018. Mining the relationship between emoji usage patterns and personality. In 12th International AAAI Conference on Web and Social Media.Google ScholarCross Ref
- [33] . 2014. Emoticon recommendation in microblog using affective trajectory model. In 2014 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA’14). IEEE, 1–5.Google ScholarCross Ref
- [34] . 2017. Multi-task learning for speaker-role adaptation in neural conversation models. arXiv preprint arXiv:1710.07388 (2017).Google Scholar
- [35] . 2019. Positive emotion elicitation in chat-based dialogue systems. IEEE/ACM Transactions on Audio, Speech, and Language Processing 27, 4 (2019), 866–877.Google ScholarDigital Library
- [36] . 2018. Emoji grammar as beat gestures. In Proceedings of the 1st International Workshop on Emoji Understanding and Applications in Social Media, Standford [en línea]. Disponible en http://knoesis.org/resources/Emoji2018/Emoji2018_Papers/Paper13_Emoji2018.pdf [Consulta 11/12/2019].Google Scholar
- [37] . 2019. Food recommendation: Framework, existing solutions, and challenges. IEEE Transactions on Multimedia 22, 10 (2019), 2659–2671.Google ScholarCross Ref
- [38] . 2003. Minimum error rate training in statistical machine translation. In Proceedings of the 41st annual meeting of the Association for Computational Linguistics. 160–167.Google ScholarDigital Library
- [39] . 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics. 311–318.Google ScholarDigital Library
- [40] . 2018. Large-scale hierarchical text classification with recursively regularized deep graph-CNN. In Proceedings of the 2018 World Wide Web Conference. 1063–1072.Google ScholarDigital Library
- [41] . 2018. Assigning personality/profile to a chatting machine for coherent conversation generation. In IJCAI. 4279–4285.Google Scholar
- [42] . 2022. Unified QA-aware knowledge graph generation based on multi-modal modeling. In Proceedings of the 30th ACM International Conference on Multimedia. 7185–7189.Google ScholarDigital Library
- [43] . 2017. An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098 (2017).Google Scholar
- [44] . 2019. The Emoji Revolution: How Technology is Shaping the Future of Communication. Cambridge University Press.Google ScholarCross Ref
- [45] . 2015. Improving neural machine translation models with monolingual data. Computer Ence (2015).Google Scholar
- [46] Iulian Vlad Serban, Ryan Lowe, Peter Henderson, Laurent Charlin, and Joelle Pineau. 2015. A survey of available corpora for building data-driven dialogue systems. arXiv preprint arXiv:1512.05742 (2015).Google Scholar
- [47] . 2017. The effects of emoji in sentiment analysis. Int. J. Comput. Electr. Eng. (IJCEE) 9, 1 (2017), 360–369.Google ScholarCross Ref
- [48] . 2018. From Eliza to XiaoIce: Challenges and opportunities with social chatbots. Frontiers of Information Technology & Electronic Engineering 19, 1 (2018), 10–26.Google ScholarCross Ref
- [49] . 2019. Exploiting persona information for diverse generation of conversational responses. arXiv preprint arXiv:1905.12188 (2019).Google Scholar
- [50] . 2015. A neural network approach to context-sensitive generation of conversational responses. arXiv preprint arXiv:1506.06714 (2015).Google Scholar
- [51] . 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems. 3104–3112.Google ScholarDigital Library
- [52] Marilyn Walker, Grace Lin, and Jennifer Sawyer. 2012. An annotated corpus of film dialogue for learning and characterizing character style. In Proceedings of the 8th International Conference on Language Resources and Evaluation (LREC’12). 1373–1378.Google Scholar
- [53] . 2016. An exploration of psychological factors on emoticon usage and implications for judgement accuracy. Computers in Human Behavior 62 (2016), 70–78.Google ScholarDigital Library
- [54] . 2017. Group linguistic bias aware neural response generation. In Proceedings of the 9th SIGHAN Workshop on Chinese Language Processing. 1–10.Google Scholar
- [55] . 2018. Sequence-to-sequence learning for task-oriented dialogue with dialogue state representation. arXiv preprint arXiv:1806.04441 (2018).Google Scholar
- [56] . 2018. Tweet emoji prediction using hierarchical model with attention. In Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers. 1337–1344.Google ScholarDigital Library
- [57] . 2018. Tweet emoji prediction using hierarchical model with attention. In Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers.Google Scholar
- [58] . 2020. What are you talking to?: Understanding children’s perceptions of conversational agents. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.Google ScholarDigital Library
- [59] . 2015. Attribute2Image: Conditional image generation from visual attributes. CoRR abs/1512.00570 (2015).
arXiv:1512.00570 http://arxiv.org/abs/1512.00570.Google Scholar - [60] . 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2204–2213.Google ScholarCross Ref
- [61] . 2021. A survey on multi-task learning. IEEE Transactions on Knowledge and Data Engineering (2021).Google Scholar
- [62] Tiancheng Zhao, Ran Zhao, and Maxine Eskénazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. ArXiv abs/1703.10960 (2017).Google Scholar
- [63] . 2019. Personalized dialogue generation with diversified traits. arXiv preprint arXiv:1901.09672 (2019).Google Scholar
- [64] . 2020. How to evaluate single-round dialogues like humans: An information-oriented metric. IEEE/ACM Transactions on Audio, Speech, and Language Processing 28 (2020), 2211–2223.Google ScholarDigital Library
- [65] . 2018. The design and implementation of XiaoIce, an empathetic social chatbot. arXiv preprint arXiv:1812.08989 (2018).Google Scholar
- [66] . 2019. Multi-task learning with language modeling for question generation. arXiv preprint arXiv:1908.11813 (2019).Google Scholar
Index Terms
- PEDM: A Multi-task Learning Model for Persona-aware Emoji-embedded Dialogue Generation
Recommendations
Interpretation and generation of dialogue with multidimensional context models
Proceedings of the Third COST 2102 international training school conference on Toward autonomous, adaptive, and context-aware multimodal interfaces: theoretical and practical issuesThis paper presents a context-based approach to the analysis and computational modeling of communicative behaviour in dialogue. This approach, known as Dynamic Interpretation Theory (DIT), claims that dialogue behaviour is multifunctional, i.e. ...
Explicit State Tracking with Semi-Supervisionfor Neural Dialogue Generation
CIKM '18: Proceedings of the 27th ACM International Conference on Information and Knowledge ManagementThe task of dialogue generation aims to automatically provide responses given previous utterances. Tracking dialogue states is an important ingredient in dialogue generation for estimating users' intention. However, the expensive nature of state ...
Diverse dialogue generation by fusing mutual persona-aware and self-transferrer
AbstractIn recent years, personalized dialogue generation has attracted researchers’ attention due to its wide range of applications. Although there has been much excellent research, current chatbots are known to generate uninteresting responses without ...
Comments