Yue Yl L
ABSTRACT
At present, generating responses based on neural network learning has become a hot spot, and has gradually entered a new stage of pre-training language model. The intelligence and transferability of the dialogue system in the open domain is becoming more and more obvious, but there are still many problems, such as single reply, logic contradiction, general security answer. So as to solve the weakness in reply, we starting from the context, this paper uses the method of Point Mutual Information (PMI)to calculate the relevant weight between the context and the current dialogue to explicitly weight the context and give full play to the effective information of the current dialogue. Further inputing into the dialogue generation of the pre-training language model for fine-tuning. In the experimental evaluation, we make a comprehensive analysis from three aspects: automatic evaluation, objective index calculation and manual evaluation. The results show that our method of explicit weighted context coding will enrich the coding information further generating more diverse and meaningful responses, can be significantly improved compared with the baseline model.
- Y. Wu, W. Wu, C. Xing, Z. Li, and M. Zhou (2016). Sequential matching network: A new architecture for multi-turn response selection in retrieval-based chatbots. arXiv preprint arXiv:1612.01627.Google Scholar
- Z. Tian, R. Yan, L. Mou, Y. Song, Y. Feng, and D. Zhao (2017, July). How to make context more useful? an empirical study on context-aware neural conversational models. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) (pp. 231--236).Google ScholarCross Ref
- A. Gravano (2009). Turn-taking and affirmative cue words in task-oriented dialogue.Google Scholar
- R. Zellers, Y. Bisk, R. Schwartz, and Y. Choi (2018). Swag: A large-scale adversarial dataset for grounded commonsense inference. arXiv preprint arXiv:1808.05326.Google Scholar
- Y. Wu, W. Wu, C. Xing, C. Xu, Z. Li, M. Zhou (2019). A sequential matching framework for multi-turn response selection in retrieval-based chatbots. Computational Linguistics, 45(1), 163--197.Google ScholarDigital Library
- Y. Wang, P. Ke, Y. Zheng, K. Huang, Y. Jiang (2020, October). A large-scale chinese short-text conversation dataset. In CCF International Conference on Natural Language Processing and Chinese Computing (pp. 91--103). Springer, Cham.Google Scholar
- L. Shang, Z. Lu, and H. Li (2015). Neural responding machine for short-text conversation. arXiv preprint arXiv:1503.02364.Google Scholar
- A. Radford, K. Narasimhan, T. Salimans, I. Sutskever (2018). Improving language understanding by generative pre-training.Google Scholar
- I. Serban, A. Sordoni, R. Lowe, L. Charlin (2017, February). A hierarchical latent variable encoder-decoder model for generating dialogues. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 31, No. 1).Google Scholar
- R. Yan, Y. Song, and H. Wu (2016, July). Learning to respond with deep neural networks for retrieval-based human-computer conversation system. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval (pp. 55--64).Google ScholarDigital Library
- I. Serban, A. Sordoni, Y. Bengio, A. Courville, and J. Pineau (2016, March). Building end-to-end dialogue systems using generative hierarchical neural network models. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 30, No. 1).Google Scholar
- C. Qu, L. Yang, M. Qiu, W. Bruce Croft, Y. Zhang, and M. Iyyer (2019, July). BERT with history answer embedding for conversational question answering. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 1133--1136).Google Scholar
- K. Yao, G. Zweig, and B. Peng (2015). Attention with intention for a neural network conversation model. arXiv preprint arXiv:1510.08565.Google Scholar
- A. Radford, J. Wu, R. Child, D. Luan, D. Amodei (2019). Language models are unsupervised multitask learners. OpenAI blog, 1(8), 9.Google Scholar
- Y. Ma, K. L. Nguyen, F. Z. Xing, and E. Cambria (2020). A Survey on Empathetic Dialogue Systems. Information Fusion.Google Scholar
- S. Bao, H. He, F. Wang, H. Wu, and H. Wang (2019). Plato: Pre-trained dialogue generation model with discrete latent variable. arXiv preprint arXiv:1910.07931.Google Scholar
- D. Bahdanau, K. Cho, and Y. Bengio (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.Google Scholar
- Y. Zhang, S. Sun, M. Galley, YC. Chen, C. Brockett (2019). Dialogpt: Large-scale generative pre-training for conversational response generation. arXiv preprint arXiv:1911.00536.Google Scholar
- C. Hao, L. Pang, Y. Lan, F. Sun, J. Guo, and X. Cheng (2020, October). Ranking Enhanced Dialogue Generation. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (pp. 465--474).Google ScholarDigital Library
- L. Ma, W. Zhang, R. Sun, and T. Liu (2020). A Compare Aggregate Transformer for Understanding Document-grounded Dialogue. arXiv preprint arXiv:2010.00190.Google Scholar
Index Terms
- Research on Dialogue Generation Algorithm Based on Explicit Weighted Context
Recommendations
Interpretation and generation of dialogue with multidimensional context models
Proceedings of the Third COST 2102 international training school conference on Toward autonomous, adaptive, and context-aware multimodal interfaces: theoretical and practical issuesThis paper presents a context-based approach to the analysis and computational modeling of communicative behaviour in dialogue. This approach, known as Dynamic Interpretation Theory (DIT), claims that dialogue behaviour is multifunctional, i.e. ...
Hard-style Selective Context Utilization for dialogue generation based on what user just said
AbstractDialogue is a process of information exchanging, where global background is stable while local focuses are transiting. Thus, at the ongoing dialogue turn, there are both relevant and irrelevant semantics existing in dialogue contexts. How to ...
Explicit State Tracking with Semi-Supervisionfor Neural Dialogue Generation
CIKM '18: Proceedings of the 27th ACM International Conference on Information and Knowledge ManagementThe task of dialogue generation aims to automatically provide responses given previous utterances. Tracking dialogue states is an important ingredient in dialogue generation for estimating users' intention. However, the expensive nature of state ...
Comments