Skip to main content
SpringerLink
Log in

End-to-End latent-variable task-oriented dialogue system with exact log-likelihood optimization

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

We propose an end-to-end dialogue model based on a hierarchical encoder-decoder, which employed a discrete latent variable to learn underlying dialogue intentions. The system is able to model the structure of utterances dominated by statistics of the language and the dependencies among utterances in dialogues without manual dialogue state design. We argue that the latent discrete variable interprets the intentions that guide machine responses generation. We also propose a model which can be refined autonomously with reinforcement learning, due to that intention selection at each dialogue turn can be formulated as a sequential decision-making process. Our experiments show that exact MLE optimized model is much more robust than neural variational inference on dialogue success rate with limited BLEU sacrifice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1

Similar content being viewed by others

Notes

  1. Similar to LIDM, all dialogues are pre-processed by delexicalization [9]. Based on the ontology, slot-value specific words are substituted with their corresponding generic tokens.

  2. We concatenate hidden states of bidirectional encoder RNNs of the last step to initialize hidden states of decoder RNNs

  3. Except for training with REINFORCE models

  4. http://gd.xxx.ai/static/chat.html

References

  1. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. Computer Science (2014)

  2. Barr, A.: Natural language understanding. AI Mag. 1(1), 5 (2017)

    Article  Google Scholar 

  3. Cuayáhuitl, H., Yu, S., Williamson, A., Carse, J.: Deep reinforcement learning for multi-domain dialogue systems. arXiv:1611.08675 (2016)

  4. Das, R., Dhuliawala, S., Zaheer, M., Vilnis, L., Durugkar, I., Krishnamurthy, A., Smola, A., McCallum, A.: Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning. arXiv:1711.05851 (2017)

  5. Dhingra, B., Li, L., Li, X., Gao, J., Chen, Y.N., Ahmed, F., Deng, L.: Towards end-to-end reinforcement learning of dialogue agents for information access. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 484–495 (2017)

  6. Eric, M., Manning, C.D.: A copy-augmented sequence-to-sequence architecture gives good performance on task-oriented dialogue. arXiv:1701.04024(2017)

  7. Gašić, M., Breslin, C., Henderson, M., Kim, D., Szummer, M., Thomson, B., Tsiakoulis, P., Young, S.: On-Line policy optimisation of bayesian spoken dialogue systems via human interaction. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8367–8371. IEEE (2013)

  8. Gruber, A., Weiss, Y., Rosen-Zvi, M.: Hidden topic markov models. In: Artificial Intelligence and Statistics, pp. 163–170 (2007)

  9. Henderson, M., Thomson, B., Young, S.: Word-based dialog state tracking with recurrent neural networks. In: Meeting of the Special Interest Group on Discourse and Dialogue, pp. 292–299 (2014)

  10. Higgins, I., Matthey, L., Pal, A., Burgess, C., Glorot, X., Botvinick, M., Mohamed, S., Lerchner, A.: beta-vae: Learning basic visual concepts with a constrained variational framework. In: Proceedings of International Conference on Learning Representations (ICLR) (2017)

  11. Kingma, D., Ba, J.: Adam: a Method for Stochastic Optimization. In: The International Conference on Learning Representations (ICLR) (2015)

  12. Liu, B., Tur, G., Hakkani-Tur, D., Shah, P., Heck, L.: End-to-end optimization of task-oriented dialogue model with deep reinforcement learning. arXiv:1711.10712 (2017)

  13. Madotto, A., Wu, C.S., Fung, P.: Mem2seq: Effectively incorporating knowledge bases into end-to-end task-oriented dialog systems. arXiv:1804.08217(2018)

  14. Mnih, A., Gregor, K.: Neural variational inference and learning in belief networks. In: Proceedings of the 34th International Conference on Machine Learning (ICML). 1402.0030 (2014)

  15. Mrkšić, N., Séaghdha, D.Ó., Thomson, B., Gasic, M., Su, P.H., Vandyke, D., Wen, T.H., Young, S.: Multi-domain dialog state tracking using recurrent neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (vol. 2: Short Papers), vol. 2, pp. 794–799 (2015)

  16. Mrksic, N., Séaghdha, D. Ó., Wen, T., Thomson, B., Young, S.J.: Neural belief tracker: Data-driven dialogue state tracking. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 1777–1788 (2017)

  17. Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: a method for automatic evaluation of machine translation. In: Meeting on Association for Computational Linguistics, pp. 311–318 (2002)

  18. Rojas-Barahona, L.M., Gasic, M., Mrksic, N., Su, P., Ultes, S., Wen, T., Young, S.J., Vandyke, D.: A network-based end-to-end trainable task-oriented dialogue system. in: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, Valencia, Spain, April 3-7, 2017, vol. 1: Long Papers, pp. 438–449 (2017)

  19. Serban, I.V., Sordoni, A., Bengio, Y., Courville, A.C., Pineau, J.: Building End-To-End dialogue systems using generative hierarchical neural network models. In: AAAI, pp. 3776–3784 (2016)

  20. Serban, I.V., Sordoni, A., Lowe, R., Charlin, L., Pineau, J., Courville, A.C., Bengio, Y.: A hierarchical latent variable encoder-decoder model for generating dialogues. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI), pp. 3295–3301 (2017)

  21. Sordoni, A., Bengio, Y., Vahabi, H., Lioma, C., Grue Simonsen, J., Nie, J.Y.: A hierarchical recurrent encoder-decoder for generative context-aware query suggestion. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp. 553–562. ACM (2015)

  22. Su, P.H., Vandyke, D., Gasic, M., Kim, D., Mrksic, N., Wen, T.H., Young, S.: Learning from real users: Rating dialogue success with neural networks for reinforcement learning in spoken dialogue systems. In: Proceedings of the 11th Annual Conference of the International Speech Communication Association (INTERSPEECH) (2015)

  23. Wen, T., Gasic, M., Mrksic, N., Su, P., Vandyke, D., Young, S.J.: Semantically conditioned lstm-based natural language generation for spoken dialogue systems. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1711–1721. 1508.01745 (2015)

  24. Wen, T.H., Miao, Y., Blunsom, P., Young, S.: Latent intention dialogue models. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol 70, pp. 3732–3741. PMLR, International Convention Centre, Sydney, Australia. http://proceedings.mlr.press/v70/wen17a.html (2017)

  25. Williams, J., Raux, A., Henderson, M.: The dialog state tracking challenge series: a review. Dialogue & Discourse 7(3), 4–33 (2016)

    Google Scholar 

  26. Williams, J.D., Asadi, K., Zweig, G.: Hybrid code networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning. arXiv:1702.03274 (2017)

  27. Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8(3-4), 229–256 (1992)

    Article  Google Scholar 

  28. Wu, C.S., Madotto, A., Winata, G.I., Fung, P.: End-To-End dynamic query memory network for entity-value independent task-oriented dialog. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6154–6158. IEEE (2018)

  29. Young, S., Gašić, M., Thomson, B., Williams, J.D.: Pomdp-based statistical spoken dialog systems: A review. Proc. IEEE 101(5), 1160–1179 (2013)

    Article  Google Scholar 

  30. Young, T., Cambria, E., Chaturvedi, I., Zhou, H., Biswas, S., Huang, M.: Augmenting End-To-End dialogue systems with commonsense knowledge. In: AAAI, pp. 4970–4977 (2018)

  31. Zhang, Y., Dai, H., Kozareva, Z., Smola, A.J., Song, L.: Variational reasoning for question answering with knowledge graph. arXiv:1709.04071(2017)

  32. Zhao, T., Lu, A., Lee, K., Eskénazi, M.: Generative encoder-decoder models for task-oriented spoken dialog systems with chatting capability. In: 18th Annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL). 1706.08476 (2017)

  33. Zhao, T., Xie, K., Eskenazi, M.: Rethinking action spaces for reinforcement learning in end-to-end dialog agents with latent variable models. arXiv:1902.08858 (2019)

Download references

Acknowledgements

This work was supported by the Shenzhen Science and Technology Innovation Committee with the project name of Intelligent Question Answering Robot, under grant NO. CKCY20170508121036342.

Author information

Authors and Affiliations

  1. Zhiyan Technology (Shenzhen) Limited, Shenzhen, China

    Haotian Xu, Liuyang Zhou & Weiguo Zheng

  2. School of Computer Science and Engineering, Nanyang Technological University, Singapore, Singapore

    Haiyun Peng & Erik Cambria

  3. Department of Mathematics and Information Technology, The Education University of Hong Kong, Tai Po, Hong Kong

    Haoran Xie

Authors
  1. Haotian Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haiyun Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haoran Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Erik Cambria
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Liuyang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Weiguo Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liuyang Zhou.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Computational Social Science as the Ultimate Web Intelligence

Guest Editors: Xiaohui Tao, Juan D. Velasquez, Jiming Liu, and Ning Zhong

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, H., Peng, H., Xie, H. et al. End-to-End latent-variable task-oriented dialogue system with exact log-likelihood optimization. World Wide Web 23, 1989–2002 (2020). https://doi.org/10.1007/s11280-019-00688-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-019-00688-8

Keywords

Search

Navigation