Skip to main content
SpringerLink
Log in
Book cover

International Conference on Intelligent Computing

ICIC 2018: Intelligent Computing Theories and Application pp 194–204Cite as

Hierarchical Hybrid Code Networks for Task-Oriented Dialogue

  • Conference paper
  • First Online:

  • 2329 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10955))

Abstract

Task-oriented dialog system is a research hotspot in natural language processing field. In recent years, the application of neural network (NN) has greatly improved the performance of dialog agent. However, there is still a big gap of performance between human beings and dialog agent, in which the domain knowledge and semantic analysis are not well exploited. In this paper we propose a model of Hierarchical Hybrid Code Networks (HHCNs), in which a word-character RNN for semantic representation and a NN-based selection for domain knowledge are integrated. Thus the proposed HHCNs can effectively conduct semantic analysis (e.g., identify proper nouns and misspelling word) and select meaningful responses for the dialog. The experimental results on the dataset of Dialog State Tracking Challenge 2 (DSTC2) have shown a superior performance of HHCNs.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Levin, E., Pieraccini, R., Eckertm, W.: A stochastic model of human-machine interaction for learning dialog strategies. IEEE Trans. Speech Audio Process. 8(1), 11–23 (2000)

    Article  Google Scholar 

  2. Singh, S., Litman, D., Kearns, M., Walker, M.: Optimizing dialogue management with reinforcement learning: experiments with the NJFun system. J. Artif. Intell. Res. 16(1), 105–133 (2011)

    MATH  Google Scholar 

  3. Williams, J.D., Young, S.: Partially Observable Markov Decision Processes for Spoken Dialog Systems. Academic Press Ltd., London (2007)

    Google Scholar 

  4. Hori, C., Ohtake, K., Misu, T., Kashioka, H., Nakamura, S.: Statistical dialog management applied to WFST-based dialog systems. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 4793–4796 (2009)

    Google Scholar 

  5. Lee, C., Jung, S., Kim, S., Lee, G.G.: Example-based dialog modeling for practical multi-domain dialog system. Speech Commun. 51(5), 466–484 (2009)

    Article  Google Scholar 

  6. Griol, D., Hurtado, L.F., Segarra, E., Sanchis, E.: A statistical approach to spoken dialog systems design and evaluation. Speech Commun. 50(8–9), 666–682 (2008)

    Article  Google Scholar 

  7. Young, S., Gai, M., Thomson, B., Williams, J.D.: Pomdp-based statistical spoken dialog systems: a review. Proc. IEEE 101(5), 1160–1179 (2013)

    Article  Google Scholar 

  8. Li, L., He, H., Williams, J.D.: Temporal supervised learning for inferring a dialog policy from example conversations. In: Spoken Language Technology Workshop, pp. 312–317 (2014)

    Google Scholar 

  9. Serban, I.V., Sordoni, A., Bengio, Y., Courville, A., Pineau, J.: Building end-to-end dialogue systems using generative hierarchical neural network models. In: Thirtieth AAAI Conference on Artificial Intelligence, pp. 3776–3783 (2016)

    Google Scholar 

  10. Sordoni, A., Galley, M., Auli, M., Brockett, C., Ji, Y., Mitchell, M., Nie, J.Y., Gao, J., Dolan, B.: A neural network approach to context-sensitive generation of conversational responses (2015)

    Google Scholar 

  11. Shang, L., Lu, Z., Li, H.: Neural responding machine for short-text conversation, pp. 52–58 (2015)

    Google Scholar 

  12. Vinyals, O., Le, Q.: A neural conversational model. Computer Science (2015)

    Google Scholar 

  13. Yao, K., Zweig, G., Peng, B.: Attention with intention for a neural network conversation model. Computer Science (2015)

    Google Scholar 

  14. Li, J., Galley, M., Brockett, C., Spithourakis, G., Gao, J., Dolan, B.: A persona-based neural conversation model. Meeting of the Association for Computational Linguistics, pp. 994–1003 (2016)

    Google Scholar 

  15. Luan, Y., Ji, Y., Ostendorf, M.: LSTM based conversation models (2016)

    Google Scholar 

  16. Xu, Z., Liu, B., Wang, B., Sun, C., and Wang, X.: In-corporating loose-structured knowledge into LSTM with recall gate for conversation modeling. pp. 3506–3513 (2016)

    Google Scholar 

  17. Li, J., Galley, M., Brockett, C., Gao, J., Dolan, B.: A diversity-promoting objective function for neural conversation models. Computer Science (2015)

    Google Scholar 

  18. Lowe, R.T., Pow, N., Serban, I.V., Charlin, L., Liu, C.-W., Pineau, J.: Training end-to-end dialogue systems with the ubuntu dialogue corpus. Dialogue Discourse 8(1), 31–65 (2017)

    Google Scholar 

  19. Serban, I.V., Sordoni, A., Lowe, R., Charlin, L., Pineau, J., Courville, A., Bengio, Y.: A hierarchical latent variable encoder-decoder model for generating dialogues (2016)

    Google Scholar 

  20. Sukhbaatar, S., Szlam, A., Weston, J., Fergus, R.: End-to-end memory networks. Computer Science (2015)

    Google Scholar 

  21. Perez, J., Liu, F.: Gated end-to-end memory networks (2016)

    Google Scholar 

  22. Gu, J., Lu, Z., Li, H., Li, V.O.K.: Incorporating copying mechanism in sequence-to-sequence learning. pp. 1631–1640 (2016)

    Google Scholar 

  23. Gulcehre, C., Ahn, S., Nallapati, R., Zhou, B., Bengio, Y.: Pointing the unknown words. pp. 140–149 (2016)

    Google Scholar 

  24. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. Computer Science (2014)

    Google Scholar 

  25. Luong, M.T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. Computer Science (2015)

    Google Scholar 

  26. Eric, M., Manning, C.D.: A copy-augmented sequence-to-sequence architecture gives good performance on task-oriented dialogue. pp. 468–473 (2017)

    Google Scholar 

  27. Liu, B., Lane, I.: Iterative policy learning in end-to-end trainable task-oriented neural dialog models (2017)

    Google Scholar 

  28. Seo, M., Min, S., Farhadi, A., Hajishirzi, H.: Query-reduction networks for question answering (2016)

    Google Scholar 

  29. Williams, J.D., Asadi, K., Zweig, G.: Hybrid code networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning. pp. 665–677 (2017)

    Google Scholar 

  30. Henderson, M., Thomson, B., Williams, J.D.: The second dialog state tracking challenge. In: Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pp. 263–272 (2014)

    Google Scholar 

  31. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. 26, 3111–3119 (2013)

    Google Scholar 

  32. Luong, M.T., Manning, C.D.: Achieving open vocabulary neural machine translation with hybrid word-character models. pp. 1054–1063 (2016)

    Google Scholar 

  33. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  34. Chung, J., Gulcehre, C., Cho, K.H., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. Eprint Arxiv (2014)

    Google Scholar 

  35. Kingma, D., Ba, J.: Adam: a method for stochastic optimization. Computer Science (2014)

    Google Scholar 

  36. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

This work is partially supported by the National Natural Science Foundation of China (Grant no. 61772568), Guangzhou Science and Technology Program (Grant no. 201804010288), and Shenzhen Scientific Research and Development Funding Program (Grant no. JCYJ20170302153827712).

Author information

Authors and Affiliations

  1. School of Data and Computer Science, Sun Yat-sen University, Guangzhou, China

    Weiri Liang & Meng Yang

Authors
  1. Weiri Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meng Yang .

Editor information

Editors and Affiliations

  1. Tongji University, Shanghai, China

    De-Shuang Huang

  2. University of Ulsan, Ulsan, Korea (Republic of)

    Kang-Hyun Jo

  3. Wuhan University of Science and Technology, Wuhan City, China

    Xiao-Long Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liang, W., Yang, M. (2018). Hierarchical Hybrid Code Networks for Task-Oriented Dialogue. In: Huang, DS., Jo, KH., Zhang, XL. (eds) Intelligent Computing Theories and Application. ICIC 2018. Lecture Notes in Computer Science(), vol 10955. Springer, Cham. https://doi.org/10.1007/978-3-319-95933-7_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-95933-7_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-95932-0

  • Online ISBN: 978-3-319-95933-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation