Abstract
In this paper, we propose a new model for building a conversational dialogue system which provides natural, realistic and flexible interaction between human and machine based on large movie subtitles dataset. Our models are a generative model that is autonomously generated word-by-word, opening up the possibility of working on many different languages. To address this goal, we extend the hierarchical recurrent encoder-decoder neural network (HRED) for dealing with extracting features from long input turns and generating long output turns. Furthermore, we also apply an attention mechanism in order to attend to particular sentences on source side when predicting a turn response. The models are trained end-to-end without labeling data. The experiments show that our proposed model has improved 36% on BLUE score compared to the HRED model. It also shows many potential improvements in chatbot models.
M. Van Quan and T. D. Le—Contributed equally to this manuscript.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ameixa, D., Coheur, L., Fialho, P., Quaresma, P.: Luke, I am your father: dealing with out-of-domain requests by using movies subtitles. In: Bickmore, T., Marsella, S., Sidner, C. (eds.) Intelligent Virtual Agents, pp. 13–21. Springer, Cham (2014)
Banchs, R.E.: Movie-DiC: a movie dialogue corpus for research and development. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers - Volume 2, ACL 2012, Stroudsburg, PA, USA, pp. 203–207. Association for Computational Linguistics (2012)
Britz, D.: Recurrent neural networks tutorial, part 1 - introduction to RNNs, September 2015. https://goo.gl/zfQtHi
Ilya, S., Oriol, V., V, L.Q.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112 (2014)
Mohandas, G.: Recurrent neural network (RNN) – part 4: attentional interfaces, November 2016. https://goo.gl/RjrwqW
Olah, C.: Understanding LSTM networks, August 2015. https://goo.gl/eyu2wT
Olah, C.: Visual information theory, October 2015. https://goo.gl/V2Up5C
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics (ACL), pp. 311–318, July 2002
Ritter, A., Cherry, C., Dolan, W.B.: Data-driven response generation in social media, January 2011
Sepp, H., Jürgen, S.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Shang, L., Lu, Z., Li, H.: Neural responding machine for short-text conversation. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1577–1586. Association for Computational Linguistics (2015)
Sordoni, A., Bengio, Y., Vahabi, H., Lioma, C., Grue Simonsen, J., Nie, J.Y.: A hierarchical recurrent encoder-decoder for generative context-aware query suggestion. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, CIKM 2015, New York, NY, USA, pp. 553–562. ACM (2015)
Serban, I.V., Sordoni, A., Bengio, Y., Courville, A., Pineau, J.: Building end-to-end dialogue systems using generative hierarchical neural network models. In: Thirtieth AAAI Conference on Artificial Intelligence (2016)
Yoshua, B., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5, 157–166 (1994)
Acknowledgment
This work is sponsored and supported by TIS INC. (Japan) under the collaboration between Hochiminh City University of Technology and TIS INC.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Van Quan, M., Le, T.D., Nguyen, D.D. (2019). An Alternative Deep Model for Turn-Based Conversations. In: Lee, S., Ismail, R., Choo, H. (eds) Proceedings of the 13th International Conference on Ubiquitous Information Management and Communication (IMCOM) 2019. IMCOM 2019. Advances in Intelligent Systems and Computing, vol 935. Springer, Cham. https://doi.org/10.1007/978-3-030-19063-7_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-19063-7_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-19062-0
Online ISBN: 978-3-030-19063-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)