Abstract
In this paper, an auto-encoder is proposed to learn conversation representation. First, the long short term memory (LSTM) neural network is used to encode the sequence of sentences in a conversation. The interactive context is encoded into a fixed-length vector. Then, through the LSTM-decoder, the learnt representation is used to reconstruct the sentence vectors of a conversation. To train our model, we construct one corpus with 32,881 conversations from the online shopping platform. Finally, experiments on topic recognition task demonstrate the effectiveness of the proposed auto-encoder on learning conversation representation, especially when training data of topic recognition is relatively small.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Hsueh, P.-Y., Moore, J.D., Renals, S.: Automatic segmentation of multi-party dialogue. In: Proceedings of EACL, pp. 273–280 (2006)
Purver, M., Körding, K., Griffiths, T.L., Tenenbaum, J.: Unsupervised topic modelling for multi-party spoken discourse. In: Proceedings of COLING-ACL, pp. 17–24 (2006)
Stolcke, A., Coccaro, N., Bates, R., Taylor, P., Van Ess-Dykema, C., Ries, K., Shriberg, E., Jurafsky, D., Martin, R., Meteer, M.: Dialogue act modeling for automatic tagging and recognition of conversational speech. Comput. Linguist. 26(3), 339–374 (2000)
Rieser, V., Lemon, O.: Natural language generation as planning under uncertainty for spoken dialogue systems. In: Proceedings of EACL, pp. 683–691 (2009)
Liu, J., Seneff, S., Zue, V.: Dialogue-oriented review summary generation for spoken dialogue recommendation systems. In: Proceedings of NAACL, pp. 64–72 (2010)
Graves, A.: Generating sequences with recurrent neural networks. CoRR, abs/1308.0850 (2013)
Sutskever, I., Vinyals, O., Le, Q.V.V.: Sequence to sequence learning with neural networks. In: Advances in NIPS, pp. 3104–3112, (2014)
Cho, K., van Merrienboer, B.., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of EMNLP, pp. 1724–1734 (2014)
Shang, L., Lu, Z., Li, H.: Neural responding machine for short-text conversation. CoRR, abs/1503.02364 (2015)
Srivastava, N., Mansimov, E., Salakhutdinov, R.: Unsupervised learning of video representations using LSTMs. CoRR, abs/1502.04681 (2015)
Zhai, K., Williams, J.: Discovering latent structure in task-oriented dialogues. In: Proceedings of ACL. pp. 36–46 (2014)
Hochreiter, S., Bengio, Y., Frasconi, P., Schmidhuber, J.: Gradient flow in recurrent nets: the difficulty of learning long-term dependencies. In: Kolen, J.F., Kremer, S. (eds.) A Field Guide to Dynamical Recurrent Neural Networks, vol. 28, pp. 297–318. IEEE Press, New York (2001)
Graves, A., Jaitly, N.: Towards end-to-end speech recognition with recurrent neural networks. In Proceedings of ICML, pp. 1764–1772, (2014)
Mikolov, T.., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. CoRR, abs/1301.3781 (2013)
Zhou, X., Hu, B., Chen, Q., Tang, B., Wang, X.: Answer sequence learning with neural networks for answer selection in community question answering. In: Proceedings of ACL-IJCNLP, pp. 713–718 (2015)
Li, J., Luong, M.-T., Jurafsky, D.: A hierarchical neural autoencoder for paragraphs and documents. In: Proceedings of ACL-IJCNLP, pp. 1106–1115 (2015)
Hu, B., Lu, Z., Li, H., Chen, Q.: Convolutional neural network architectures for matching natural language sentences. In: Advances in NIPS, pp. 2042–2050 (2014)
Socher, R., Pennington, J., Huang, E.H., Ng, A.Y., Manning, C.D.: Semi-supervised recursive autoencoders for predicting sentiment distributions. In: Proceedings of EMNLP, pp. 151–161 (2011)
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Le, Q., Mikolov, T.: Distributed representations of sentences and documents. In: Proceedings of ICML, pp. 1189–1196 (2014)
Hu, B., Chen, Q., Zhu, F.: LCSTS: a large scale chinese short text summarization dataset. CoRR, abs/1506.05865 (2015)
Jurafsky, D., Shriberg, E., Biasca, D.: Switchboard SWBD-DAMSL shallowdiscourse-function annotation coders manual. Institute of Cognitive Science Technical report, pp. 97–02 (1997)
Acknowledgements
This paper is supported in part by grants: National 863 Program of China (2015AA015405), National Natural Science Foundation of China (61473101 and 61272383).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Zhou, X., Hu, B., Chen, Q., Wang, X. (2015). An Auto-Encoder for Learning Conversation Representation Using LSTM. In: Arik, S., Huang, T., Lai, W., Liu, Q. (eds) Neural Information Processing. ICONIP 2015. Lecture Notes in Computer Science(), vol 9489. Springer, Cham. https://doi.org/10.1007/978-3-319-26532-2_34
Download citation
DOI: https://doi.org/10.1007/978-3-319-26532-2_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26531-5
Online ISBN: 978-3-319-26532-2
eBook Packages: Computer ScienceComputer Science (R0)