Abstract
Most natural-language-processing methods are designed for estimating context given an entire set of sentences at once. However, dialogue is incremental in nature. SCAIN (Simultaneous Contextualization and Interpretation) is an algorithm for incremental dialogue processing. Along with the progress of the dialogue, it can solve the interdependence problem in which the interpretation of words depends on the context, and the context is determined by the interpreted words. However, SCAIN cannot process texts that contain more words insignificant to context estimation such as in longer texts. We propose SCAIN with keyword extraction (SCAIN/KE), which extracts keywords that contribute to context estimation and eliminates the effect of insignificant words so that it can process longer texts. In the case study, SCAIN/KE updates context and interpretation better than SCAIN and obtains the keywords that contribute to context estimation better than other statistical methods. In the experiments, we evaluated SCAIN/KE on solving the ambiguity of polysemous words using the Wikipedia disambiguation pages. The results indicate that SCAIN/KE is more accurate than SCAIN.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: 3rd International Conference on Learning Representations, ICLR 2015, January 2015
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 4171–4186. Association for Computational Linguistics, June 2019
Mihalcea, R., Tarau, P.: TextRank: bringing order into text. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pp. 404–411 (2004)
Montemerlo, M., Thrun, S., Koller, D., Wegbreit, B., et al.: FastSLAM: a factored solution to the simultaneous localization and mapping problem. In: AAAI/IAAI, p. 593598 (2002)
Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)
Peters, M., et al.: Deep contextualized word representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), New Orleans, Louisiana, pp. 2227–2237. Association for Computational Linguistics, June 2018. https://doi.org/10.18653/v1/N18-1202, https://www.aclweb.org/anthology/N18-1202
Rose, S., Engel, D., Cramer, N., Cowley, W.: Automatic keyword extraction from individual documents. Text Min. Appl. Theor. 1, 1–20 (2010)
Salton, G., Buckley, C.: Term-weighting approaches in automatic text retrieval. Inf. Process. Manage. 24(5), 513–523 (1988)
Serban, I.V., et al.: A hierarchical latent variable encoder-decoder model for generating dialogues. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pp. 3295–3301 (2017)
Sukhbaatar, S., Weston, J., Fergus, R., et al.: End-to-end memory networks. In: Advances in Neural Information Processing Systems, pp. 2440–2448 (2015)
Takimoto, Y., Fukuchi, Y., Matsumori, S., Imai, M.: Slam-inspired simultaneous contextualization and interpreting for incremental conversation sentences. arXiv preprint arXiv:2005.14662 (2020)
Vilnis, L., McCallum, A.: Word representations via gaussian embedding. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015)
Acknowledgements
This work was supported by JST CREST Grant Number JPMJCR19A1, Japan.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Yoshino, T., Matsumori, S., Fukuchi, Y., Imai, M. (2021). Simultaneous Contextualization and Interpretation with Keyword Awareness. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing. ICAISC 2021. Lecture Notes in Computer Science(), vol 12855. Springer, Cham. https://doi.org/10.1007/978-3-030-87897-9_36
Download citation
DOI: https://doi.org/10.1007/978-3-030-87897-9_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87896-2
Online ISBN: 978-3-030-87897-9
eBook Packages: Computer ScienceComputer Science (R0)