Abstract
The problem of unknown words in neural machine translation (NMT), which not only affects the semantic integrity of the source sentences but also adversely affects the generating of the target sentences. The traditional methods usually replace the unknown words according to the similarity of word vectors, these approaches are difficult to deal with rare words and polysemous words. Therefore, this paper proposes a new method of unknown words processing in NMT based on the semantic concept of the source language. Firstly, we use the semantic concept of source language semantic dictionary to find the candidate in-vocabulary words. Secondly, we propose a method to calculate the semantic similarity by integrating the source language model and the semantic concept network, to obtain the best replacement word. Experiments on English to Chinese translation task demonstrate that our proposed method can achieve more than 2.6 BLEU points over the conventional NMT method. Compared with the traditional method based on word vector similarity, our method can also obtain an improvement by nearly 0.8 BLEU points.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Kalchbrenner, N., Blunsom, P.: Recurrent continuous translation models (2013)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. Adv. Neural. Inf. Process. Syst. 4, 3104–3112 (2014)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. Comput. Sci. (2014)
Li, X., Zhang, J., Zong, C.: Towards zero unknown word in neural machine translation. In: International Joint Conference on Artificial Intelligence. AAAI Press, pp. 2852–2858 (2016)
Miller, G.A.: WordNet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)
Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: BLEU: a method for automatic valuation of machine translation. In: Proceedings of 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, Pennsylvania, USA, pp. 311–318, July 2002
Zeiler, M.D.: ADADELTA: an adaptive learning rate method. Comput. Sci. (2012)
Srivastava, N., Hinton, G., Krizhevsky, A., et al.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Collins, M., Koehn, P.: Clause restructuring for statistical machine translation. In: Meeting on Association for Computational Linguistics. Association for Computational Linguistics, pp. 531–540 (2005)
Meng, F., Lu, Z., Li, H., et al.: Interactive attention for neural machine translation (2016)
Luong, M.T., Sutskever, I., Le, Q.V., et al.: Addressing the rare word problem in neural machine translation. Vet. Med. 27(2), 82–86 (2014). Bulletin of University of Agricultural Sciences and Veterinary Medicine Cluj-Napoca
Mikolov, T., Sutskever, I., Chen, K., et al.: Distributed representations of words and phrases and their compositionality. Adv. Neural. Inf. Process. Syst. 26, 3111–3119 (2013)
Stolcke, A.: SRILM—an extensible language modeling toolkit. In: International Conference on Spoken Language Processing. pp. 901–904 (2002)
Acknowledgments
The authors are supported by the National Nature Science Foundation of China (Contract 61370130 and 61473294), and the Fundamental Research Funds for the Central Universities (2015JBM033), and the International Science and Technology Cooperation Program of China under grant No. 2014DFA11350.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Li, S., Xu, J., Miao, G., Zhang, Y., Chen, Y. (2018). A Semantic Concept Based Unknown Words Processing Method in Neural Machine Translation. In: Huang, X., Jiang, J., Zhao, D., Feng, Y., Hong, Y. (eds) Natural Language Processing and Chinese Computing. NLPCC 2017. Lecture Notes in Computer Science(), vol 10619. Springer, Cham. https://doi.org/10.1007/978-3-319-73618-1_20
Download citation
DOI: https://doi.org/10.1007/978-3-319-73618-1_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-73617-4
Online ISBN: 978-3-319-73618-1
eBook Packages: Computer ScienceComputer Science (R0)