Abstract
Neural machine translation has achieved remarkable progress recently, but it is restricted by a limited vocabulary due to the computation complexity. All words out of the vocabulary are replaced with a single UNK, and the UNK in translation results will hurt the quality of translation. In this paper, a UNK translation method is proposed to handle the unknown word issue in neural machine translation. It uses n-best source alignment candidates for UNK translation, and can handle both word level (one-to-one) and phrase level (many-to-one) source-UNK alignment. Experiments on Chinese-to-English task shows that our method achieves a +0.73 BLEU improvement over the NMT baseline that has already employed a good UNK translation module.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)
Bengio, Y., Senecal, J.: Adaptive importance sampling to accelerate training of a neural probabilistic language model. IEEE Trans. Neural Netw. 19(4), 713–722 (2008)
Cho, K., Van Merri Senboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation, encoder-decoder approaches. arXiv preprint arXiv:1409.1259 (2014)
Chorowski, J.K., Bahdanau, D., Serdyuk, D., Cho, K., Bengio, Y.: Attention-based models for speech recognition. In: Advances in Neural Information Processing Systems, pp. 577–585 (2015)
Chung, J., Cho, K., Bengio, Y.: A character-level decoder without explicit segmentation for neural machine translation. arXiv preprint arXiv:1603.06147 (2016)
Gers, F.A., Schmidhuber, J., Cummins, F.: Learning to forget: continual prediction with LSTM. Neural Comput. 12(10), 2451–2471 (2000)
Gillick, D., Brunk, C., Vinyals, O., Subramanya, A.: Multilingual language processing from bytes. arXiv preprint arXiv:1512.00103 (2015)
Heafield, K.: Kenlm: faster and smaller language model queries. In: Proceedings of 6th Workshop on Statistical Machine Translation, pp. 187–197. Association for Computational Linguistics (2011)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Jean, S., Cho, K., Memisevic, R., Bengio, Y.: On using very large target vocabulary for neural machine translation, pp. 1–10 (2014)
Koehn, P., Och, F.J., Marcu, D.: Statistical phrase-based translation, pp. 48–54 (2003)
Luong, M.-T., Pham, H., Manning, C.D.: Effective approaches to attention based neural machine translation. In: Proceedings of NAACL 2015, pp. 1412–1421 (2015a)
Luong, M.-T., Sutskever, I., Le, Q.V., Vinyals, O., Zaremba, W.: Addressing the rare word problem in neural machine translation. In: Proceedings of ACL 2015, pp. 11–19.s (2015b)
Morin, F., Bengio, Y.: Hierarchical probabilistic neural network language model. In: Aistats, vol. 5, pp. 246–252. Citeseer (2005)
Och, F.J.: Minimum error rate training in statistical machine translation, pp. 160–167 (2003)
Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of 40th Annual Meeting on Association for Computational Linguistics, pp. 311–318. Association for Computational Linguistics (2002)
Sennrich, R., Haddow, B., Birch, A.R.: Neural machine translation of rare words with subword units, pp. 1715–1725 (2015)
Sennrich, R., Haddow, B., Birch, A.R.: Edinburgh neural machine translation systems for WMT 16. arXiv preprint arXiv:1606.02891 (2016)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112 (2014)
Xiao, T., Zhu, J., Zhang, H., Li, Q.: NiuTrans: an open source toolkit for phrase based and syntax-based machine translation, pp. 19–24 (2012)
XiaoQing, L., Chengqing Z., Jiajun, Z.: Towards zero unknown word in neural machine translation (2016)
Zhu, J., Xiao, T., Zhang, C.: Learning better rule extraction with translation span alignment, pp. 280–284 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Li, F., Quan, D., Qiang, W., Tong, X., Zhu, J. (2017). Handling Many-To-One UNK Translation for Neural Machine Translation. In: Wong, D., Xiong, D. (eds) Machine Translation. CWMT 2017. Communications in Computer and Information Science, vol 787. Springer, Singapore. https://doi.org/10.1007/978-981-10-7134-8_10
Download citation
DOI: https://doi.org/10.1007/978-981-10-7134-8_10
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-7133-1
Online ISBN: 978-981-10-7134-8
eBook Packages: Computer ScienceComputer Science (R0)