Abstract
Current neural machine translation (NMT) usually extracts a fixed-length semantic representation for source sentence, and then depends on this representation to generate corresponding target translation. In this paper, we proposed a pivot-based semantic splicing model (PBSSM) to obtain a semantic representation including more translation information for source sentence, thus improving the translation performance of NMT. The spliced semantic representation is derived from source languages of trilingual parallel corpus by the pivot-based NMT. Besides, the proposed PBSSM only depends on one source language to generate its semantic representation during the encoding process. We integrated it into the NMT architecture. Experiments on the English-Japanese translation task show that our model achieves a substantial improvement by up to 22.9% (3.74 BLEU) over the baseline.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Mikolov, T., Karafiat, M., Burget, L., Cernock, J., Khudanpur, S.: Recurrent neural network based language model. In: INTERSPEECH, pp. 1045–1048 (2010)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Cogn. Model. 5(3), 1 (1988)
Sundermeyer, M., Schlüter, R., Ney, H.: LSTM neural networks for language modeling. Interspeech. (2012)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: NIPS (2014)
Cho, K., van Merrienboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: Encoder – Decoder approaches. In: Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, October 2014a
Luong, M.-T., Le, Q.V., Sutskever, I., Vinyals, O., Kaiser, L.: Multi-task sequence to sequence learning (2015a). arXiv preprint arXiv:1511.06114
Daxiang Dong, Hua Wu, Wei He, Dianhai Yu, and Haifeng Wang. 2015. Multi-task learning for multiple language translation. ACL
Cho, K., Van Merriënboer, B., Gulcehre, C., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint:1406.1078 (2014)
Ando, R.K., Zhang, T.: A framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data. Technical report RC23462, IBM T.J. Watson Research Center (2004)
Cohn, T., Lapata, M.: Machine translation by triangulation: Making effective use of multi-parallel corpora. In: Proceedings ACL (2007)
Hua, W., Wang, H.: Revisiting pivot language approach for machine translation. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, vol. 1. Association for Computational Linguistics (2009)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: ICLR (2015)
Utiyama, M., Isahara, H.: A Comparison of Pivot Methods for Phrase-Based Statistical Machine Translation. HLT-NAACL (2007)
Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45(11), 2673–2681 (1997)
Boulanger-Lewandowski, N., Bengio, Y., Vincent, P.: Audio Chord Recognition with Recurrent Neural Networks. ISMIR (2013)
Orhan, F., Cho, K., Bengio, Y.: Multi-way, multilingual neural machine translation with a shared attention mechanism (2016). arXiv preprint arXiv:1601.01-073
Yang, M., Jiang, H., Zhao, T., Li, S.: Construct Trilingual Parallel Corpus on Demand. In: Huo, Q., Ma, B., Chng, E.-S., Li, H. (eds.) ISCSLP 2006. LNCS (LNAI), vol. 4274, pp. 760–767. Springer, Heidelberg (2006). doi:10.1007/11939993_76
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: The International Conference on Learning Representations (ICLR) (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Liu, D., Zhu, C., Zhao, T., Wang, X., Yang, M. (2016). Pivot-Based Semantic Splicing for Neural Machine Translation. In: Yang, M., Liu, S. (eds) Machine Translation. CWMT 2016. Communications in Computer and Information Science, vol 668. Springer, Singapore. https://doi.org/10.1007/978-981-10-3635-4_2
Download citation
DOI: https://doi.org/10.1007/978-981-10-3635-4_2
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-3634-7
Online ISBN: 978-981-10-3635-4
eBook Packages: Computer ScienceComputer Science (R0)