Abstract
Back translation refers to the method of using machine translation to automatically translate target language monolingual data into source language data, which is a commonly used data augmentation method in machine translation tasks. Previous researchers’ works on back translation only focus on rich resource languages, while ignoring the low resource language with different quality. In this paper, we compare various monolingual selection methods, different model performance, pseudo-data and parallel corpus ratios, and different data generation methods for the validity of pseudo-data in machine translation tasks. Experiments on Lithuanian and Gujarati, two low-resource languages have shown that increasing the distribution of low-frequency words and increasing data diversity are more effective for models with sufficient training, while the results of insufficient models are opposite. In this paper, different back-translation strategies are used for different languages, and compared with common back-translation methods in WMT news tasks of two languages, and the effectiveness of the strategies is verified by experiments. At the same time, we find that combined back-translation strategies are more effective than simply increasing the amount of pseudo-data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bahdanau, D., Cho, K., Bengio, Y., et al.: Neural machine translation by jointly learning to align and translate. In: International Conference on Learning Representations (2015)
Luong, T., Pham, H., Manning, C.D., et al.: Effective approaches to attention-based neural machine translation. In: Empirical Methods in Natural Language Processing, pp. 1412–1421 (2015)
Sennrich, R., Haddow, B., Birch, A., et al.: Improving neural machine translation models with monolingual data. In: Meeting of the Association for Computational Linguistics, pp. 86–96 (2016)
Fadaee, M., Monz, C.: Back-translation sampling by targeting difficult words in neural machine translation. In: Empirical Methods in Natural Language Processing, pp. 436–446 (2018)
Axelrod, A., Vyas, Y., Martindale, M., Carpuat, M., Hopkins, J.: Classbased n-gram language difference models for data selection. In: IWSLT (International Workshop on Spoken Language Translation), pp. 180–187 (2015)
Edunov, S., Ott, M., Auli, M., et al.: Understanding back-translation at scale. In: Empirical Methods in Natural Language Processing, pp. 489–500 (2018)
Moore, R.C., Lewis, W.D.: Intelligent selection of language model training data. In: Meeting of the Association for Computational Linguistics, pp. 220–224 (2010)
Post, M.: A Call for Clarity in Reporting BLEU Scores. arXiv: Computation and Language, pp. 186–191 (2018)
Sennrich, R., Haddow, B., Birch, A., et al.: Neural machine translation of rare words with subword units. In: Meeting of the Association for Computational Linguistics, pp. 1715–1725 (2016)
Vaswani, A., Shazeer, N., Parmar, N., et al.: Attention is all you need. In: Neural Information Processing Systems, pp. 5998–6008 (2017)
Brown, P.F., Pietra, V.J., Pietra, S.D., et al.: The mathematics of statistical machine translation: parameter estimation. Computat. Linguist. 19(2), 263–311 (1993)
Acknowledgments
This work was supported in part by the National Science Foundation of China (Nos. 61876035, 61732005 and 61432013) and the National Key R&D Program of China (No. 2019QY1801).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Xu, N. et al. (2019). Analysis of Back-Translation Methods for Low-Resource Neural Machine Translation. In: Tang, J., Kan, MY., Zhao, D., Li, S., Zan, H. (eds) Natural Language Processing and Chinese Computing. NLPCC 2019. Lecture Notes in Computer Science(), vol 11839. Springer, Cham. https://doi.org/10.1007/978-3-030-32236-6_42
Download citation
DOI: https://doi.org/10.1007/978-3-030-32236-6_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32235-9
Online ISBN: 978-3-030-32236-6
eBook Packages: Computer ScienceComputer Science (R0)
