Abstract
The transformer-based neural machine translation (NMT) model has achieved remarkable success in the sequence-to-sequence NMT paradigm, exhibiting state-of-the-art performance. However, its reliance on abundant bilingual data resources poses a significant challenge, especially when dealing with scarce parallel sentence pairs. In such scenarios, the translation performance often deteriorates sharply. To alleviate this issue, this paper introduces a novel data augmentation (DA) approach for the NMT model. It leverages bilingual templates to augment the training set, thereby enhancing the generalization ability of the NMT model. Firstly, a template extraction algorithm is devised to generate sentence templates for both the source and target sentences in the original bilingual corpus. Subsequently, two data augmentation strategies are employed to expand the training corpus. By incorporating these augmented data into the training process, the NMT model is exposed to a broader range of linguistic phenomena, enabling it to better handle low-resource scenarios. The experimental results conducted on both simulated and real low-resource translation tasks reveal that the proposed DA approach significantly enhances translation performance. When compared to a robust baseline and several other data augmentation techniques, the proposed method consistently outperforms its counterparts, demonstrating its efficacy and versatility in addressing the challenges posed by limited parallel data.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)
Gehring, J., Auli, M., Grangier, D., Yarats, D., Dauphin, Y.N.: Convolutional sequence to sequence learning. In: International Conference on Machine Learning, pp. 1243–1252. PMLR (2017)
Wu, Y., et al.: Google’s neural machine translation system: bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 (2016)
Vaswani, A., et al.: Attention is all you need. Adv. Neural Inf. Process. Syst. 30 (2017)
Zoph, B., Yuret, D., May, J., Knight, K.: Transfer learning for low-resource neural machine translation. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 1568–1575 (2016)
Gu, J., Wang, Y., Chen, Y., Cho, K., Li, V.O.: Meta-learning for low-resource neural machine translation. In: 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018, pp. 3622–3631. Association for Computational Linguistics (2018)
Ren, S., Chen, W., Liu, S., Li, M., Zhou, M., Ma, S.: Triangular architecture for rare language translation. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, vol. 1: Long Papers, pp. 56–65 (2018)
Sennrich, R., Haddow, B., Birch, A.: Improving neural machine translation models with monolingual data. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, vol. 1: Long Papers. Association for Computational Linguistics (2016)
Zhang, J., Zong, C.: Exploiting source-side monolingual data in neural machine translation. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 1535–1545 (2016)
Fadaee, M., Bisazza, A., Monz, C.: Data augmentation for low-resource neural machine translation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, vol. 2: Short Papers, pp. 567–573 (2017)
Wang, X., Pham, H., Dai, Z., Neubig, G.: Switchout: an efficient data augmentation algorithm for neural machine translation. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (2018)
Edunov, S., Ott, M., Auli, M., Grangier, D.: Understanding back-translation at scale. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, p. 489. Association for Computational Linguistics (2018)
Fadaee, M., Monz, C., et al.: Back-translation sampling by targeting difficult words in neural machine translation (2018)
Sennrich, R., Haddow, B., Birch, A.: Edinburgh neural machine translation systems for wmt 16. In: Proceedings of the First Conference on Machine Translation, vol. 2, Shared Task Papers, pp. 371–376 (2016)
Gal, Y., Ghahramani, Z.: A theoretically grounded application of dropout in recurrent neural networks. Adv. Neural Inf. Process. Syst. 29 (2016)
Gao, F., et al.: Soft contextual data augmentation for neural machine translation. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5539–5544 (2019)
Poncelas, A., Shterionov, D., Way, A., Wenniger, G., Passban, P.: Investigating backtranslation in neural machine translation 2018 (1804)
Ueffing, N., Haffari, G., Sarkar, A.: Transductive learning for statistical machine translation. In: Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pp. 25–32 (2007)
Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J.R., Bethard, S., McClosky, D.: The stanford corenlp natural language processing toolkit. In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55–60 (2014)
Artetxe, M., Labaka, G., Agirre, E., Cho, K.: Unsupervised neural machine translation. In: 6th International Conference on Learning Representations, ICLR 2018 (2018)
Xie, Z., et al.: Data noising as smoothing in neural network language models. In: 5th International Conference on Learning Representations, ICLR 2017 (2019)
Burlot, F., Yvon, F.: Using monolingual data in neural machine translation: a systematic study. In: Conference on Machine Translation (2018)
Cotterell, R., Kreutzer, J.: Explaining and generalizing back-translation through wake-sleep. arXiv preprint arXiv:1806.04402 (2018)
Hoang, C.D.V., Koehn, P., Haffari, G., Cohn, T.: Iterative back-translation for neural machine translation. In: 2nd Workshop on Neural Machine Translation and Generation, pp. 18–24. Association for Computational Linguistics (2018)
He, D., et al.: Dual learning for machine translation. Adv. Neural Inf. Process. Syst. 29 (2016)
Li, F., Chi, C., Yan, H., Liu, B., Shao, M.: STA: an efficient data augmentation method for low-resource neural machine translation. J. Intell. Fuzzy Syst. 1–12 (2023)
Zhang, J., Matsumoto, T.: Corpus augmentation by sentence segmentation for low-resource neural machine translation. arXiv preprint arXiv:1905.08945 (2019)
Chi, C., Li, F., Yan, H., Guan, H., Zhao, Z.: A data augmentation method based on sub-tree exchange for low-resource neural machine translation. In: Huang, D.S., Premaratne, P., Jin, B., Boyang, Q., Jo, K.-H., Hussain, A. (eds.) Advanced Intelligent Computing Technology and Applications: 19th International Conference, ICIC 2023, Zhengzhou, China, August 10–13, 2023, Proceedings, Part IV, pp. 646–657. Springer Nature Singapore, Singapore (2023). https://doi.org/10.1007/978-981-99-4752-2_53
Chen, K., Wang, R., Utiyama, M., Sumita, E.: Content word aware neural machine translation. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 358–364 (2020)
Ott, M., et al.: fairseq: a fast, extensible toolkit for sequence modeling. arXiv preprint arXiv:1904.01038 (2019)
Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909 (2015)
Cheng, Q., Huang, J., Duan, Y.: Semantically consistent data augmentation for neural machine translation via conditional masked language model. arXiv preprint arXiv:2209.10875 (2022)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318 (2002)
Duan, S., Zhao, H., Zhang, D.: Syntax-aware data augmentation for neural machine translation. IEEE/ACM Trans. Audio Speech Lang. Process. 31, 2988–2999 (2023)
Acknowledgments
This work was supported by the Natural Science Foundation of Liaoning Province of China (Joint Fund of Regional Innovation) under Grant No. 2022-YKLH-18; The Project of Yingkou Institute of Technology under Gant No. ZDIL202305 and YBL202316.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Li, F. et al. (2024). A Bilingual Templates Data Augmentation Method for Low-Resource Neural Machine Translation. In: Huang, DS., Si, Z., Zhang, Q. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2024. Lecture Notes in Computer Science(), vol 14877. Springer, Singapore. https://doi.org/10.1007/978-981-97-5669-8_4
Download citation
DOI: https://doi.org/10.1007/978-981-97-5669-8_4
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-5668-1
Online ISBN: 978-981-97-5669-8
eBook Packages: Computer ScienceComputer Science (R0)