Skip to main content

A Bilingual Templates Data Augmentation Method for Low-Resource Neural Machine Translation

  • Conference paper
  • First Online:
Advanced Intelligent Computing Technology and Applications (ICIC 2024)

Abstract

The transformer-based neural machine translation (NMT) model has achieved remarkable success in the sequence-to-sequence NMT paradigm, exhibiting state-of-the-art performance. However, its reliance on abundant bilingual data resources poses a significant challenge, especially when dealing with scarce parallel sentence pairs. In such scenarios, the translation performance often deteriorates sharply. To alleviate this issue, this paper introduces a novel data augmentation (DA) approach for the NMT model. It leverages bilingual templates to augment the training set, thereby enhancing the generalization ability of the NMT model. Firstly, a template extraction algorithm is devised to generate sentence templates for both the source and target sentences in the original bilingual corpus. Subsequently, two data augmentation strategies are employed to expand the training corpus. By incorporating these augmented data into the training process, the NMT model is exposed to a broader range of linguistic phenomena, enabling it to better handle low-resource scenarios. The experimental results conducted on both simulated and real low-resource translation tasks reveal that the proposed DA approach significantly enhances translation performance. When compared to a robust baseline and several other data augmentation techniques, the proposed method consistently outperforms its counterparts, demonstrating its efficacy and versatility in addressing the challenges posed by limited parallel data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)

  2. Gehring, J., Auli, M., Grangier, D., Yarats, D., Dauphin, Y.N.: Convolutional sequence to sequence learning. In: International Conference on Machine Learning, pp. 1243–1252. PMLR (2017)

    Google Scholar 

  3. Wu, Y., et al.: Google’s neural machine translation system: bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 (2016)

  4. Vaswani, A., et al.: Attention is all you need. Adv. Neural Inf. Process. Syst. 30 (2017)

    Google Scholar 

  5. Zoph, B., Yuret, D., May, J., Knight, K.: Transfer learning for low-resource neural machine translation. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 1568–1575 (2016)

    Google Scholar 

  6. Gu, J., Wang, Y., Chen, Y., Cho, K., Li, V.O.: Meta-learning for low-resource neural machine translation. In: 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018, pp. 3622–3631. Association for Computational Linguistics (2018)

    Google Scholar 

  7. Ren, S., Chen, W., Liu, S., Li, M., Zhou, M., Ma, S.: Triangular architecture for rare language translation. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, vol. 1: Long Papers, pp. 56–65 (2018)

    Google Scholar 

  8. Sennrich, R., Haddow, B., Birch, A.: Improving neural machine translation models with monolingual data. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, vol. 1: Long Papers. Association for Computational Linguistics (2016)

    Google Scholar 

  9. Zhang, J., Zong, C.: Exploiting source-side monolingual data in neural machine translation. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 1535–1545 (2016)

    Google Scholar 

  10. Fadaee, M., Bisazza, A., Monz, C.: Data augmentation for low-resource neural machine translation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, vol. 2: Short Papers, pp. 567–573 (2017)

    Google Scholar 

  11. Wang, X., Pham, H., Dai, Z., Neubig, G.: Switchout: an efficient data augmentation algorithm for neural machine translation. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (2018)

    Google Scholar 

  12. Edunov, S., Ott, M., Auli, M., Grangier, D.: Understanding back-translation at scale. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, p. 489. Association for Computational Linguistics (2018)

    Google Scholar 

  13. Fadaee, M., Monz, C., et al.: Back-translation sampling by targeting difficult words in neural machine translation (2018)

    Google Scholar 

  14. Sennrich, R., Haddow, B., Birch, A.: Edinburgh neural machine translation systems for wmt 16. In: Proceedings of the First Conference on Machine Translation, vol. 2, Shared Task Papers, pp. 371–376 (2016)

    Google Scholar 

  15. Gal, Y., Ghahramani, Z.: A theoretically grounded application of dropout in recurrent neural networks. Adv. Neural Inf. Process. Syst. 29 (2016)

    Google Scholar 

  16. Gao, F., et al.: Soft contextual data augmentation for neural machine translation. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5539–5544 (2019)

    Google Scholar 

  17. Poncelas, A., Shterionov, D., Way, A., Wenniger, G., Passban, P.: Investigating backtranslation in neural machine translation 2018 (1804)

    Google Scholar 

  18. Ueffing, N., Haffari, G., Sarkar, A.: Transductive learning for statistical machine translation. In: Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pp. 25–32 (2007)

    Google Scholar 

  19. Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J.R., Bethard, S., McClosky, D.: The stanford corenlp natural language processing toolkit. In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55–60 (2014)

    Google Scholar 

  20. Artetxe, M., Labaka, G., Agirre, E., Cho, K.: Unsupervised neural machine translation. In: 6th International Conference on Learning Representations, ICLR 2018 (2018)

    Google Scholar 

  21. Xie, Z., et al.: Data noising as smoothing in neural network language models. In: 5th International Conference on Learning Representations, ICLR 2017 (2019)

    Google Scholar 

  22. Burlot, F., Yvon, F.: Using monolingual data in neural machine translation: a systematic study. In: Conference on Machine Translation (2018)

    Google Scholar 

  23. Cotterell, R., Kreutzer, J.: Explaining and generalizing back-translation through wake-sleep. arXiv preprint arXiv:1806.04402 (2018)

  24. Hoang, C.D.V., Koehn, P., Haffari, G., Cohn, T.: Iterative back-translation for neural machine translation. In: 2nd Workshop on Neural Machine Translation and Generation, pp. 18–24. Association for Computational Linguistics (2018)

    Google Scholar 

  25. He, D., et al.: Dual learning for machine translation. Adv. Neural Inf. Process. Syst. 29 (2016)

    Google Scholar 

  26. Li, F., Chi, C., Yan, H., Liu, B., Shao, M.: STA: an efficient data augmentation method for low-resource neural machine translation. J. Intell. Fuzzy Syst. 1–12 (2023)

    Google Scholar 

  27. Zhang, J., Matsumoto, T.: Corpus augmentation by sentence segmentation for low-resource neural machine translation. arXiv preprint arXiv:1905.08945 (2019)

  28. Chi, C., Li, F., Yan, H., Guan, H., Zhao, Z.: A data augmentation method based on sub-tree exchange for low-resource neural machine translation. In: Huang, D.S., Premaratne, P., Jin, B., Boyang, Q., Jo, K.-H., Hussain, A. (eds.) Advanced Intelligent Computing Technology and Applications: 19th International Conference, ICIC 2023, Zhengzhou, China, August 10–13, 2023, Proceedings, Part IV, pp. 646–657. Springer Nature Singapore, Singapore (2023). https://doi.org/10.1007/978-981-99-4752-2_53

    Chapter  Google Scholar 

  29. Chen, K., Wang, R., Utiyama, M., Sumita, E.: Content word aware neural machine translation. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 358–364 (2020)

    Google Scholar 

  30. Ott, M., et al.: fairseq: a fast, extensible toolkit for sequence modeling. arXiv preprint arXiv:1904.01038 (2019)

  31. Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909 (2015)

  32. Cheng, Q., Huang, J., Duan, Y.: Semantically consistent data augmentation for neural machine translation via conditional masked language model. arXiv preprint arXiv:2209.10875 (2022)

  33. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  34. Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318 (2002)

    Google Scholar 

  35. Duan, S., Zhao, H., Zhang, D.: Syntax-aware data augmentation for neural machine translation. IEEE/ACM Trans. Audio Speech Lang. Process. 31, 2988–2999 (2023)

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the Natural Science Foundation of Liaoning Province of China (Joint Fund of Regional Innovation) under Grant No. 2022-YKLH-18; The Project of Yingkou Institute of Technology under Gant No. ZDIL202305 and YBL202316.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hong Yan .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, F. et al. (2024). A Bilingual Templates Data Augmentation Method for Low-Resource Neural Machine Translation. In: Huang, DS., Si, Z., Zhang, Q. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2024. Lecture Notes in Computer Science(), vol 14877. Springer, Singapore. https://doi.org/10.1007/978-981-97-5669-8_4

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-5669-8_4

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-5668-1

  • Online ISBN: 978-981-97-5669-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics