Skip to main content
SpringerLink
Log in

Neural Machine Translation for Morphologically Rich Languages with Improved Sub-word Units and Synthetic Data

  • Conference paper
  • First Online:
Text, Speech, and Dialogue (TSD 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10415))

Included in the following conference series:

Abstract

This paper analyses issues of rare and unknown word splitting with byte pair encoding for neural machine translation and proposes two methods that allow improving the quality of word splitting. The first method linguistically guides byte pair encoding and the second method limits splitting of unknown words. We also evaluate corpus re-translation for a new language pair – English-Latvian. We show a significant improvement in translation quality over baseline systems in all reported experiments. We envision that the proposed methods will allow improving the translation of named entities and technical texts in production systems that often receive data not represented in the training corpus.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/rsennrich/nematus.

  2. 2.

    https://github.com/rsennrich/wmt16-scripts/blob/master/sample/config.py.

  3. 3.

    http://statmt.org/wmt17/translation-task.html.

References

  1. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: Proceedings of the International Conference on Learning Representations (ICLR) (2015)

    Google Scholar 

  2. Bojar, O., Chatterjee, R., Federmann, C., Graham, Y., Haddow, B., Huck, M., Jimeno Yepes, A., Koehn, P., Logacheva, V., Monz, C., Negri, M., Neveol, A., Neves, M., Popel, M., Post, M., Rubino, R., Scarton, C., Specia, L., Turchi, M., Verspoor, K., Zampieri, M.: Findings of the 2016 conference on machine translation. In: Proceedings of the First Conference on Machine Translation (WMT 2016), vol. 2, pp. 131–198 (2016). Shared Task Papers

    Google Scholar 

  3. Caglayan, O., Barrault, L., Bougares, F.: Multimodal Attention for Neural Machine Translation (2016). http://arxiv.org/abs/1609.03976

  4. Devlin, J., Zbib, R., Huang, Z., Lamar, T., Schwartz, R.M., Makhoul, J.: Fast and robust neural network joint models for statistical machine translation. In: ACL (1), pp. 1370–1380. Citeseer (2014)

    Google Scholar 

  5. Dyer, C., Chahuneau, V., Smith, N.A.: A Simple, fast, and effective reparameterization of IBM model 2. In: Proceedings of NAACL HLT 2013, pp. 644–648, Atlanta, June 2013

    Google Scholar 

  6. Firat, O., Cho, K., Bengio, Y.: Multi-way, multilingual neural machine translation with a shared attention mechanism. In: NAACL-HLT 2016, pp. 866–875 (2016)

    Google Scholar 

  7. Girgždis, V., Kāle, M., Vaicekauskis, M., Zariņa, I., Skadiņa, I.: Tracing mistakes and finding gaps in automatic word alignments for Latvian-English translation. In: Proceedings of Baltic HLT 2014, pp. 87–94. IOS Press (2014)

    Google Scholar 

  8. Jean, S., Firat, O., Cho, K., Memisevic, R., Bengio, Y.: Montreal neural machine translation systems for WMT15. In: Proceedings of WMT 2015, pp. 134–140 (2015)

    Google Scholar 

  9. Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., Dyer, C., Bojar, O., Constantin, A., Herbst, E.: Moses: open source toolkit for statistical machine translation. In: Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions, ACL 2007, pp. 177–180. Association for Computational Linguistics, Stroudsburg (2007)

    Google Scholar 

  10. Koehn, P., Och, F.J., Marcu, D.: Statistical phrase-based translation. In: Proceedings of NAACL HLT 2013, pp. 48–54. Association for Computational Linguistics (2003)

    Google Scholar 

  11. Lee, J., Cho, K., Hofmann, T.: Fully Character-Level Neural Machine Translation without Explicit Segmentation (2016)

    Google Scholar 

  12. Luong, M.T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. In: Proceedings of EMNLP 2015, pp. 1412–1421. Association for Computational Linguistics, Lisbon (2015)

    Google Scholar 

  13. Meng, F., Lu, Z., Li, H., Liu, Q.: Interactive attention for neural machine translation. In: Proceedings of the 26th International Conference on Computational Linguistics, Osaka, Japan, pp. 2174–2185 (2016)

    Google Scholar 

  14. Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 311–318. Association for Computational Linguistics (2002)

    Google Scholar 

  15. Sennrich, R., Haddow, B.: Linguistic input features improve neural machine translation. In: Proceedings of the First Conference on Machine Translation (WMT 2016), vol. 1, pp. 83–91 (2016). Research Papers

    Google Scholar 

  16. Sennrich, R., Haddow, B., Birch, A.: Edinburgh neural machine translation systems for WMT 16. In: Proceedings of the First Conference on Machine Translation (WMT 2016), vol. 2 (2016). Shared Task Papers

    Google Scholar 

  17. Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with subword units. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL 2016), Berlin, Germany (2016)

    Google Scholar 

  18. Skadiņš, R., Goba, K., Šics, V.: Improving SMT for baltic languages with factored models. In: Proceedings of the Fourth International Conference on Human Language Technologies: The Baltic Perspective, Baltic HLT 2010, vol. 219, pp. 125–132. IOS Press (2010)

    Google Scholar 

  19. Stanojevic, M., Sima’an, K.: BEER: BEtter evaluation as ranking. In: Proceedings of the Ninth Workshop on Statistical Machine Translation, pp. 414–419 (2014)

    Google Scholar 

  20. Vasiļjevs, A., Skadiņš, R., Tiedemann, J.: LetsMT!: a cloud-based platform for do-it-yourself machine translation. In: Proceedings of the ACL 2012 System Demonstrations, pp. 43–48. Association for Computational Linguistics, Jeju Island (2012)

    Google Scholar 

  21. Wang, W., Peter, J.T., Rosendahl, H., Ney, H.: CharacTER: translation edit rate on character level. In: Proceedings of the First Conference on Machine Translation (WMT 2016), Berlin, Germany, vol. 2, pp. 505–510 (2016). Shared Task Papers

    Google Scholar 

Download references

Acknowledgments

The research has been supported by the European Regional Development Fund within the research project “Neural Network Modelling for Inflected Natural Languages” No. 1.1.1.1/16/A/215.

Author information

Authors and Affiliations

  1. Tilde, Vienibas gatve 75A, Riga, Latvia

    Mārcis Pinnis, Rihards Krišlauks, Daiga Deksne & Toms Miks

Authors
  1. Mārcis Pinnis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rihards Krišlauks
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daiga Deksne
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Toms Miks
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mārcis Pinnis .

Editor information

Editors and Affiliations

  1. University of West Bohemia, Pilsen, Czech Republic

    Kamil Ekštein

  2. University of West Bohemia, Pilsen, Czech Republic

    Václav Matoušek

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Pinnis, M., Krišlauks, R., Deksne, D., Miks, T. (2017). Neural Machine Translation for Morphologically Rich Languages with Improved Sub-word Units and Synthetic Data. In: Ekštein, K., Matoušek, V. (eds) Text, Speech, and Dialogue. TSD 2017. Lecture Notes in Computer Science(), vol 10415. Springer, Cham. https://doi.org/10.1007/978-3-319-64206-2_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-64206-2_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-64205-5

  • Online ISBN: 978-3-319-64206-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation