Abstract
In this paper, we propose to use rich semantic and typological information of languages to improve the language selection method for multilingual NMT. In particular, we first use a graph-based model to output the most semantic similarity languages; then, a random forest model is built which integrates features such as data size, language family, word formation, morpheme overlap, word order, POS tag and syntax similarity together to predict the final target language(s). Experimental results on several datasets show that our method achieves consistent improvements over existing approaches both on language selection and multilingual NMT.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alaux, J., Grave, E., Cuturi, M., Joulin, A.: Unsupervised hyperalignment for multilingual word embeddings. arXiv preprint arXiv:1811.01124 (2018)
Belinkov, Y., Màrquez, L., Sajjad, H., Durrani, N., Dalvi, F., Glass, J.: Evaluating layers of representation in neural machine translation on part-of-speech and semantic tagging tasks. arXiv preprint arXiv:1801.07772 (2018)
Burges, C.J.: From RankNet to LambdaRank to LambdaMart: an overview. Learning 11(23–581), 81 (2010)
Eger, S., Hoenen, A., Mehler, A.: Language classification from bilingual word embedding graphs. arXiv preprint arXiv:1607.05014 (2016)
Hammarström, H.: Linguistic diversity and language evolution. J. Lang. Evol. 1(1), 19–29 (2016)
Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst. (TOIS) 20(4), 422–446 (2002)
Johnson, M., et al.: Google’s multilingual neural machine translation system: enabling zero-shot translation. Trans. Assoc. Comput. Linguist. 5, 339–351 (2017)
Ke, G., et al.: LightGBM: a highly efficient gradient boosting decision tree. Adv. Neural Inf. Process. Syst. 30, 3146–3154 (2017)
Khurana, D., Koli, A., Khatter, K., Singh, S.: Natural language processing: state of the art, current trends and challenges. arXiv preprint arXiv:1708.05148 (2017)
Kim, Y.B.: Universal morphological analysis using structured nearest neighbor prediction (2011)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Lin, Y.H., et al.: Choosing transfer languages for cross-lingual learning. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3125–3135, July 2019
Littell, P., Mortensen, D.R., Lin, K., Kairis, K., Turner, C., Levin, L.: URIEL and lang2vec: representing languages as typological, geographical, and phylogenetic vectors. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, vol. 2, Short Papers, pp. 8–14 (2017)
Naseem, T., Snyder, B., Eisenstein, J., Barzilay, R.: Multilingual part-of-speech tagging: two unsupervised approaches. J. Artif. Intell. Res. 36, 341–385 (2009)
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318 (2002)
Sennrich, R., Haddow, B., Birch, A.: Improving neural machine translation models with monolingual data. arXiv preprint arXiv:1511.06709 (2015)
Song, L., Gildea, D., Zhang, Y., Wang, Z., Su, J.: Semantic neural machine translation using AMR. Trans. Assoc. Comput. Linguist. 7, 19–31 (2019)
Tan, X., Chen, J., He, D., Xia, Y., Qin, T., Liu, T.Y.: Multilingual neural machine translation with language clustering. arXiv preprint arXiv:1908.09324 (2019)
Vaswani, A., et al.: Attention is all you need. arXiv preprint arXiv:1706.03762 (2017)
Wang, Y., Zhou, L., Zhang, J., Zhai, F., Xu, J., Zong, C.: A compact and language-sensitive multilingual translation method. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1213–1223 (2019)
Wu, Y., et al.: Google’s neural machine translation system: bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 (2016)
Acknowledgments
This research was funded by the National Natural Science Foundation of China (No. 61906158).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Mi, C., Zhu, S., Fan, Y., Xie, L. (2021). Incorporating Typological Features into Language Selection for Multilingual Neural Machine Translation. In: U, L.H., Spaniol, M., Sakurai, Y., Chen, J. (eds) Web and Big Data. APWeb-WAIM 2021. Lecture Notes in Computer Science(), vol 12858. Springer, Cham. https://doi.org/10.1007/978-3-030-85896-4_27
Download citation
DOI: https://doi.org/10.1007/978-3-030-85896-4_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-85895-7
Online ISBN: 978-3-030-85896-4
eBook Packages: Computer ScienceComputer Science (R0)