Skip to main content
SpringerLink
Log in

Bridging the Resource Gap in Cross-Lingual Embedding Space

  • Conference paper
  • First Online:
Modelling and Development of Intelligent Systems (MDIS 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1761))

  • 265 Accesses

Abstract

The mapping based methods for inducing a cross-lingual embedding space involves learning a linear mapping from the individual monolingual embedding spaces to a shared semantic space where English is often chosen as the hub language. Such methods are based on the orthogonal assumption. Resource limitations and typological distance from English often result in a deviation from this assumption and subsequently poor performance for the low-resource languages. In this research, we will present a method for identifying optimal bridge languages to achieve better mapping for the low-resource languages in the cross-lingual embedding space. We also report Bilingual Induction Task (BLI) performances for the shared semantic space achieved using different cross-lingual signals.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ruder, S., Vulić, I., Søgaard, A.: A survey of cross-lingual word embedding models. J. Artif. Intell. Res. 65, 569–631 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  2. Mikolov, T., Le, Q.V., Sutskever, I.: Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168 (2013)

  3. Xing, C., Wang, D., Liu, C., Lin, Y.: Normalized word embedding and orthogonal transform for bilingual word translation. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1006–1011 (2015)

    Google Scholar 

  4. Artetxe, M., Labaka, G., Agirre, E.: Learning principled bilingual mappings of word embeddings while preserving monolingual invariance. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2289–2294 (2016)

    Google Scholar 

  5. Zhang, M., Liu, Y., Luan, H., Sun, M.: Earth mover’s distance minimization for unsupervised bilingual lexicon induction. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 1934–1945 (2017)

    Google Scholar 

  6. Søgaard, A., Ruder, S., Vulić, I.: On the limitations of unsupervised bilingual dictionary induction. arXiv preprint arXiv:1805.03620 (2018)

  7. Ahia, O., Kreutzer, J., Hooker, S.: The low-resource double bind: an empirical study of pruning for low-resource machine translation. arXiv preprint arXiv:2110.03036 (2021)

  8. Oughton, E.: Policy options for digital infrastructure strategies: a simulation model for broadband universal service in Africa. arXiv preprint arXiv:2102.03561 (2021)

  9. Joulin, A., Grave, E., Bojanowski, P., Douze, M., Jégou, H., Mikolov, T.: FastText.zip: compressing text classification models. arXiv preprint arXiv:1612.03651 (2016)

  10. Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017)

    Article  Google Scholar 

  11. Conneau, A., Lample, G., Ranzato, M.A., Denoyer, L., Jégou, H.: Word translation without parallel data. arXiv preprint arXiv:1710.04087 (2017)

  12. Comrie, B. (ed.): The World’s Major Languages. Routledge, London (1987)

    Google Scholar 

  13. Vulić, I., Ruder, S., Søgaard, A.: Are all good word vector spaces isomorphic?. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 3178–3192 (2020)

    Google Scholar 

  14. Zadeh, L. A.: Fuzzy sets. In: Fuzzy Sets, Fuzzy Logic, and Fuzzy Systems: Selected Papers by Lotfi A Zadeh, pp. 394–432 (1996)

    Google Scholar 

  15. Dunn, J. C.: A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters, pp. 32–57 (1973)

    Google Scholar 

  16. Bezdek, J.C., Ehrlich, R., Full, W.: FCM: the fuzzy c-means clustering algorithm. Comput. Geosci. 10(2–3), 191–203 (1984)

    Article  Google Scholar 

  17. Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithms. Springer, Heidelberg (2013)

    MATH  Google Scholar 

  18. Anastasopoulos, A., Neubig, G.: Should all cross-lingual embeddings speak English?. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 8658–8679 (2020)

    Google Scholar 

  19. Smith, S.L., Turban, D.H., Hamblin, S., Hammerla, N.Y.: Offline bilingual word vectors, orthogonal transformations and the inverted softmax. arXiv preprint arXiv:1702.03859 (2017)

  20. Nakashole, N., Flauger, R.: Knowledge distillation for bilingual dictionary induction. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2497–2506 (2017)

    Google Scholar 

  21. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. Vancouver (2018)

  22. Conneau, A., et al.: Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116 (2019)

  23. Wu, S., Dredze, M.: Are all languages created equal in multilingual BERT? arXiv preprint arXiv:2005.09093 (2020)

Download references

Author information

Authors and Affiliations

  1. The College of Wooster, Wooster, OH, 44691, USA

    Kowshik Bhowmik

  2. University of Cincinnati, Cincinnati, OH, 45220, USA

    Kowshik Bhowmik & Anca Ralescu

Authors
  1. Kowshik Bhowmik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anca Ralescu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kowshik Bhowmik .

Editor information

Editors and Affiliations

  1. Lucian Blaga University of Sibiu, Sibiu, Romania

    Dana Simian

  2. Lucian Blaga University of Sibiu, Sibiu, Romania

    Laura Florentina Stoica

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bhowmik, K., Ralescu, A. (2023). Bridging the Resource Gap in Cross-Lingual Embedding Space. In: Simian, D., Stoica, L.F. (eds) Modelling and Development of Intelligent Systems. MDIS 2022. Communications in Computer and Information Science, vol 1761. Springer, Cham. https://doi.org/10.1007/978-3-031-27034-5_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-27034-5_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-27033-8

  • Online ISBN: 978-3-031-27034-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation