Abstract
Sentence alignment is a crucial task in the process of building parallel corpora. Off-the-shelf tools for sentence alignment generally perform well to this end. However in certain cases, depending on factors such as the sentence structure and the amount of contextual information, the sentence alignment task can be challenging and require further resources that may be difficult to find, such as domain-specific bilingual dictionaries. Although investing in creating additional linguistic resources is frequently the chosen option in these circumstances, leveraging extra-linguistic information such as sentence-level metadata can be an easier alternative to narrow the alignment search space. This paper presents a method designed for the alignment of bilingual survey questionnaires’ texts, which leverages sentence-level metadata annotations. We build eight gold standards in four distinct languages to measure our sentence aligner performance, namely Catalan, French, Portuguese, and Spanish.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
preprocess() is a generic preprocessing function that removes punctuation and tokenizes words in the sentences. heuristic() refers to a generic length-based heuristic to decide the best alignment candidates. merge() is a function that merges two sets of responses based on the values associated with each response.
- 2.
- 3.
References
Brown, P.F., Lai, J.C., Mercer, R.L.: Aligning sentences in parallel corpora. In: 29th Annual Meeting of the Association for Computational Linguistics, pp. 169–176 (1991)
Gale, W.A., Church, K.W., et al.: A program for aligning sentences in bilingual corpora. Comput. Linguist. 19(1), 75–102 (1994)
Luo, S., Ying, H., Yu, S.: Sentence alignment with parallel documents helps biomedical machine translation. arXiv preprint arXiv:2104.08588 (2021)
Niyogi, M., Ghosh, K., Bhattacharya, A.: Learning multilingual embeddings for cross-lingual information retrieval in the presence of topically aligned corpora. arXiv preprint arXiv:1804.04475 (2018)
Philip, J., Siripragada, S., Namboodiri, V.P., Jawahar, C.: Revisiting low resource status of Indian languages in machine translation. In: 8th ACM IKDD CODS and 26th COMAD, pp. 178–187 (2021)
Ramesh, S.H., Sankaranarayanan, K.P.: Neural machine translation for low resource languages using bilingual lexicon induced from comparable corpora. arXiv preprint arXiv:1806.09652 (2018)
Santos, A.: A survey on parallel corpora alignment. In: Proceedings of MI-Star, pp. 117–128 (2011)
Sharma, V., Mittal, N.: Refined stop-words and morphological variants solutions applied to Hindi-English cross-lingual information retrieval. J. Intell. Fuzzy Syst. 36(3), 2219–2227 (2019)
Shi, X., Huang, H., Jian, P., Tang, Y.K.: Improving neural machine translation with sentence alignment learning. Neurocomputing 420, 15–26 (2021)
Sorato, D., Zavala-Rojas, D.: The multilingual corpus of survey questionnaires query interface. In: Proceedings of the 5th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, pp. 43–48 (2021)
Varga, D., Halácsy, P., Kornai, A., Nagy, V., Németh, L., Trón, V.: Parallel corpora for medium density languages. In: Amsterdam Studies in the Theory and History of Linguistic Science Series, vol. 4 pp. 292, 247 (2007)
Zavala Rojas, D., Sorato, D., Hareide, L., Hofland, K.: [MCSQ]: the multilingual corpus of survey questionnaires (2021)
Acknowledgements
This work was developed in the SSHOC project, an EU Horizon 2020 Research and Innovation Programme (2014–2020) under Grant Agreement No. 823782. We thank Elsa Peris for her support to build the gold standards.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sorato, D., Zavala-Rojas, D. (2022). Sentence Alignment of Bilingual Survey Texts Applying a Metadata-Aware Strategy. In: Rosso, P., Basile, V., Martínez, R., Métais, E., Meziane, F. (eds) Natural Language Processing and Information Systems. NLDB 2022. Lecture Notes in Computer Science, vol 13286. Springer, Cham. https://doi.org/10.1007/978-3-031-08473-7_43
Download citation
DOI: https://doi.org/10.1007/978-3-031-08473-7_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-08472-0
Online ISBN: 978-3-031-08473-7
eBook Packages: Computer ScienceComputer Science (R0)