Abstract
Like full-text translation, cross-language information retrieval (CLIR) is a task that requires some form of knowledge transfer across languages. Although robust translation resources are critical for constructing high quality translation tools, manually constructed resources are limited both in their coverage and in their adaptability to a wide range of applications. Automatic mining of translingual knowledge makes it possible to complement hand-curated resources. This chapter describes a growing body of work that seeks to mine translingual knowledge from text data, in particular, data found on the Web. We review a number of mining and filtering strategies, and consider them in the context of statistical machine translation, showing that these techniques can be effective in collecting large quantities of translingual knowledge necessary for CLIR.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Adafre, S.F. and de Rijke, M. (2006). Finding similar sentences acorss multiple languages in Wikipedia. 11 th Conference of the European Chapter of the Association for Computational Linguistics, pp. 62–69.
Ballesteros, L. and Croft, W. (1997). Phrasal translation and query expansion techniques for cross-language information retrieval. In Proceedings of SIGIR Conf. pp. 84-91.
Berger, A. and Lafferty, J. (1999). Information retrieval as statistical translation. In Proceedings of SIGIR Conf., pp. 222-229.
Braschler, M., and Schäuble, P. (1998). Multilingual information retrieval based on document alignment techniques. ECDL ’98: Proceedings of the Second European Conference on Research and Advanced Technology for Digital Libraries, pp. 183–197.
Braschler, M., and Schäuble, P. (2001). Experiments with the Eurospider Retrieval System for CLEF 2000, in Proceedings of CLEF Conference. pp. 140-148.
Brown, P., Della Pietra, S., Della Pietra, V., and Mercer, R. (1993). The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2), pp. 263-311.
Cao, G., Gao, J., Nie, J.Y. (2007) A system to mine large-scale bilingual dictionaries from monolingual Web pages, MT Summit, pp. 57-64.
Carbonell, J.G, Yang, Y, Frederking, R.E., Brown, R., Geng, Y. and Lee, D. (1997) Translingual information retrieval: A comparative evaluation. In: Proceedings of the International Joint Conference on Arti?cial Intelligence (IJCAI ’97).
Chiang, D., (2005) A Hierarchical Phrase-Based Model for Statistical Machine Translation. ACL.
Chen, J., Nie, J.Y., (2000) Automatic construction of parallel English-Chinese corpus for cross-language information retrieval. ANLP pp. 21-28
Chen, H.H., Lin, W.C. and Yang, C.H. (2006). Translation-Transliterating Named Entities for Multilingual Information Access. Journal of the American Society for Information Science and Technology, 57(5):645-659
Cheng, P., Teng, J., Chen, R., Wang, J., Lu, W., and Chien, L. (2004). Translating Unknown Queries with Web Corpora for Cross- Language Information Retrieval. In Proceedings of SIGIR Conf., pp.162-169.
Dumais, S. T., Letsche, T. A., Littman, M. L. and Landauer, T. K. (1997) Automatic cross-language retrieval using Latent Semantic Indexing. AAAI Spring Symposuim on Cross-Language Text and Speech Retrieval, March 1997.
Franz, M., McCarley, J.S. and Koukos, S. (1999) Ad hoc and multilingual information retrieval at IBM. Proceedings of the Seventh Text Retrieval Conference (TREC-7), pp. 157–168.
Fung, P. (1995). A Pattern Matching Method for Finding Noun and Proper Noun Translations from Noisy Parallel Corpora. Proceedings of the Association for Computational Linguistics, pp. 236-243.
Pascale Fung and Yuen Yee Lo. 1998. An IR approach for translating new words from nonparallel, comparable texts. Proceedings of COLING-ACL98, pp. 414– 420.
Fung, P. and McKeown, K. (1997) Finding terminology translations from non-parallel corpora. In: The 5th Annual Workshop on Very Large Corpora.
Fung, P. and Cheung, P. (2004) Multilevel boot-strapping for extracting parallel sentences from a quasi parallel corpus. Conference on Empirical Methods in Natural Language Processing (EMNLP 04), pp. 1051–1057.
Gale, W. A., Church K. W. 1993. A Program for Aligning Sentences in Bilingual Corpora. Computational Linguistics, 19(3): 75-102.
Galley, M., Hopkins, M., Knight, K., Marcu, D., (2004) What’s in a translation rule? HLT-NAACL, pp. 273-280
Pablo Gamallo Otero, Isaac Gonzalez Lopez, (2009) Wikipedia as Multilingual Source of Comparable Corpora, Proceedings of the 3rd Workshop on Building and Using Comparable Corpora, LREC 2010, pp. 21–25
Gao, J., Nie, J.Y., Xun, E., Zhang, J., Zhou, M., and Huang, C. (2001). Improving query translation for cross-language information retrieval using statistical models. In Proceedings of SIGIR Conf., pp. 96-104.
Gao, J., Zhou, M., Nie, J.Y., He, H., Chen, W. (2002) Resolving query translation ambiguity using a decaying co-occurrence model and syntactic dependence relations. SIGIR, pp. 183-190
Gao, J., Nie, J.Y. (2006) Study of Statistical Models for Query Translation: Finding a Good Unit of Translation. SIGIR, pp 194- 201, 2006.
Gao, J., He, X., Nie. J.Y. (2010) Clickthrough-based translation models for web search: from word models to phrase models. CIKM, pp 1139-1148, 2010.
Hong, Gumwon, Li, Chi-Ho, Zhou, Ming and Rim, Hae-Chang (2010) An Empirical Study on Web Mining of Parallel Data, COLING, pp. 474–482.
Huang, Degen, Zhao, Lian, Li, Lishuang Yu, Haitao (2010) Mining Large-scale Comparable Corpora from Chinese-English News Collections, COLING, pp. 472-480.
Huang, F., Zhang, Y., and Vogel, S. (2005). Mining Key Phrase Translations from Web Corpora. In Proceedings of HLT-EMNLP Conf., pp. 483-490.
Jeon, J. Lavrenko, V. and Manmatha, R. (2003) Automatic Image Annotation and Retrieval using Cross-Media Relevance Models, SIGIR, pp. 119-126.
Jeong, K.S., Myaeng, S.H., Lee, J.S, and Choi, K.S., (1999) Automatic identification and back-transliteration of foreign words for information retrieval, Information Processing and Management, 35(4), pp. 523-540.
Ji, Heng (2009) Mining Name Translations from Comparable Corpora by Creating Bilingual Information Networks, Proceedings of the 2 nd Workshop on Building and Using Comparable Corpora, ACL-IJCNLP 2009, pages34–37.
Koehn, P., Och, F.J., Marcus, D., (2003) Statistical phrase-based translation, In Proceedings of HLT-NAACL, pp. 48-54.
Koehn, P. (2009) Statistical Machine Translation. Cambridge University Press.
Kraaij, W., Nie, J.Y., and Simard, M. (2003). Embedding Web-Based Statistical Translation Models in Cross-Language Information Retrieval. Computational Linguistics, 29(3): 381-420.
Kumano, T. and Tanaka, H., Tokunaga, T. (2007) Extracting phrasal alignments from comparable corpora by using joint probability SMT model. 11th International Conference on Theoretical and Methodological Issues in Machine Translation (TMI’07).
Kuo, J.S., Li, H., and Yang Y.K (2006). Learning Transliteration Lexicon from the Web. In the Proceedings of COLING/ACL, pp.1129-1136
Lam, W., Chan, S.K., and Huang, R. (2007). Named Entity Translation Matching and Learning: With Application for Mining Unseen Translations. ACM Transactions on Information Systems, 25(1), pp.
Liu, Y., Jin R. and Chai, Joyce Y. (2005). A maximum coherence model for dictionary-based cross-language information retrieval, In Proceedings of SIGIR conf., pp. 536-543.
Lu, W. Chien, L.F. and Lee, H. (2004). Anchor Text Mining for Translation of Web Queries: A Transitive Translation Approach. ACM Transactions on Information Systems, Vol.22, pp. 242-269.
Ma, X. and Liberman, M., (1999). Bits: A Method for Bilingual Text Search over the Web. Proceedings of Machine Translation Summit VII.
Munteanu, D. S., Marcu, D. (2005) Improving Machine Translation Performance by Exploiting Non-Parallel Corpora. 2005. Computational Linguistics. 31(4). pp: 477-504.
Munteanu, D. S. and Marcu D. (2006). Extracting parallel subsentential fragments from non-parallel corpora. ACL, pp. 81–88.
Nagata, M., Saito, T., and Suzuki, K. (2001). Using the web as a bilingual dictionary. In Proceedings of the Workshop on Data-Driven Methods in Machine Translation (with ACL Conf.), pp. 1-8.
Nie, J.Y., Cai, J. (2001) Filtering parallel corpora of web pages, IEEE symposium on NLP and Knowledge Engineering, pp. 453-458.
Nie, J.Y., Simard, M., Isabelle, P., Durand, R. (1999) Cross-Language Information Retrieval based on Parallel Texts and Automatic Mining of Parallel Texts in the Web, In Proceedings of SIGIR Conf., pp. 74-81
Och, F., and Ney, H. (2002) Discriminative Training and Maximum Entropy Models for Statistical Machine Translation. ACL, pp. 295-302
Och, F. (2003). Minimum error rate training in statistical machine translation. In Proceedings of ACL. pp. 160-67
Oumohmed, A.I., Mignotte, M., Nie, J.Y. (2005) Semantic-Based Cross-Media Image Retrieval, Pattern Recognition and Image Analysis: Third International Conference on Advances in Pattern Recognition (ICAPR), LNCS 3687, pp. 414-423.
Potthast, M., Stein, B., Anderka, M. (2008) A Wikipedia-based Multilingual Retrieval Model. ECIR, LNCS 4956, pp. 522–530.
Qu, Y., Grefenstette, G., and Evans, D. A. (2003). Automatic transliteration for Japanese-to-English text retrieval. In Proceedings of SIGIR Conference, pp. 353-360.
Rapp, R. (1995). Identifying Word Translations in Non-Parallel Texts. Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics, pp. 320-322.
Resnik, P., (1999) Mining the Web for Bilingual Text, 37th Annual Meeting of the Association for Computational Linguistics (ACL’99).
Resnik P. and Smith. N.A. (2003) The Web as a Parallel Corpus, Computational Linguistics, 29(3), pp. 349-380, September 2003.
Sheridan, P. and Ballerini, J. P. (1996). Experiments in multilingual information retrieval using the SPIDER system. In Proceedings of SIGIR Conf., pp. 58-65.
Schönhofen, P., Benczúr, A., Bíró, I., Csalogány, K. (2007) Performing cross-language retrieval with Wikipedia, CLEF-2007 (http://www.clefcampaign.org/2007/working notes/schonhofenCLEF2007.pdf)
Shi, L., Niu, C., Zhou, M., and Gao, J. (2006) A DOM Tree Alignment Model for Mining Parallel Data from the Web, ACL, pp. 489-496.
Smith, J. R., Quirk, C., and Toutanova, K. (2010) Extracting parallel sentences from comparable corpora using document level alignment. HLT, pp. 403–411
Sproat, R., Tao, T., Zhai, C. (2006) Named Entity Transliteration with Comparable Corpora. In Proceedings of ACL.
Tuomas Talvensaari, Jorma Laurikkala, Kalervo Järvelin, Martti Juhola (2006) A study on automatic creation of a comparable document collection in cross-language information retrieval, Journal of Documentation, Vol. 62 No. 3, pp. 372-387
Tuomas Talvensaari, Jorma Laurikkala, Kalervo Järvelin, Martti Juhola, and Heikki Keskustalo (2007). Creating and exploiting a comparable corpus in cross-language information retrieval. ACM Trans. Inf. Syst. 25, 1, Article 4.
Utiyama M. and Isahara, H. (2003) Reliable Measures for Aligning Japanese-English News Articles and Sentences. ACL, pp. 72–79.
Jinxi Xu, W. Bruce Croft (1996) Query Expansion Using Local and Global Document Analysis. SIGIR, pp. 4-11
Yang, Christopher C., and Kar Wing Li. 2003. Automatic construction of English/Chinese parallel corpora. Journal of the American Society for Information Science and Technology, 54(8), pp. 730–742.
Zhang, Y. and Vines, P. (2004). Using the Web for Automated Translation Extraction in Cross-Language Information Retrieval. In Proceedings of SIGIR Conf., pp.162-169.
Zhang, Y., Huang, F., Vogel, S. (2005) Mining Translations of OOV Terms from the Web through Cross-lingual Query Expansion, SIGIR, pp. 669-670.
Zhao, B., and Vogel, S. (2002). Adaptive Parallel Sentences Mining from Web Bilingual News Collection. In Proceedings of IEEE international conference on data mining, pages 745-750.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Nie, JY., Gao, J., Cao, G. (2012). Translingual Mining from Text Data. In: Aggarwal, C., Zhai, C. (eds) Mining Text Data. Springer, Boston, MA. https://doi.org/10.1007/978-1-4614-3223-4_10
Download citation
DOI: https://doi.org/10.1007/978-1-4614-3223-4_10
Published:
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4614-3222-7
Online ISBN: 978-1-4614-3223-4
eBook Packages: Computer ScienceComputer Science (R0)