Abstract
This study evaluates the retrieval effectiveness of English-Chinese (EC) cross-language information retrieval (CLIR) on four common search engines along the dimensions of recall and precision. We formulated a set of simple and complex queries on different topics including queries with translation ambiguity. Three independent bilingual proficient evaluators reviewed a total of 960 returned web pages each to assess document relevance. Findings showed that CLIR effectiveness is poor with average recall and precision values of 0.165 and 0.539 for monolingual EE/CC searches, and 0.078 and 0.282 for cross lingual CE/EC searches. Google outperformed Yahoo! in the experiments, and EC and EE searches returned better results than CE and CC results respectively. As this is the first set CLIR retrieval effectiveness measurements reported in literature, these findings can serve as a benchmark and provide a better understanding of the current CLIR capabilities of Web search engines.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hansen, P., Petrelli, D., Beaulieu, M., Sanderson, M.: User-centred interface design for cross-language information retrieval. In: 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 383–384. IEEE Press, New York (2002)
Kralisch, A., Mandl, T.: Barriers to information access across languages on the internet: network and language effects. In: 39th Annual Hawaii International Conference on Systems Sciences, vol. 3, pp. 4–7 (2006)
Clarke, J.S., Willett, P.: Estimating the recall performance of web search engines. Aslib Proceedings 49(7), 184–189 (1997)
Youssef, M.: Cross language information retrieval: Universal usability in practice. Department of Computer Science, University of Maryland (2001), http://otal.umd.edu/uupractice/clir/#r11 (retrieved May 30, 2011)
Kumar, B.T., Prakash, J.N.: Precision and relative recall of search engines: A comparative study of Google and Yahoo. Singapore Journal of Library & Information Management 38, 124–137 (2009)
Zhang, J., Lin, S.Y.: Multiple language supports in search engines. Online Information Review 31(4), 516–532 (2007)
Ogden, W.C., Cowie, J., Davis, M., Ludovik, E., Nirenburg, S., Molina-Salgado, H., et al.: Keizai: An interactive cross-language text retrieval system. In: Ananiadou, S., Hayashi, Y., Jacquemin, C., Leong, M.K., Chen, H.H. (eds.) Workshop on Machine Translation for Cross Language Information (1999), http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.37.5775&rep=rep1&type=pdf (retrieved May 30, 2011)
Ogden, W.C., Davis, M.W.: Improving cross-language text retrieval with human interactions. In: 33rd Annual Hawaii International Conference on System Sciences, vol. (3), p. 3044 (2000)
Capstick, J., Diagne, A.K., Erbach, G., Uszkoreit, H., Leisenberg, A., Leisenberg, M.: A system for supporting cross-lingual information retrieval. Information Processing & Management 36(2), 275–289 (2000)
Peñas, A., Gonzalo, J., Verdejo, F., Lenguajes, D., Informáticos, S.: Cross-language information access through phrase browsing. In: Applications of Natural Language to Information Systems. LNI, pp. 121–130 (2001)
Airio, E.: Who benefits from CLIR in web retrieval. Journal of Documentation 64(5), 760–778 (2007)
Gey, F., Kando, N., Peters, C.: Cross-language information retrieval: the way ahead. Information Processing and Management 41, 415–431 (2005)
Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval, 2nd edn. Addison-Wesley, Reading (2010)
Manning, C.D., Raghavan, P., Schutze, H.: Introduction to information retrieval. Cambridge University Press, Cambridge (2008)
Foo, S., Li, H.: Chinese word segmentation and its effect on information retrieval. Information Processing and Management 40(1), 161–190 (2004)
Shafi, S., Rather, R.: Precision and recall of five search engines for retrieval of scholarly information in the field of biotechnology. Webology 2(2), Article 15 (2005), http://www.webology.org/2005/v2n2/a12.html (retrieved May 30, 2011)
Croft, B., Metzler, D., Strohman, T.: Search engines: Information retrieval in practice. Addison Wesley, Reading (2009)
Chinchor, N., Dungca, G.: Four scores and seven years ago: The scoring method for MUC-6. In: Proc. MUC-6 Conference, Columbia, MD (1995)
Makhoul, J., Kubala, F., Schwartz, R., Weischedel, R.: Performance measures for information extractors. In: Proc. DARPA Broadcast News Workshop, Herndon, VA, USA (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Foo, S. (2011). Retrieval Effectiveness of Cross Language Information Retrieval Search Engines. In: Xing, C., Crestani, F., Rauber, A. (eds) Digital Libraries: For Cultural Heritage, Knowledge Dissemination, and Future Creation. ICADL 2011. Lecture Notes in Computer Science, vol 7008. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24826-9_37
Download citation
DOI: https://doi.org/10.1007/978-3-642-24826-9_37
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24825-2
Online ISBN: 978-3-642-24826-9
eBook Packages: Computer ScienceComputer Science (R0)