Abstract
This paper presents a first attempt of an application-driven evaluation exercise of WSD. We used a CLIR testbed from the Cross Lingual Evaluation Forum. The expansion, indexing and retrieval strategies where fixed by the organizers. The participants had to return both the topics and documents tagged with WordNet 1.6 word senses. The organization provided training data in the form of a pre-processed Semcor which could be readily used by participants. The task had two participants, and the organizer also provided an in-house WSD system for comparison. The results do not improve over the baseline, which is not surprising given the simplistic CLIR strategy used. Other than that the exercise was succesful, and provides the foundation for more ambitious follow-up exercises where the participants would be able to build up on the WSD results already available.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Agirre, E., de Lacalle, O.L., Martinez, D.: Exploring feature set combinations for WSD. In: Proc. of the SEPLN (2006)
Atserias, J., Villarejo, L., Rigau, G., Agirre, E., Carroll, J., Magnini, B., Vossen, P.: The MEANING Multilingual Central Repository. In: Proceedings of the 2.nd Global WordNet Conference, GWC 2004, pp. 23–30. Masaryk University, Brno (2004)
Carpuat, M., Wu, D.: Improving Statistical Machine Translation using Word Sense Disambiguation. In: Proc. of EMNLP-CoNLL, Prague (2007)
Chan, Y.S., Ng, H.T., Chiang, D.: Word Sense Disambiguation Improves Statistical Machine Translation. In: Proc. of ACL, Prague (2007)
Clough, P., Stevenson, M.: Cross-language information retrieval using EuroWordNet and word sense disambiguation. In: Proc. of ECIR, Sunderland (2004)
Daude, J., Padro, L., Rigau, G.: Mapping WordNets Using Structural Information. In: Proc. of ACL, Hong Kong (2000)
Gonzalo, J., Penas, A., Verdejo, F.: Lexical ambiguity and information retrieval revisited. In: Proc. of EMNLP, Maryland (1999)
Harman, D.: Beyond English. In: Voorhees, E.M., Harman, D. (eds.) TREC: Experiment and Evaluation in Information Retrieval, pp. 153–181. MIT press, Cambridge (2005)
Hiemstra, D., Kraaij, W.: Twenty-One in ad-hoc and CLIR. In: Voorhees, E.M., Harman, D.K. (eds.) Proc. of TREC-7, pp. 500–540. NIST Special Publication (1998)
Krovetz, B.: Homonymy and polysemy in information retrieval. In: Proc. of EACL, pp. 72–79, Madrid (1997)
Krovetz, B.: On the importance of word sense disambiguation for information retrieval. In: Proc. of LREC Workshop on Creating and Using Semantics for Information Retrieval and Filtering, Las Palmas (2002)
Pasca, M., Harabagiu, S.: High performance question answering. In: Proc. of ACM SIGIR, New Orleans (2001)
Resnik, P.: Word sense disambiguation in nlp applications. In: Agirre, E., Edmonds, P. (eds.) Word Sense Disambiguation: Algorithms and Applications. Springer, Heidelberg (2006)
Voorhees, E.M.: Natural language processing and information retrieval. In: Pazienza, M.T. (ed.) Information Extraction: Towards Scalable, Adaptable Systems. Springer, Heidelberg (1999)
Vossen, P., Rigau, G., Alegria, I., Agirre, E., Farwell, D., Fuentes, M.: Meaningful results for Information Retrieval in the MEANING project. In: Proc. of the 3rd Global Wordnet Conference, pp. 22–26 (2006)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Agirre, E., Lopez de Lacalle, O., Magnini, B., Otegi, A., Rigau, G., Vossen, P. (2008). SemEval-2007 Task 01: Evaluating WSD on Cross-Language Information Retrieval. In: Peters, C., et al. Advances in Multilingual and Multimodal Information Retrieval. CLEF 2007. Lecture Notes in Computer Science, vol 5152. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-85760-0_116
Download citation
DOI: https://doi.org/10.1007/978-3-540-85760-0_116
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-85759-4
Online ISBN: 978-3-540-85760-0
eBook Packages: Computer ScienceComputer Science (R0)