Abstract
Senseval was the first open, community-based evaluation exercise for WordSense Disambiguation programs. It took place in the summer of 1998,with tasks for English, French and Italian. There were participating systems from 23 researchgroups. This special issueis an account of the exercise. In addition to describing the contentsof the volume, this introduction considers how the exercise has shedlight on some general questions about wordsenses and evaluation.
This is a preview of subscription content, log in via an institution to check access.
References
Buckley, C. and C. Cardie. “EMPIRE and SMART Working Together”. Presentation at the DARPA/Tipster 24-Month Meeting, 1998.
Gaizauskas, R. “Evaluation in Language and Speech Technology: Introduction to the Special Issue”. Computer Speech and Language, 12(4) (1998), 249–262.
Gale, W., K. Church and D. Yarowsky. “Estimating Upper and Lower Bounds on the Performance of Word-Sense Disambiguation Programs”. In Proceedings, 30th ACL. 1992, pp. 249–256.
Hirschman, L. “The Evolution of Evaluation: Lessons from the Message Understanding Conferences”. Computer Speech and Language, 12(4) (1998), 281–307.
Ide, N. and J. Véronis. “Introduction to the Special Issue on Word Sense Disambiguation: The State of the Art'. Computational Linguistics, 24(1) (1998), 1–40.
Kilgarriff, A. “Foreground and Background Lexicons and Word Sense Disambiguation for Information Extraction”. In Proc. Workshop on Lexicon Driven Information Extraction. Frascati, Italy, 1997, pp. 51–62.
Kilgarriff, A. “Generative Lexicon Meets Corpus Data: The Case of Non-Standard Word Uses”. In Word Meaning and Creativity. Ed. P. Bouillon and F. Busa, Cambridge: Cambridge University Press, forthcoming, 2000.
Palmer, M., H.T. Dang and J. Rosenzweig. “Sense Tagging the Penn Treebank”. Submitted to the Second Language Resources and Evaluation Conference. Athens, Greece, 2000.
Palmer, M., T. Finin and S. Walters. “Evaluation of Natural Language Processing Systems”. Computational Linguistics, 16(3) (1990), 175–181.
Palmer, M. and Z. Wu. “Verb Semantics for English-Chinese Translation”. Machine Translation, 10, (1995), 59–92.
Resnik, P. and D. Yarowsky. “A Perspective on Word Sense Disambiguation Methods and Their Evaluation”. In Tagging Text with Lexical Semantics: Why, What and How? Ed. M. Light, Washington, 1997, pp. 79–86.
Resnik, P. and D. Yarowsky. “Distinguishing Systems and Distinguishing Senses: New Evaluation Methods for Word Sense Disambiguation”. Natural Language Engineering Journal, to appear.
Sparck Jones, K. and J. Galliers. Evaluating Natural Language Processing Systems: An Analysis and Review. Berlin: Springer-Verlag, 1996.
Sproat, R., M. Ostendorf, and A. Hunt: 1999, “The Need for Increased Speech Synthesis Research”. Report of the 1998 NSF Workshop for Discussing Research Priorities and Evaluation Strategies in Speech Synthesis.
Voorhees, E.M.: 1999, “Natural Language Processing and Information Retrieval”. In Proceedings of Second Summer School on Information Extraction. Springer-Verlag, Lecture Notes in Artificial Intelligence.
Yarowsky, D. “Word-Sense Disambiguation Using Statistical Models of Roget's Categories Trained on Large Corpora”. In COLING 92. Nantes, 1992.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Kilgarriff, A., Palmer, M. Introduction to the Special Issue on SENSEVAL. Computers and the Humanities 34, 1–13 (2000). https://doi.org/10.1023/A:1002619001915
Issue Date:
DOI: https://doi.org/10.1023/A:1002619001915