Abstract
We describe the system we used at the PASCAL-2005 Recognizing Textual Entailment Challenge. Our method for recognizing entailment is based on calculating “directed” sentence similarity: checking the directed “semantic” word overlap between the text and the hypothesis. We use frequency-based term weighting in combination with two different word similarity measures.
Although one version of the system shows significant improvement over randomly guessing decisions (with an accuracy score of 57.3), we show that this is only due to a subset of the data that can be equally well handled by simple word overlap. Furthermore, we give an in-depth analysis of the system and the data of the challenge.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. ACM Press, New York (1999)
Bayer, S., Burger, J., Ferro, L., Henderson, J., Yeh, A.: MITRE’s submission to the EU Pascal RTE challenge. In: Dagan, et al (2005b)
Budanitsky, A., Hirst, G.: Semantic distance in wordnet: An experimental, application-oriented evaluation of five measures. In: Workshop on WordNet and Other Lexical Resources, Second meeting of the North American Chapter of the Association for Computational Linguistics (2001)
Dagan, I., Glickman, O., Magnini, B.: The PASCAL Recognizing Textual Entailment Challenge. In: Dagan, et al (2005b)
Dagan, I., Glickman, O., Magnini, B. (eds.): PASCAL. Proceedings of the first challenge workshop Recognizing Textual Entailment (2005b)
Glickman, O., Dagan, I., Koppel, M.: Web based probabilistic textual entailment. In: Dagan, et al (2005b)
Hirst, G., St-Onge, D.: Lexical chains as representation of context for the detection and correction of malapropisms. In: Christiane, F. (ed.) WordNet: An electronic lexical database. The MIT Press, Cambridge (1998)
Jijkoun, V., de Rijke, M.: Recognizing textual entailment using lexical similarity. In: Dagan, et al (2005b)
Lin, D.: An information-theoretic definition of similarity. In: Proceedings of International Conference on Machine Learning (1998)
Monz, C., de Rijke, M.: Light-weight entailment checking for computational semantics. In: Proceedings of the Workshop on Inference in Computational Semantics, ICoS-3 (2001)
Pascal. Pascal Recognising Textual Entailment Challenge (2005), http://www.pascalnetwork.org/Challenges/RTE/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jijkoun, V., de Rijke, M. (2006). Recognizing Textual Entailment: Is Word Similarity Enough?. In: Quiñonero-Candela, J., Dagan, I., Magnini, B., d’Alché-Buc, F. (eds) Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment. MLCW 2005. Lecture Notes in Computer Science(), vol 3944. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11736790_25
Download citation
DOI: https://doi.org/10.1007/11736790_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-33427-9
Online ISBN: 978-3-540-33428-6
eBook Packages: Computer ScienceComputer Science (R0)