Abstract
Textual entailment (TE) relation determines whether a text can be inferred from another. Given two texts, one is called the “Text” denoted as T and the other one is called “Hypothesis” denoted as H, the process of textual entailment is to decide whether or not the meaning of H can be logically inferred from the meaning of T. Different semantic, lexical and vector based similarity metrics are used as features for different machine learning classifiers to take the entailment decision in this study. We also considered two machine translation evaluation metrics, namely BLEU and METEOR, as similarity metrics for this task. We carried out the experiments on the datasets released in the shared tasks on textual entailment organized in RTE-1, RTE-2, and RTE-3. We experimented with different feature combinations. Best accuracies were obtained on different feature combinations by different classifiers. The best classification accuracies obtained by our system on the RTE-1, RTE-2 and RTE-3 dataset are 55.91%, 58.88% and 63.38% respectively. MT evaluation metrics based feature alone produced the best classification accuracies of 53.9%, 59.3%, and 62.8% on the RTE-1, RTE-2, and RTE-3 datasets respectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: BLEU: a Method for Automatic Evaluation of Machine Translation. In: Proceedings of 40th Annual Meeting of the Association for Computational Linguistics (ACL), Philadelphia, PA, pp. 311–318 (2002)
Banerjee, S., Lavie, A.: METEOR:An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, Ann Arbor, Michigan, pp. 65–72 (2005)
Dagan, I., Glickman, O.: Probabilistic textual entailment: Generic applied modeling of language variability. In: Monz, C., de Rijke, M. (eds.) PASCAL Workshop on Text Understanding and Mining 2001. Light-Weight Entailment Checking for Computational Semantic. Proceedings ICoS-3 (2004)
Lavie, A., Agarwal, A.: METEOR: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments. In: Proceedings of the Second ACL Workshop on Statistical Machine Translation, Prague, Czech Republic, pp. 228–231 (2007)
Li, B., Irwin, J., Garcia, E.V., Ram, A.: Machine Learning Based Semantic Inference: Experiments and Observations at RTE-3. In: Proceedings of the Third Challenge Workshop Recognising Textual Entailment, Prague, Czech Republic, pp. 159–164 (2007)
Malakasiotis, P., Androutsopoulos, I.: Learning Textual Entailment using SVMs and String Similarity Measures. In: Proceedings of the Third Challenge Workshop Recognising Textual Entailment, Prague, Czech Republic, pp. 42–47 (2007)
Ferrés, D., Rodrí́guez, H.: Machine Learning with Semantic-Based Distances Between Sentences for Textual Entailment. In: Proceedings of the Third Challenge Workshop Recognising Textual Entailment, Prague, Czech Republic, pp. 60–65 (2007)
Pakray, P., Bandyopadhyay, S., Gelbukh, A.: Binary-class and Multiclass based Textual Entailment System. In: Proceedings of the 10th NTCIR Conference on Evaluation of Information Access Technologies, Japan, National Institute of Informatics (2013)
Perez, D., Alfonseca, E.: Application of the Bleu algorithm for recognising textual entailments. In: Proceedings of the First Challenge Workshop Recognising Textual Entailment, pp. 9–12 (2005)
Volokh, A., Neumann, G.: Using MT-based metrics for RTE. In: The Fourth Text Analysis Conference. NIST (2011)
Ríos Gaona, M.A., Gelbukh, A., Bandyopadhyay, S.: Recognizing textual entailment using a machine learning approach. In: Sidorov, G., Hernández Aguirre, A., Reyes García, C.A. (eds.) MICAI 2010, Part II. LNCS, vol. 6438, pp. 177–185. Springer, Heidelberg (2010)
Hickl, et al.: Recognizing textual entailment with LCC’s GROUNDHOG system. In: Proceedings of the Second PASCAL Challenges Workshop (2006)
Hickl, A., Bensley, J.: A discourse commitment-based framework for recognizing textual entailment. In: Association for Computational Linguistics, pp. 171–176 (2007)
Vidal, E.: Finite-State Speech-to-Speech Translation. In: Proceedings of the International Conference on Acoustics, Speech and Signal Processing, Munich, Germany, pp. 111–114 (1997)
Tillmann, C., Vogel, S., Ney, H., Sawaf, H., Zubiaga, A.: Accelerated DP based Search for Statistical Translation. In: Proceedings of the 5th European Conference on Speech Communication and Technology, Rhodes, Greece, pp. 2667–2670 (1997)
Doddington, G.: Automatic evaluation of machine translation quality using n-gram cooccurrence statistics. In: Proceedings of the Second International Conference on Human Language Technology Research, pp. 138–145. Morgan Kaufmann Publishers Inc. (2002)
Snover, M., Dorr, B., Schwartz, R., Micciulla, L., Makhoul, J.: A study of translation edit rate with targeted human annotation. In: Proceedings of Association for Machine Translation in the Americas, Cambridge, Massachusetts, USA, pp. 223–231 (2006)
Turian, J.P., Shen, L., Dan Melamed, I.: Evaluation of Machine Translation and Its Evaluation. In: Proceedings of MT Summit 2003, New Orleans, Luisiana, pp. 386–393 (2003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Saikh, T., Naskar, S.K., Giri, C., Bandyopadhyay, S. (2015). Textual Entailment Using Different Similarity Metrics. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2015. Lecture Notes in Computer Science(), vol 9041. Springer, Cham. https://doi.org/10.1007/978-3-319-18111-0_37
Download citation
DOI: https://doi.org/10.1007/978-3-319-18111-0_37
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-18110-3
Online ISBN: 978-3-319-18111-0
eBook Packages: Computer ScienceComputer Science (R0)