Skip to main content
Springer Nature Link
Log in

Recognizing Textual Entailment Using Weighted Dependency Relations

  • Conference paper
  • First Online:
Computational Linguistics and Intelligent Text Processing (CICLing 2018)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13397))

  • 280 Accesses

Abstract

In this paper, we describe a hybrid approach for Recognizing Textual Entailment (RTE) that makes use of dependency parsing and semantic similarity measures. Dependency triplet matching is performed between dependency parsed Text (T) and Hypothesis (H). In case of dependency relation match, we also consider partial matching and semantic similarity between the associated words is calculated with the help of various semantic similarity measures. Importance of various dependency relations with respect to the TE task is computed in terms of their information gain and the dependency relations are weighted accordingly. This paper reports our experiments carried out on the RTE-1, RTE-2 and RTE-3 benchmark datasets using three approaches namely greedy approach, exhaustive search and greedy approach with weighted dependency relations. Experimental results show that weighted dependency relations significantly improve TE performance over the baseline.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://nlp.stanford.edu/software/stanford-dependencies.html.

  2. 2.

    http://pascallin.ecs.soton.ac.uk/Challenges/.

  3. 3.

    http://www.nist.gov/tac/tracks/index.html.

  4. 4.

    http://semeval2.fbk.eu/semeval2.php.

  5. 5.

    http://research.nii.ac.jp/ntcir/ntcir-9/.

References

  1. Tatar, D., Serban, G., Mihis, A.D., Mihalcea, R., et al.: Textual entailment as a directional relation. J. Res. Pract. Inf. Technol. 41(1), 53 (2009)

    Google Scholar 

  2. Hutchins, W.J., Somers, H.L.: An Introduction to Machine Translation. Academic Press, London (1992)

    Google Scholar 

  3. Radev, D.R., McKeown, K.R.: Generating natural language summaries from multiple on-line sources. Comput. Linguist. 24(3), 470–500 (1998)

    Google Scholar 

  4. Green, Jr., B.F., Wolf, A.K., Chomsky, C., Laughery, K.: Baseball: an Automatic Question-Answerer. In: Papers Presented at the 9–11 May 1961, Western Joint IRE-AIEE-ACM Computer Conference. IRE-AIEE-ACM 1961 (Western), New York, USA, ACM, pp. 219–224 (1961)

    Google Scholar 

  5. Socher, R., Huang, E.H., Pennington, J., Ng, A.Y., Manning, C.D.: Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In: Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011. Proceedings of a meeting held 12–14 December 2011, Granada, Spain, pp. 801–809 (2011)

    Google Scholar 

  6. Lambert, P., Blanchard, J., Guillet, F., Kuntz, P., Suignard, P.: Novelty detection in text streams - a survey. In: European Conference on Data Analysis. Proceedings of the European Conference on Data Analysis, Luxembourg, Luxembourg (2013)

    Google Scholar 

  7. Dagan, I., Glickman, O., Magnini, B.: The PASCAL recognising textual entailment challenge. In: Quiñonero-Candela, J., Dagan, I., Magnini, B., d’Alché-Buc, F. (eds.) MLCW 2005. LNCS (LNAI), vol. 3944, pp. 177–190. Springer, Heidelberg (2006). https://doi.org/10.1007/11736790_9

    Chapter  Google Scholar 

  8. Vanderwende, L., Dolan, W.B.: What syntax can contribute in the entailment task. In: Quiñonero-Candela, J., Dagan, I., Magnini, B., d’Alché-Buc, F. (eds.) MLCW 2005. LNCS (LNAI), vol. 3944, pp. 205–216. Springer, Heidelberg (2006). https://doi.org/10.1007/11736790_11

    Chapter  Google Scholar 

  9. Burchardt, A., Reiter, N., Thater, S., Frank, A.: A semantic approach to textual entailment: system evaluation and task analysis. In: Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing. RTE 2007, pp. 10–15. Stroudsburg, PA, USA, Association for Computational Linguistics (2007)

    Google Scholar 

  10. Bowman, S.R., Angeli, G., Potts, C., Manning, C.D.: Learning natural language inference from a large annotated corpus. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 632–642. Stroudsburg, PA, Association for Computational Linguistics (2015)

    Google Scholar 

  11. Rocktäschel, T., Grefenstette, E., Hermann, K.M., Kocisky, T., Blunsom, P.: Reasoning about entailment with neural attention. In: International Conference on Learning Representations (ICLR) (2016)

    Google Scholar 

  12. Wang, S., Jiang, J.: Learning natural language inference with LSTM. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, California, Association for Computational Linguistics, pp. 1442–1451 (June 2016)

    Google Scholar 

  13. Parikh, A., Täckström, O., Das, D., Uszkoreit, J.: A decomposable attention model for natural language inference. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, Association for Computational Linguistics, pp. 2249–2255 (November 2016)

    Google Scholar 

  14. Sha, L., Chang, B., Sui, Z., Li, S.: Reading and thinking: re-read LSTM unit for textual entailment recognition. In: COLING. (2016)

    Google Scholar 

  15. Wu, Z., Palmer, M.: Verbs Semantics and Lexical Selection. In: Proceedings of the 32Nd Annual Meeting on Association for Computational Linguistics. ACL 1994, pp. 133–138. Stroudsburg, PA, USA, Association for Computational Linguistics (1994)

    Google Scholar 

  16. Lin, D.: An Information-Theoretic Definition of Similarity. In: Proceedings of the Fifteenth International Conference on Machine Learning. ICML 1998, pp. 296–304. San Francisco, CA, USA, Morgan Kaufmann Publishers Inc. (1998)

    Google Scholar 

  17. Pedersen, T., Patwardhan, S., Michelizzi, J.: WordNet: :Similarity: measuring the relatedness of concepts. In: Demonstration Papers at HLT-NAACL 2004. HLT-NAACL-Demonstrations 2004, pp. 38–41. Stroudsburg, PA, USA, Association for Computational Linguistics (2004)

    Google Scholar 

  18. Basak, R., Naskar, S.K., Pakray, P., Gelbukh, A.F.: Recognizing Textual Entailment by Soft Dependency Tree Matching. Computación y Sistemas 19(4), 685–700 (2015)

    Google Scholar 

  19. Herrera, J., Peñas, A., Verdejo, F.: Textual entailment recognition based on dependency analysis and WordNet. In: Quiñonero-Candela, J., Dagan, I., Magnini, B., d’Alché-Buc, F. (eds.) MLCW 2005. LNCS (LNAI), vol. 3944, pp. 231–239. Springer, Heidelberg (2006). https://doi.org/10.1007/11736790_13

    Chapter  Google Scholar 

  20. Snow, R., Vanderwende, L., Menezes, A.: Effectively using syntax for recognizing false entailment. In: Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, New York, USA, Association for Computational Linguistics, pp. 33–40 (June 2006)

    Google Scholar 

  21. Pakray, P., Bandyopadhyay, S., Gelbukh, A.: Dependency parser based textual entailment system. In: Artificial Intelligence and Computational Intelligence, International Conference on(AICI). Vol. 01, pp. 393–397 (Nov 2010)

    Google Scholar 

  22. Rus, V., G.A.M.P.L.K.: A study on textual entailment. In: Proc. of 17th International Conference on Tools with Artificial Intelligence (ICTAI 2005), pp. 326–333 IEEE. (2005)

    Google Scholar 

  23. Marsi, E., Krahmer, E., Bosma, W., Theune, M.: Normalized alignment of dependency trees for detecting textual entailment. In: Springer Verlag, pp. 56–61 (Apr 2006)

    Google Scholar 

  24. Kouylekov, M., Magnini, B.: Recognizing textual entailment with tree edit distance algorithms. In: PASCAL Challenges on RTE, pp. 17–20 (2005)

    Google Scholar 

  25. Haghighi, A.D., Ng, A.Y., Manning, C.D.: Robust textual inference via graph matching. In: Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing. HLT 2005, Stroudsburg, PA, USA, Association for Computational Linguistics, pp. 387–394 (2005)

    Google Scholar 

  26. Sidorov, G.: Should syntactic N-grams contain names of syntactic relations? Int. J. Comput. Linguistics Appl. 5(2), 25–47 (2014)

    Google Scholar 

  27. Dash, S.K., Pakray, P., Gelbukh, A.F.: Learning physiotherapy through virtual action. Computación y Sistemas 20(3), 477–482 (2016)

    Article  Google Scholar 

  28. Alabbas, M., Ramsay, A.: Dependency tree matching with extended tree edit distance with subtrees for textual entailment. In: Federated Conference on Computer Science and Information Systems - FedCSIS 2012, Wroclaw, Poland, 9–12 September 2012, Proceedings, pp. 11–18 (2012)

    Google Scholar 

  29. Marsi, E., Krahmer, E., Bosma, W., Theune, M.: Normalized alignment of dependency trees for detecting textual entailment. In: Magnini, B., Dagan, I., (eds.): Second PASCAL Recognising Textual Entailment Challenge, pp. 56–61 (April 2006)

    Google Scholar 

  30. Dagan, I., Glickman, O.: Probabilistic textual entailment: generic applied modeling of language variability. In: Learning Methods for Text Understanding and Mining. (Jan 2004)

    Google Scholar 

  31. Perez, D., Alfonsecaia, E., Rodríguez, P.: Application of the Bleu Algorithm for recognising textual entailments. In: Proceedings of the Recognising Textual Entailment Pascal Challenge. (2005)

    Google Scholar 

  32. Tatu, M., Moldovan, D.: A Semantic Approach to Recognizing Textual Entailment. In: Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing. HLT 2005, Stroudsburg, PA, USA, Association for Computational Linguistics, pp. 371–378 (2005)

    Google Scholar 

  33. Saikh, T., Naskar, S.K., Giri, C., Bandyopadhyay, S.: Textual entailment using different similarity metrics. In: Gelbukh, A. (ed.) CICLing 2015. LNCS, vol. 9041, pp. 491–501. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-18111-0_37

    Chapter  Google Scholar 

  34. Saikh, T., Naskar, S.K., Bandyopadhyay, S.: JU_NLP@DPIL-FIRE2016: Paraphrase Detection in Indian Languages - A Machine Learning Approach. In: Working notes of FIRE 2016 - Forum for Information Retrieval Evaluation, Kolkata, India, December 7–10, 2016, pp. 275–278 (2016)

    Google Scholar 

  35. Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. ACL 2002, pp. 311–318 Stroudsburg, PA, USA, Association for Computational Linguistics (2002)

    Google Scholar 

  36. Banerjee, S., Lavie, A.: METEOR: an automatic metric for MT evaluation with improved correlation with human judgments. In: Proceedings of Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization at the 43rd Annual Meeting of the Association of Computational Linguistics (ACL-2005), Ann Arbor, Michigan, June 2005. (2005)

    Google Scholar 

  37. Lavie, A., Agarwal, A.: METEOR: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments. In: Proceedings of the Second Workshop on Statistical Machine Translation. StatMT 2007, Stroudsburg, PA, USA, Association for Computational Linguistics, pp. 228–231 (2007)

    Google Scholar 

  38. Saikh, T., Naskar, S., Ekbal, A., Bandyopadhyay, S.: Textual entailment using machine translation evaluation metrics. In: Computational Linguistics and Intelligent Text Processing - 18th International Conference on Computational Linguistics and Intelligent Text Processing, pp. 17–23. CICLing 2017, Budapest, Hungary (April 2017)

    Google Scholar 

  39. Snover, M., Dorr, B., Schwartz, R., Micciulla, L., Makhoul, J.: A study of translation edit rate with targeted human annotation. In: Proceedings of Association for Machine Translation in the Americas, pp. 223–231 (2006)

    Google Scholar 

  40. Lin, C.Y.: ROUGE: a package for automatic evaluation of summaries. In: Proceedings of the Workshop on Text Summarization Branches Out (WAS 2004), Barcelona, Spain, 25–26 July 2004. (2004)

    Google Scholar 

  41. Marsi, E., Krahmer, E., Bosma, W.: Dependency-based paraphrasing for recognizing Textual Entailment. In: Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing. RTE 2007, Stroudsburg, PA, USA, Association for Computational Linguistics, pp. 83–88 (2007)

    Google Scholar 

  42. Lin, D., Pantel, P.: DIRT @SBT@Discovery of Inference Rules from Text. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD 2001, New York, USA, pp. 323–328 ACM (2001)

    Google Scholar 

  43. Yatbaz, M.A.: RTE4: Normalized dependency tree alignment using unsupervised N-gram word similarity score. In: Proceedings of the First Text Analysis Conference, TAC 2008, Gaithersburg, Maryland, USA, 17–19 November 2008, Gaithersburg, Maryland, USA, NIST (2008)

    Google Scholar 

  44. Bowman, S.R., Angeli, G., Potts, C., Manning, C.D.: A Large Annotated Corpus for Learning Natural Language Inference. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17–21, 2015, pp. 632–642 (2015)

    Google Scholar 

  45. Hickl, A., Bensley, J., Williams, J., Roberts, K., Rink, B., Shi, Y.: Recognizing Textual Entailment with LCCs Groundhog System. (2005)

    Google Scholar 

  46. Hickl, A., Bensley, J.: A Discourse Commitment-based Framework for Recognizing Textual Entailment. In: Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing. RTE 2007, Stroudsburg, PA, USA, Association for Computational Linguistics pp.171–176 (2007)

    Google Scholar 

  47. Pakray, P., Gelbukh, A., Bandyopadhyay, S.: A syntactic textual entailment system based on dependency parser. In: Gelbukh, A. (ed.) CICLing 2010. LNCS, vol. 6008, pp. 269–278. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12116-6_22

    Chapter  Google Scholar 

  48. Williams, A., Nangia, N., Bowman, S.R.: A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. CoRR abs/1704.05426 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Indian Institute of Technology Patna, Bihta, India

    Tanik Saikh & Asif Ekbal

  2. Jadavpur University, Kolkata, India

    Sudip Kumar Naskar

Authors
  1. Tanik Saikh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sudip Kumar Naskar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Asif Ekbal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tanik Saikh .

Editor information

Editors and Affiliations

  1. Instituto Politécnico Nacional, Mexico City, Mexico

    Alexander Gelbukh

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Saikh, T., Naskar, S.K., Ekbal, A. (2023). Recognizing Textual Entailment Using Weighted Dependency Relations. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2018. Lecture Notes in Computer Science, vol 13397. Springer, Cham. https://doi.org/10.1007/978-3-031-23804-8_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-23804-8_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-23803-1

  • Online ISBN: 978-3-031-23804-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics