Skip to main content
SpringerLink
Log in

A Neural Framework for English-Hindi Cross-Lingual Natural Language Inference

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2020)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12532))

Included in the following conference series:

  • 2300 Accesses

Abstract

Recognizing Textual Entailment (RTE) between two pieces of texts is a very crucial problem in Natural Language Processing (NLP), and it adds further challenges when involving two different languages, i.e. in cross-lingual scenario. The paucity of a large volume of datasets for this problem has become the key bottleneck of nourishing research in this line. In this paper, we provide a deep neural framework for cross-lingual textual entailment involving English and Hindi. As there are no large dataset available for this task, we first create this by translating the premises and hypotheses pairs of Stanford Natural Language Inference (SNLI) (https://nlp.stanford.edu/projects/snli/) dataset into Hindi. We develop a Bidirectional Encoder Representations for Transformers (BERT) based baseline on this newly created dataset. We perform experiments in both mono-lingual and cross-lingual settings. For the mono-lingual setting, we obtain the accuracy scores of 83% and 72% for English and Hindi languages, respectively. In the cross-lingual setting, we obtain the accuracy scores of 69% and 72% for English-Hindi and Hindi-English language pairs, respectively. We hope this dataset can serve as valuable resource for research and evaluation of Cross Lingual Textual Entailment (CLTE) models.

T. Saikh and A. De—Equal Contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.iitp.ac.in/~ai-nlp-ml/resources.html.

References

  1. Bowman, S.R., Angeli, G., Potts, C., Manning, C.D.: A large annotated corpus for learning natural language inference. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 632–642, Lisbon, Portugal. Association for Computational Linguistics, September 2015. https://doi.org/10.18653/v1/D15-1075. https://www.aclweb.org/anthology/D15-1075

  2. Chen, Q., Zhu, X., Ling, Z.H., Wei, S., Jiang, H., Inkpen, D.: Enhanced LSTM for natural language inference. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1657–1668, Vancouver, Canada. Association for Computational Linguistics, July 2017. https://doi.org/10.18653/v1/P17-1152. https://www.aclweb.org/anthology/P17-1152

  3. Conneau, A., et al.: XNLI: evaluating cross-lingual sentence representations. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, pp. 2475–2485. Association for Computational Linguistics, October–November 2018. https://doi.org/10.18653/v1/D18-1269. https://www.aclweb.org/anthology/D18-1269

  4. Dagan, I., Glickman, O., Magnini, B.: The PASCAL Recognising Textual Entailment Challenge. In: Quiñonero-Candela, J., Dagan, I., Magnini, B., d’Alché-Buc, F. (eds.) MLCW 2005. LNCS (LNAI), vol. 3944, pp. 177–190. Springer, Heidelberg (2006). https://doi.org/10.1007/11736790_9

  5. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 4171–4186. Association for Computational Linguistics, June 2019. https://doi.org/10.18653/v1/N19-1423. https://www.aclweb.org/anthology/N19-1423

  6. Fleiss, J.L.: Measuring nominal scale agreement among many raters. Psychol. Bull. 76(5), 378 (1971)

    Article  Google Scholar 

  7. Harabagiu, S., Hickl, A.: Methods for using textual entailment in open-domain question answering. In: Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, pp. 905–912. Association for Computational Linguistics (2006)

    Google Scholar 

  8. Lai, G., Xie, Q., Liu, H., Yang, Y., Hovy, E.: RACE: large-scale reading comprehension dataset from examinations. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark, pp. 785–794. Association for Computational Linguistics, September 2017. https://doi.org/10.18653/v1/D17-1082. https://www.aclweb.org/anthology/D17-1082

  9. Landis, J.R., Koch, G.G.: The measurement of observer agreement for categorical data. Biometrics 33, 159–174 (1977)

    Article  Google Scholar 

  10. Liu, X., He, P., Chen, W., Gao, J.: Multi-task deep neural networks for natural language understanding. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp. 4487–4496. Association for Computational Linguistics, July 2019. https://doi.org/10.18653/v1/P19-1441. https://www.aclweb.org/anthology/P19-1441

  11. MacCartney, B., Grenager, T., de Marneffe, M.C., Cer, D., Manning, C.D.: Learning to recognize features of valid textual entailments. In: Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, New York City, USA, pp. 41–48. Association for Computational Linguistics, June 2006. https://www.aclweb.org/anthology/N06-1006

  12. Mehdad, Y., Negri, M., Federico, M.: Towards cross-lingual textual entailment. In: Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Los Angeles, California, pp. 321–324. Association for Computational Linguistics, June 2010. https://www.aclweb.org/anthology/N10-1045

  13. Negri, M., Marchetti, A., Mehdad, Y., Bentivogli, L., Giampiccolo, D.: Semeval-2012 task 8: cross-lingual textual entailment for content synchronization. In: *SEM 2012: The First Joint Conference on Lexical and Computational Semantics - Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012), Montréal, Canada, 7–8 June 2012, pp. 399–407. Association for Computational Linguistics (2012). https://www.aclweb.org/anthology/S12-1053

  14. Negri, M., Marchetti, A., Mehdad, Y., Bentivogli, L., Giampiccolo, D.: Semeval-2013 task 8: cross-lingual textual entailment for content synchronization. In: Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013), Atlanta, Georgia, USA, pp. 25–33. Association for Computational Linguistics, June 2013. https://www.aclweb.org/anthology/S13-2005

  15. Padó, S., Galley, M., Jurafsky, D., Manning, C.D.: Robust machine translation evaluation with entailment features. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, Suntec, Singapore, pp. 297–305. Association for Computational Linguistics, August 2009. https://www.aclweb.org/anthology/P09-1034

  16. Parikh, A., Täckström, O., Das, D., Uszkoreit, J.: A decomposable attention model for natural language inference. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, pp. 2249–2255. Association for Computational Linguistics, November 2016. https://doi.org/10.18653/v1/D16-1244, https://www.aclweb.org/anthology/D16-1244

  17. Radford, A., Narasimhan, K., Salimans, T., Sutskever, I.: Improving language understanding by generative pre-training (2018). https://s3-us-west-2.amazonaws.com/openai-assets/researchcovers/languageunsupervised/languageunderstanding paper.pdf

  18. Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: SQuAD: 100,000+ questions for machine comprehension of text. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, pp. 2383–2392. Association for Computational Linguistics, November 2016. https://doi.org/10.18653/v1/D16-1264. https://www.aclweb.org/anthology/D16-1264

  19. Rocktäschel, T., Grefenstette, E., Hermann, K.M., Kociský, T., Blunsom, P.: Reasoning about entailment with neural attention. In: 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, 2–4 May 2016. Conference Track Proceedings (2016)

    Google Scholar 

  20. Saikh, T., Anand, A., Ekbal, A., Bhattacharyya, P.: A novel approach towards fake news detection: deep learning augmented with textual entailment features. In: Métais, E., Meziane, F., Vadera, S., Sugumaran, V., Saraee, M. (eds.) NLDB 2019. LNCS, vol. 11608, pp. 345–358. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23281-8_30

    Chapter  Google Scholar 

  21. Saikh, T., Ghosal, T., Ekbal, A., Bhattacharyya, P.: document level novelty detection: textual entailment lends a helping hand. In: Proceedings of the 14th International Conference on Natural Language Processing (ICON-2017), Kolkata, India, pp. 131–140. NLP Association of India, December 2017. http://www.aclweb.org/anthology/W/W17/W17-7517

  22. Saikh, T., Naskar, S.K., Ekbal, A., Bandyopadhyay, S.: Textual entailment using machine translation evaluation metrics. In: Gelbukh, A. (ed.) CICLing 2017. LNCS, vol. 10761, pp. 317–328. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-77113-7_25

    Chapter  Google Scholar 

  23. Saikh, T., Naskar, S.K., Giri, C., Bandyopadhyay, S.: Textual entailment using different similarity metrics. In: Gelbukh, A. (ed.) CICLing 2015. LNCS, vol. 9041, pp. 491–501. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-18111-0_37

    Chapter  Google Scholar 

  24. Saini, N., Saha, S., Bhattacharyya, P., Tuteja, H.: Textual entailment-based figure summarization for biomedical articles. ACM Trans. Multimedia Comput. Commun. Appl. (TOMM) 16(1s), 1–24 (2020)

    Article  Google Scholar 

  25. Singh, J., McCann, B., Keskar, N.S., Xiong, C., Socher, R.: XLDA: cross-lingual data augmentation for natural language inference and question answering. arXiv preprint arXiv:1905.11471 (2019)

  26. Wang, S., Jiang, J.: Learning natural language inference with LSTM. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, California, pp. 1442–1451. Association for Computational Linguistics, June 2016. https://doi.org/10.18653/v1/N16-1170. https://www.aclweb.org/anthology/N16-1170

  27. Williams, A., Nangia, N., Bowman, S.: A broad-coverage challenge corpus for sentence understanding through inference. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 1112–1122. Association for Computational Linguistics (2018). http://aclweb.org/anthology/N18-1101

  28. Wu, Y., Schuster, M., et al.: Google’s neural machine translation system: bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 (2016)

  29. Young, P., Lai, A., Hodosh, M., Hockenmaier, J.: From image descriptions to visual denotations: new similarity metrics for semantic inference over event descriptions. Trans. Assoc. Comput. Linguist. 2, 67–78 (2014). https://doi.org/10.1162/tacla000166

    Article  Google Scholar 

Download references

Acknowledgements

We would like to acknowledge “Elsevier Centre of Excellence for Natural Language Processing” at Indian Institute of Technology Patna for partial support of the research work carried out in this paper. Asif Ekbal gratefully acknowledges Visvesvaraya Young Faculty Research Fellowship Award. We also acknowledge the annotators for manually checking the translated outputs.

Author information

Authors and Affiliations

  1. Indian Institute of Technology Patna, Patna, India

    Tanik Saikh & Asif Ekbal

  2. Indian Institute of Technology Hyderabad, Hyderabad, India

    Arkadipta De

  3. Government College of Engineering and Textile Technology, Berhampore, India

    Dibyanayan Bandyopadhyay & Baban Gain

Authors
  1. Tanik Saikh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arkadipta De
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dibyanayan Bandyopadhyay
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Baban Gain
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Asif Ekbal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tanik Saikh .

Editor information

Editors and Affiliations

  1. Department of AI, Ping An Life, Shenzhen, China

    Haiqin Yang

  2. Faculty of Information Technology, King Mongkut's Institute of Technology Ladkrabang, Bangkok, Thailand

    Kitsuchart Pasupa

  3. City University of Hong Kong, Kowloon, Hong Kong

    Andrew Chi-Sing Leung

  4. Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong, Hong Kong

    James T. Kwok

  5. School of Information Technology, King Mongkut’s University of Technology Thonburi, Bangkok, Thailand

    Jonathan H. Chan

  6. The Chinese University of Hong Kong, New Territories, Hong Kong

    Irwin King

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Saikh, T., De, A., Bandyopadhyay, D., Gain, B., Ekbal, A. (2020). A Neural Framework for English-Hindi Cross-Lingual Natural Language Inference. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Lecture Notes in Computer Science(), vol 12532. Springer, Cham. https://doi.org/10.1007/978-3-030-63830-6_55

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-63830-6_55

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-63829-0

  • Online ISBN: 978-3-030-63830-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation