Skip to main content
SpringerLink
Log in

Multi-grained Logical Graph Network for Reasoning-Based Machine Reading Comprehension

  • Conference paper
  • First Online:
Advanced Data Mining and Applications (ADMA 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14179))

Included in the following conference series:

  • 386 Accesses

Abstract

Machine reading comprehension (MRC) is a crucial and challenging task in natural language processing (NLP). In order to equip machines with logical reasoning abilities, the challenging logical reasoning tasks are proposed. Existing approaches use graph-based neural models based on either sentence-level or entity-level graph construction methods which designed to capture a logical structure and enable inference over it. However, sentence-level methods result in a loss of fine-grained information and difficulty in capturing implicit relationships, while entity-level methods fail to capture the overall logical structure of the text. To address these issues, we propose a multi-grained graph-based mechanism for solving logical reasoning MRC. To combine the advantages of sentence-level and entity-level information, we mine elementary discourse units (EDUs) and entities from texts to construct graph, and learn the logical-aware features through a graph network for subsequent answer prediction. Furthermore, we implement a positional embedding mechanism to enforce the positional dependence, which facilitates logical reasoning. Our experimental results demonstrate that our approach provides significant and consistent improvements via multi-grained graphs, outperforming competitive baselines on both ReClor and LogiQA benchmarks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Cai, L., Li, J., Wang, J., Ji, S.: Line Graph Neural Networks for Link Prediction (2020)

    Google Scholar 

  2. Cetto, M., Niklaus, C., Freitas, A., Handschuh, S.: Graphene: Semantically-Linked Propositions in Open Information Extraction (2018)

    Google Scholar 

  3. Chen, K., et al.: Question Directed Graph Attention Network for Numerical Reasoning over Text (2020)

    Google Scholar 

  4. Cho, K., et al.: Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation (2014)

    Google Scholar 

  5. Chowdhury, G.G.: Introduction to Modern Information Retrieval. Facet Publishing (2010)

    Google Scholar 

  6. Christopoulou, F., Miwa, M., Ananiadou, S.: Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs (2019)

    Google Scholar 

  7. Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805 (2019)

  8. Dua, D., et al.: DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs (2019)

    Google Scholar 

  9. Fan, F., Feng, Y., Zhao, D.: Multi-grained attention network for aspect-level sentiment classification. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium, pp. 3433–3442. Association for Computational Linguistics (2018)

    Google Scholar 

  10. Hermann, K.M., et al.: Teaching machines to read and comprehend. In: Advances in Neural Information Processing Systems, vol. 28. Curran Associates Inc. (2015)

    Google Scholar 

  11. Hirschman, L., Light, M., Breck, E., Burger, J.D.: Deep Read: a reading comprehension system. In: Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics. College Park, Maryland, USA, pp. 325–332. Association for Computational Linguistics (1999)

    Google Scholar 

  12. "History of programming languages–II | ACM Other Books. https://dl.acm.org/doi/abs/10.1145/234286

  13. Huang, L., Bras, R.L., Bhagavatula, C., Choi, Y.: Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning (2019)

    Google Scholar 

  14. Huang, Y., Fang, M., Cao, Y., Wang, L., Liang, X.: DAGN: Discourse-Aware Graph Network for Logical Reasoning (2021)

    Google Scholar 

  15. Jia, R., Cao, Y., Tang, H., Fang, F., Cao, C., Wang, S.: Neural extractive summarization with hierarchical attentive heterogeneous graph network. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 3622–3631. Online: Association for Computational Linguistics (2020)

    Google Scholar 

  16. Jiao, F., Guo, Y., Song, X., Nie, L.: MERIt: Meta-Path Guided Contrastive Learning for Logical Reasoning (2022)

    Google Scholar 

  17. Khashabi, D., Chaturvedi, S., Roth, M., Upadhyay, S., Roth, D.: Looking beyond the surface: a challenge set for reading comprehension over multiple sentences. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 252–262. Association for Computational Linguistics, New Orleans, Louisiana (2018)

    Google Scholar 

  18. Li, X., Cheng, G., Chen, Z., Sun, Y., Qu, Y.: AdaLoGN: Adaptive Logic Graph Network for Reasoning-Based Machine Reading Comprehension (2022)

    Google Scholar 

  19. Liu, S., Zhang, X., Zhang, S., Wang, H., Zhang, W.: Neural machine reading comprehension: methods and trends. Appl. Surf. Sci. 471, 18–22 (2019)

    Google Scholar 

  20. Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach (2019)

    Google Scholar 

  21. Liu, J., Cui, L., Liu, H., Huang, D., Wang, Y., Zhang, Y.: LogiQA: a challenge dataset for machine reading comprehension with logical reasoning (2020)

    Google Scholar 

  22. McCarthy, J.: Artificial intelligence, logic and formalizing common sense. In: Thomason, R.H. (ed.) Philosophical Logic and Artificial Intelligence, pp. 161–190. Springer, Netherlands, Dordrecht (1989). https://doi.org/10.1007/978-94-009-2448-2_6

    Chapter  Google Scholar 

  23. Mihaylov, T., Clark, P., Khot, T., Sabharwal, A.: Can a suit of armor conduct electricity? A new dataset for open book question answering (2018)

    Google Scholar 

  24. Nguyen, T. et al.: MS MARCO: A Human Generated MAchine Reading COmprehension Dataset (2016)

    Google Scholar 

  25. Ouyang, S., Zhang, Z., Zhao, H.: Fact-driven Logical Reasoning (2021)

    Google Scholar 

  26. Paszke, A., et al.: Automatic differentiation in PyTorch (2017)

    Google Scholar 

  27. Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: SQuAD: 100,000+ questions for machine comprehension of text (2016)

    Google Scholar 

  28. Rajpurkar, P., Jia, R., Liang, P.: Know what you don’t know: unanswerable questions for SQuAD (2018)

    Google Scholar 

  29. Ran, Q., Lin, Y., Li, P., Zhou, J., Liu, Z.: NumNet: machine reading comprehension with numerical reasoning (2019)

    Google Scholar 

  30. Riloff, E., Thelen, M.: A rule-based question answering system for reading comprehension tests. In: ANLP-NAACL 2000 Workshop: Reading Comprehension Tests as Evaluation for Computer-Based Language Understanding Systems (2000)

    Google Scholar 

  31. Sahu, S.K., Christopoulou, F., Miwa, M., Ananiadou, S.: Inter-sentence relation extraction with document-level graph convolutional neural network (2019)

    Google Scholar 

  32. Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE Trans. Neural Netw. 20(1), 61–80 (2009)

    Article  Google Scholar 

  33. Seo, M., Kembhavi, A., Farhadi, A., Hajishirzi, H.: Bidirectional Attention Flow for Machine Comprehension (2018)

    Google Scholar 

  34. Sugawara, S., Aizawa, A.: An analysis of prerequisite skills for reading comprehension. In: Proceedings of the Workshop on Uphill Battles in Language Processing: Scaling Early Achievements to Robust Methods, pp. 1–5. Association for Computational Linguistics, Austin, TX (2016)

    Google Scholar 

  35. Tandon, N., Mishra, B.D., Sakaguchi, K., Bosselut, A., Clark, P.: WIQA: a dataset for “What if..." reasoning over procedural text (2019)

    Google Scholar 

  36. Wang, S., Jiang, J.: Machine Comprehension Using Match-LSTM and Answer Pointer (2016)

    Google Scholar 

  37. Wang, S., et al.: Logic-driven context extension and data augmentation for logical reasoning of text (2021)

    Google Scholar 

  38. Wang, S., et al.: From LSAT: the progress and challenges of complex reasoning (2021)

    Google Scholar 

  39. Welbl, J., Stenetorp, P., Riedel, S.: Constructing datasets for multi-hop reading comprehension across documents. Trans. Assoc. Comput. Linguist. 6, 287–302 (2018)

    Article  Google Scholar 

  40. Xu, K., Wu, L., Wang, Z., Yu, M., Chen, L., Sheinin, V.: Exploiting rich syntactic information for semantic parsing with graph-to-sequence model (2018)

    Google Scholar 

  41. Xu, M., Li, L., Wong, D.F., Liu, Q., Chao, L.S.: Document graph for neural machine translation (2021)

    Google Scholar 

  42. Xu, F., Liu, J., Lin, Q., Pan, Y., Zhang, L.: Logiformer: a two-branch graph transformer network for interpretable logical reasoning. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1055–1065 (2022)

    Google Scholar 

  43. Yang, Z., et al.: HotpotQA: a dataset for diverse, explainable multi-hop question answering (2018)

    Google Scholar 

  44. Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R.R., Le, Q.V.: XLNet: generalized autoregressive pretraining for language understanding

    Google Scholar 

  45. Yao, L., Mao, C., Luo, Y.: Graph convolutional networks for text classification. Proc. AAAI Conf. Artif. Intell. 33(01), 7370–7377 (2019)

    Google Scholar 

  46. Ying, C.T., et al.: Transformers really perform bad for graph representation?"

    Google Scholar 

  47. Yu, A.W., et al.: QANet: combining local convolution with global self-attention for reading comprehension (2018)

    Google Scholar 

  48. Yu, W., Jiang, Z., Dong, Y., Feng, J.: ReClor: a reading comprehension dataset requiring logical reasoning (2020)

    Google Scholar 

  49. Zeng, S., Xu, R., Chang, B., Li, L.: Double graph based reasoning for document-level relation extraction (2020)

    Google Scholar 

  50. Zhang, Z., Zhao, H., Wang, R.: Machine reading comprehension: the role of contextualized language models and beyond. arXiv:2005.06249 (2020)

  51. Zheng, B., et al.: Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension (2020)

    Google Scholar 

  52. Zhou, B., Khashabi, D., Ning, Q., Roth, D.: Going on a vacation takes longer than “Going for a walk”: a study of temporal commonsense understanding (2019)

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Associate Editor and anonymous reviewers for their valuable comments and suggestions. This work is funded in part by the National Natural Science Foundation of China under Grants No.62176029, and in part by the graduate research and innovation foundation of Chongqing, China under Grants No.CYB21063. This work also is supported in part by the Chongqing Technology Innovation and Application Development Special under Grants CSTB2022TIAD-KPX0206.

Author information

Authors and Affiliations

  1. Chongqing University, Chongqing, China

    Jiaqi Wang, Jia Zhong, Hong Yin, Chen Wang, Qizhu Dai & Rongzhen Li

  2. Chongqing Academy of Big Data Co., Ltd., Chongqing, China

    Yang Xia

Authors
  1. Jiaqi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jia Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hong Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qizhu Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yang Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Rongzhen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jia Zhong .

Editor information

Editors and Affiliations

  1. Northeastern University, Shenyang, China

    Xiaochun Yang

  2. The University of Indonesia, Depok, Indonesia

    Heru Suhartanto

  3. Beijing Institute of Technology, Beijing, China

    Guoren Wang

  4. Northeastern University, Shenyang, China

    Bin Wang

  5. University of Technology Sydney, Sydney, NSW, Australia

    Jing Jiang

  6. Agency for Science, Technology and Research (A*STAR), Singapore, Singapore

    Bing Li

  7. Sun Yat-sen University, Guangzhou, China

    Huaijie Zhu

  8. Anhui University, Hefei, China

    Ningning Cui

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, J. et al. (2023). Multi-grained Logical Graph Network for Reasoning-Based Machine Reading Comprehension. In: Yang, X., et al. Advanced Data Mining and Applications. ADMA 2023. Lecture Notes in Computer Science(), vol 14179. Springer, Cham. https://doi.org/10.1007/978-3-031-46674-8_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-46674-8_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-46673-1

  • Online ISBN: 978-3-031-46674-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation