Abstract
This paper introduces the system proposed by the "Guess Right or Not (Ours)" team for NLPCC 2023 Shared Task 2 (https://github.com/Yottaxx/NLPCC23_SciMRC)--Multi-perspective Scientific Machine Reading Comprehension. This task requires participants to develop a reading comprehension model based on state-of-the-art Natural Language Processing (NLP) and deep learning techniques to extract word sequences or sentences from the given scientific texts as answers to relevant questions. In response to this task, we use a fine-grained contextual encoder to highlight key contextual information in scientific texts that is highly relevant to the question. Besides, based on existing advanced model CGSN [7], we utilize a local graph network and a global graph network to capture global structural information in scientific texts, as well as the evidence memory network to further alleviate the redundancy issues by saving the selected result in the previous steps. Experiments show that our proposed model performs well on datasets released by NLPCC 2023, and our approach ranks 1st for SMRC Task 2 according to the official results.
Supported by the Natural Science Foundation of China (No.61976026), the Fundamental Research Funds for the Central Universities.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Poon, H., Christensen, J., Domingos, P., et al.: Machine reading at the University of Washington. In: Proceedings of the NAACL HLT 2010 First International Workshop on Formalisms and Methodology for Learning by Reading, pp. 87–95 (2010)
Hirschman, L., Light, M., Breck, E., et al.: Deep read: a reading comprehension system. In: Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics, pp. 325–332 (1999)
Riloff E, Thelen M.: A rule-based question answering system for reading comprehension testsIn. In: ANLP-NAACL 2000 Workshop: Reading Comprehension Tests as Evaluation for Computer-based Language Understanding Systems (2000)
Seo, M., Kembhavi, A., Farhadi, A., et al.: Bidirectional attention flow for machine comprehension. arXiv preprint arXiv:1611.01603 (2016)
Gong, H., Shen, Y., Yu, D., et al.: Recurrent chunking mechanisms for long-text machine reading comprehension. arXiv preprint arXiv:2005.08056 (2020)
Devlin, J., Chang, M.W., Lee, K., et al.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Nie, Y., Huang, H., Wei, W., et al.: Capturing global structural information in long document question answering with compressive graph selector network. arXiv preprint arXiv:2210.05499 (2022)
Liu, S., Zhang, X., Zhang, S., et al.: Neural machine reading comprehension: methods and trends. Appl. Sci. 9(18), 3698 (2019)
Gu, Y., Gui, X., Li, D., Shen, Y., Liao, D.: A review of machine reading comprehension based on neural networks. J. Softw. 31(07), 2095–2126 (2020)
Wang S, Jiang J.: Machine comprehension using match-LSTM and answer pointer. arXiv preprint arXiv:1608.07905 (2016)
Clark C, Gardner M.: Simple and effective multi-paragraph reading comprehension. arXiv preprint arXiv:1710.10723 (2017)
Wang, Y., Liu, K., Liu, J., et al.: Multi-passage machine reading comprehension with cross-passage answer verification. arXiv preprint arXiv:1805.02220 (2018)
Zhang, W., Ren, F.: ELMo+ gated self-attention network based on BiDAF for machine reading comprehension. In: 2020 IEEE 11th International Conference on Software Engineering and Service Science (ICSESS), pp. 1–6 (2020)
Lee, H., Kim, H.: GF-Net: Improving machine reading comprehension with feature gates. Pattern Recogn. Lett. 129, 8–15 (2020)
Wang, C., Jiang, H.: Explicit utilization of general knowledge in machine reading comprehension. arXiv preprint arXiv:1809.03449 (2018)
Ma, X., Zhang, J.: GSA-Net: gated scaled dot-product attention based neural network for reading comprehension. Automatika: časopis za automatiku, mjerenje, elektroniku, računarstvo i komunikacije, 61(4), 643–650 (2020)
Wang, W., Yang, N., Wei, F., et al.: Gated self-matching networks for reading comprehension and question answering. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 189–198 (2017)
Wu, C., Wu, F., Qi, T., et al.: Fastformer: additive attention can be all you need. arXiv preprint arXiv:2108.09084 (2021)
Shen, T., Zhou, T., Long, G., et al.: DiSAN: directional self-attention network for RNN/CNN-free language understanding. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, issue 1 (2018)
Shaw, P., Uszkoreit, J., Vaswani, A.: Self-attention with relative position representations. arXiv preprint arXiv:1803.02155 (2018)
Liu, Y., Ott, M., Goyal, N., et al.: RoBERTa: a robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
Lan, Z., Chen, M., Goodman, S., et al.: ALBERT: a lite BERT for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942 (2019)
Su, C., Fukumoto, F., Huang, X., et al.: DeepMet: a reading comprehension paradigm for token-level metaphor detection. In: Proceedings of the Second Workshop on Figurative Language Processing, pp. 30–39 (2020)
Zhao J, Bao J, Wang Y, et al.: RoR: Read-over-read for long document machine reading comprehension. arXiv preprint arXiv:2109.04780 (2021)
Ding, M., Zhou, C., Yang, H., et al.: Cogltx: applying BERT to long texts. Adv. Neural. Inf. Process. Syst. 33, 12792–12804 (2020)
Yang, Z., Dai, Z., Yang, Y., et al.: XLNet: generalized autoregressive pretraining for language understanding. In: Advances in Neural Information Processing Systems (2019)
Beltagy, I., Lo, K., Cohan, A.: SciBERT: a pretrained language model for scientific text. arXiv preprint arXiv:1903.10676 (2019)
Veličković, P., Cucurull, G., Casanova, A., et al.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
Dasigi, P., Lo, K., Beltagy, I., et al.: A dataset of information-seeking questions and answers anchored in research papers. arXiv preprint arXiv:2105.03011 (2021)
Ainslie, J., Ontanon, S., Alberti, C., et al.: ETC: encoding long and structured inputs in transformers. arXiv preprint arXiv:2004.08483 (2020)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zheng, L., Jia, H., Xie, H., Zhang, X., Shang, Y. (2023). Enhanced CGSN System for Machine Reading Comprehension. In: Liu, F., Duan, N., Xu, Q., Hong, Y. (eds) Natural Language Processing and Chinese Computing. NLPCC 2023. Lecture Notes in Computer Science(), vol 14304. Springer, Cham. https://doi.org/10.1007/978-3-031-44699-3_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-44699-3_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44698-6
Online ISBN: 978-3-031-44699-3
eBook Packages: Computer ScienceComputer Science (R0)
