Abstract
This study focuses on multi-span reading comprehension (RC), which requires answering questions with multiple text spans. Existing approaches for extracting multiple answers require an elaborate dataset that contains questions requiring multiple answers. We propose a method for rewriting single-span answers extracted using several different models to detect single/multiple answer(s). With this approach, only a simple dataset and models for single-span RC are required. We consider multi-span RC with zero-shot learning. Experimental results using the DROP and QUOREF datasets demonstrate that the proposed method improves the exact match (EM) and F1 scores by a large margin on multi-span RC, compared to the baseline models. We further analyzed the effectiveness of combining different models and a strategy for such combinations when applied to multi-span RC.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
30 examples were randomly sampled from each single- and multi-span RC (Dev.). The best u and l were selected from [0.5, 0.6, 0.7, 0.8, 0.9, 1.0] and [0.1, 0.2, 0.3, 0.4, 0.5], respectively.
- 2.
We found the error pattern of 36.0% from randomly sampled 100 error examples.
- 3.
A naive top-k extractor implemented on a single-span baseline repeatedly extracts top-k spans (\(k \ge 2\)) until the number of extracted spans is reached at the fixed number of spans.
References
Andor, D., He, L., Lee, K., Pitler, E.: Giving BERT a calculator: finding operations and arguments with reading comprehension. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp. 5947–5952. Association for Computational Linguistics, November 2019. https://doi.org/10.18653/v1/D19-1609. https://www.aclweb.org/anthology/D19-1609
Clark, K., Luong, M.T., Le, Q.V., Manning, C.D.: Electra: Pre-training text encoders as discriminators rather than generators. In: International Conference on Learning Representations (2020). https://openreview.net/forum?id=r1xMH1BtvB
Dasigi, P., Liu, N.F., Marasović, A., Smith, N.A., Gardner, M.: Quoref: A reading comprehension dataset with questions requiring coreferential reasoning. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp. 5925–5932. Association for Computational Linguistics, November 2019. https://doi.org/10.18653/v1/D19-1606. https://www.aclweb.org/anthology/D19-1606
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 4171–4186. Association for Computational Linguistics, June 2019. https://doi.org/10.18653/v1/N19-1423. https://www.aclweb.org/anthology/N19-1423
Dua, D., Wang, Y., Dasigi, P., Stanovsky, G., Singh, S., Gardner, M.: DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 2368–2378. Association for Computational Linguistics, June 2019. https://doi.org/10.18653/v1/N19-1246. https://www.aclweb.org/anthology/N19-1246
Hu, M., Peng, Y., Huang, Z., Li, D.: A multi-type multi-span network for reading comprehension that requires discrete reasoning. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp. 1596–1606. Association for Computational Linguistics, November 2019. https://doi.org/10.18653/v1/D19-1170.https://www.aclweb.org/anthology/D19-1170
Joshi, M., Choi, E., Weld, D., Zettlemoyer, L.: TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, Canada, pp. 1601–1611. Association for Computational Linguistics, July 2017. https://doi.org/10.18653/v1/P17-1147. https://www.aclweb.org/anthology/P17-1147
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: Albert: a lite BERT for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942 (2019)
Rajpurkar, P., Jia, R., Liang, P.: Know what you don’t know: unanswerable questions for SQuAD. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Melbourne, Australia, pp. 784–789. Association for Computational Linguistics, July 2018. https://doi.org/10.18653/v1/P18-2124. https://www.aclweb.org/anthology/P18-2124
Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: SQuAD: 100,000+ questions for machine comprehension of text. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, pp. 2383–2392. Association for Computational Linguistics, November 2016. https://doi.org/10.18653/v1/D16-1264. https://www.aclweb.org/anthology/D16-1264
Segal, E., Efrat, A., Shoham, M., Globerson, A., Berant, J.: A simple and effective model for answering multi-span questions. arXiv preprint arXiv:1909.13375v3 (2019)
Seo, M., Kembhavi, A., Farhadi, A., Hajishirzi, H.: Bidirectional attention flow for machine comprehension. arXiv preprint arXiv:1611.01603 (2016)
Trischler, A., Wang, T., Yuan, X., Harris, J., Sordoni, A., Bachman, P., Suleman, K.: NewsQA: A machine comprehension dataset. In: Proceedings of the 2nd Workshop on Representation Learning for NLP, Vancouver, Canada, pp. 191–200. Association for Computational Linguistics, August 2017. https://doi.org/10.18653/v1/W17-2623. https://www.aclweb.org/anthology/W17-2623
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., Le, Q.V.: XLNET: generalized autoregressive pretraining for language understanding. arXiv preprint arXiv:1906.08237 (2019)
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., Manning, C.D.: HotpotQA: a dataset for diverse, explainable multi-hop question answering. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, pp. 2369–2380. Association for Computational Linguistics, Oct-Nov 2018. https://doi.org/10.18653/v1/D18-1259, https://www.aclweb.org/anthology/D18-1259
Yu, A.W., Dohan, D., Luong, M.T., Zhao, R., Chen, K., Norouzi, M., Le, Q.V.: QANET: combining local convolution with global self-attention for reading comprehension. arXiv preprint arXiv:1804.09541 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Takahashi, T., Taniguchi, M., Taniguchi, T., Ohkuma, T. (2021). Multi-span Extractive Reading Comprehension Without Multi-span Supervision. In: Hiemstra, D., Moens, MF., Mothe, J., Perego, R., Potthast, M., Sebastiani, F. (eds) Advances in Information Retrieval. ECIR 2021. Lecture Notes in Computer Science(), vol 12657. Springer, Cham. https://doi.org/10.1007/978-3-030-72240-1_41
Download citation
DOI: https://doi.org/10.1007/978-3-030-72240-1_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-72239-5
Online ISBN: 978-3-030-72240-1
eBook Packages: Computer ScienceComputer Science (R0)