Abstract
Given a passage and a question, Multiple-choice Machine Reading Comprehension (MRC) requires to select the correct answer from several candidates. Existing methods consider more about the accuracy of the prediction of the final answer in a “black box”, which provides no concrete evidence to explain the choice making process. Intuitively, evidence is helpful in building a convincing multiple-choice MRC model. Due to the lack of the golden evidence labels and high cost of manual annotation, we realize weak evidence labels in an automatic way to integrate evidence extraction into the training of MRC models. This auxiliary task learns to select sentences that are more relevant to the question during training, and makes our MRC model interpretable. We come up with an end-to-end model called (EAM) Evidence Augment Model, which learns evidence extraction and answer prediction jointly. More accurate results can be obtained by our model without the need of additional manual annotation. Experimental results on RACE datasets show that we obtain an improvement in accuracy over the previous best result based on a fair Pre-trained Language Model (PrLM).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
paraphrase-distilroberta-base-v1.
- 3.
References
Berzak, Y., Malmaud, J., Levy, R.: Starc: Structured annotations for reading comprehension. arXiv preprint arXiv:2004.14797 (2020)
Chen, D., Bolton, J., Manning, C.D.: A thorough examination of the CNN/Daily Mail reading comprehension task. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2358–2367. Association for Computational Linguistics, Berlin, Germany (2016). https://doi.org/10.18653/v1/P16-1223, https://www.aclweb.org/anthology/P16-1223
Chen, Z., Cui, Y., Ma, W., Wang, S., Hu, G.: Convolutional spatial attention model for reading comprehension with multiple-choice questions. Proc. AAAI Conf. Artif. Intell. 33, 6276–6283 (2019)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Joshi, M., Choi, E., Weld, D.S., Zettlemoyer, L.: Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551 (2017)
Lai, G., Xie, Q., Liu, H., Yang, Y., Hovy, E.: Race: Large-scale reading comprehension dataset from examinations. arXiv preprint arXiv:1704.04683 (2017)
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942 (2019)
Malmaud, J., Levy, R., Berzak, Y.: Bridging information-seeking human gaze and machine reading comprehension. arXiv preprint arXiv:2009.14780 (2020)
Miao, H., Liu, R., Gao, S.: A multiple granularity co-reasoning model for multi-choice reading comprehension. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE (2019)
Parikh, S., Sai, A., Nema, P., Khapra, M.: Eliminet: a model for eliminating options for reading comprehension with multiple choice questions. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pp. 4272–4278. International Joint Conferences on Artificial Intelligence Organization (2018). https://doi.org/10.24963/ijcai.2018/594
Rajpurkar, P., Jia, R., Liang, P.: Know what you don’t know: Unanswerable questions for squad. arXiv preprint arXiv:1806.03822 (2018)
Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016)
Ran, Q., Li, P., Hu, W., Zhou, J.: Option comparison network for multiple-choice reading comprehension. arXiv preprint arXiv:1903.03033 (2019)
Reimers, N., Gurevych, I.: Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 (2019)
Richardson, M., Burges, C.J., Renshaw, E.: Mctest: a challenge dataset for the open-domain machine comprehension of text. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 193–203 (2013)
Sun, K., Yu, D., Chen, J., Yu, D., Choi, Y., Cardie, C.: Dream: a challenge data set and models for dialogue-based reading comprehension. Trans. Assoc. Comput. Linguist. 7, 217–231 (2019)
Sun, K., Yu, D., Yu, D., Cardie, C.: Improving machine reading comprehension with general reading strategies. arXiv preprint arXiv:1810.13441 (2018)
Tang, M., Cai, J., Zhuo, H.H.: Multi-matching network for multiple choice reading comprehension. Proc. AAAI Conf. Artif. Intell. 33, 7088–7095 (2019)
Tay, Y., Tuan, L.A., Hui, S.C.: Multi-range reasoning for machine comprehension. arXiv preprint arXiv:1803.09074 (2018)
Trischler, A., et al.: Newsqa: A machine comprehension dataset. arXiv preprint arXiv:1611.09830 (2016)
Wang, H., et al.: Evidence sentence extraction for machine reading comprehension. arXiv preprint arXiv:1902.08852 (2019)
Wang, S., Yu, M., Chang, S., Jiang, J.: A co-matching model for multi-choice reading comprehension. arXiv preprint arXiv:1806.04068 (2018)
Xu, Y., Liu, J., Gao, J., Shen, Y., Liu, X.: Dynamic fusion networks for machine reading comprehension. arXiv preprint arXiv:1711.04964 (2017)
Yu, S., Zhang, H., Jing, W., Jiang, J.: Context modeling with evidence filter for multiple choice question answering. arXiv preprint arXiv:2010.02649 (2020)
Zhang, S., Zhao, H., Wu, Y., Zhang, Z., Zhou, X., Zhou, X.: Dcmn+: dual co-matching network for multi-choice reading comprehension. Proc. AAAI Conf. Artif. Intell. 34, 9563–9570 (2020)
Zhang, Z., Wu, Y., Zhou, J., Duan, S., Zhao, H., Wang, R.: Sg-net: syntax-guided machine reading comprehension. Proc. AAAI Conf. Artif. Intell. 34, 9636–9643 (2020)
Zhao, Y., Zhang, Z., Zhao, H.: Reference knowledgeable network for machine reading comprehension. arXiv preprint arXiv:2012.03709 (2020)
Zhu, H., Wei, F., Qin, B., Liu, T.: Hierarchical attention flow for multiple-choice reading comprehension. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Zhu, P., Zhao, H., Li, X.: Duma: Reading comprehension with transposition thinking. arXiv: Computation and Language (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Luo, D. et al. (2021). Evidence Augment for Multiple-Choice Machine Reading Comprehension by Weak Supervision. In: Farkaš, I., Masulli, P., Otte, S., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2021. ICANN 2021. Lecture Notes in Computer Science(), vol 12895. Springer, Cham. https://doi.org/10.1007/978-3-030-86383-8_29
Download citation
DOI: https://doi.org/10.1007/978-3-030-86383-8_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86382-1
Online ISBN: 978-3-030-86383-8
eBook Packages: Computer ScienceComputer Science (R0)