Angelica Willis
James Landay
ABSTRACT
Automatic question generation is a promising tool for developing the learning systems of the future. Research in this area has mostly relied on having answers (key phrases) identified beforehand and given as a feature, which is not practical for real-world, scalable applications of question generation. We describe and implement an end-to-end neural question generation system that generates question and answer pairs given a context paragraph only. We accomplish this by first generating answer candidates (key phrases) from the paragraph context, and then generating questions using the key phrases. We evaluate our method of key phrase extraction by comparing our output over the same paragraphs with question-answer pairs generated by crowdworkers and by educational experts. Results demonstrate that our system is able to generate educationally meaningful question and answer pairs with only context paragraphs as input, significantly increasing the potential scalability of automatic question generation.
- Paul E Black. 2004. Ratcliff/Obershelp pattern recognition. Dictionary of Algorithms and Data Structures 17 (2004).Google Scholar
- Xinya Du, Junru Shao, and Claire Cardie. 2017. Learning to Ask: Neural Question Generation for Reading Comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 1342--1352.Google ScholarCross Ref
- Qi Guo, Chinmay Kulkarni, Aniket Kittur, Jeffrey P. Bigham, and Emma Brunskill. 2016. Questimator: Generating Knowledge Assessments for Arbitrary Topics. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI'16). AAAI Press, 3726--3732. http://dl.acm.org/citation.cfm?id=3061053.3061140 Google ScholarDigital Library
- Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander Rush. 2017. OpenNMT: Open-Source Toolkit for Neural Machine Translation. In Proceedings of ACL 2017, System Demonstrations. Association for Computational Linguistics, 67--72. http://aclweb.org/anthology/P17-4012Google ScholarCross Ref
- Alon Lavie and Abhaya Agarwal. 2007. Meteor: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments. In Proceedings of the Second Workshop on Statistical Machine Translation (StatMT '07). Association for Computational Linguistics, Stroudsburg, PA, USA, 228--231. http://dl.acm.org/citation.cfm?id=1626355.1626389 Google ScholarDigital Library
- Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out. http://aclweb.org/anthology/W04-1013Google Scholar
- Ming Liu, Rafael A Calvo, and Vasile Rus. 2012. G-Asks: An intelligent automatic question generation system for academic writing support. Dialogue & Discourse 3, 2 (2012), 101--124.Google ScholarCross Ref
- Rui Meng, Sanqiang Zhao, Shuguang Han, Daqing He, Peter Brusilovsky, and Yu Chi. 2017. Deep Keyphrase Generation. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2017).Google ScholarCross Ref
- Ruslan Mitkov and Le An Ha. 2003. Computer-aided Generation of Multiple-choice Tests. In Proceedings of the HLT-NAACL 03 Workshop on Building Educational Applications Using Natural Language Processing - Volume 2 (HLT-NAACL-EDUC '03). Association for Computational Linguistics, Stroudsburg, PA, USA, 17--22. Google ScholarDigital Library
- Nasrin Mostafazadeh, Ishan Misra, Jacob Devlin, Margaret Mitchell, Xiaodong He, and Lucy Vanderwende. 2016. Generating Natural Questions About an Image. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 1802--1813.Google ScholarCross Ref
- Jack Mostow and Hyeju Jang. 2012. Generating diagnostic multiple choice comprehension cloze questions. In Proceedings of the Seventh Workshop on Building Educational Applications Using NLP. Association for Computational Linguistics, 136--146. Google ScholarDigital Library
- Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. http://aclweb.org/anthology/P02-1040 Google ScholarDigital Library
- Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global Vectors for Word Representation. In Empirical Methods in Natural Language Processing (EMNLP). 1532--1543. http://www.aclweb.org/anthology/D14-1162Google Scholar
- Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In EMNLP 2016: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2383--2392.Google ScholarCross Ref
- Vasile Rus, Brendan Wyse, Paul Piwek, Mihai Lintean, Svetlana Stoyanchev, and Cristian Moldovan. 2010. The First Question Generation Shared Task Evaluation Challenge. In Proceedings of the 6th International Natural Language Generation Conference (INLG '10). Association for Computational Linguistics, Stroudsburg, PA, USA, 251--257. http://dl.acm.org/citation.cfm?id=1873738.1873777 Google ScholarDigital Library
- Iulian Vlad Serban, Alberto García-Durán, Caglar Gulcehre, Sungjin Ahn, Sarath Chandar, Aaron Courville, and Yoshua Bengio. 2016. Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 588--598.Google ScholarCross Ref
- Sandeep Subramanian, Tong Wang, Xingdi Yuan, Saizheng Zhang, Yoshua Bengio, and Adam Trischler. 2017 Neural Models for Key Phrase Detection and Question Generation. arXiv preprint arXiv:1706.04560 (2017).Google Scholar
- Zichao Wang, Andrew E. Waters, Andrew S. Lan, Phillip J. Grimaldi, Weili Nie, and Richard G. Baraniuk. 2018. QG-Net: A data-driven question generation model for educational content. In L@S'18: Proceedings of the fifth annual ACM conference on learning at scale. Google ScholarDigital Library
- Zhilin Yang, Junjie Hu, Ruslan Salakhutdinov, and William Cohen. 2017. Semi-Supervised QA with Generative Domain-Adaptive Nets. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 1040--1050.Google ScholarCross Ref
Index Terms
- Key Phrase Extraction for Generating Educational Question-Answer Pairs
Recommendations
Generating Question-Answer Pairs for Few-Shot Learning
Artificial Neural Networks and Machine Learning – ICANN 2023AbstractIn the real world, obtaining question-answer pairs for a target domain text is often an expensive process, an approach to tackle the problem is to use automatically generated question-answer pairs from the problem context and large amount of ...
A Comparative Study on Question-Worthy Sentence Selection Strategies for Educational Question Generation
Artificial Intelligence in EducationAbstractAutomatic question generation, which aims at converting sentences in an article to high-quality questions, is an important task for educational practices. Recent work mainly focuses on designing effective generation architectures based on deep ...
Automating Reading Comprehension by Generating Question and Answer Pairs
Advances in Knowledge Discovery and Data MiningAbstractNeural network-based methods represent the state-of-the-art in question generation from text. Existing work focuses on generating only questions from text without concerning itself with answer generation. Moreover, our analysis shows that handling ...
Comments