Shichen Chen
ABSTRACT
Abstract—Question Generation (QG) is an essential task in natural language processing, which aims to generate grammatical questions for given sentences or paragraphs. This task focuses on understanding the structures and semantics of sentences. This paper proposes a graph-enhanced question generation model based on Variational Autoencoders (VAE). We construct a semantic graph by dependency parsing and then encode the graph by iteration of Gated Graph Neural Networks (GGNN). Then we get the graph-enhanced representation by fusing the sentence-level and graph-level representation to perform pre-training of statement recovery and joint training of question type prediction and generation. Experiments show promising results of our proposed model on the most extensively used Question Answering (QA) dataset SQuAD.
- Nan Duan, Duyu Tang, Peng Chen, and Ming Zhou. 2017. Question Generation for Question Answering. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 866–874.Google Scholar
Cross Ref
- Jingjing Xu, Yuechen Wang, Duyu Tang, Nan Duan, Pengcheng Yang, Qi Zeng, Ming Zhou, and Xu Sun. 2019. Asking clarification questions in knowledgebased question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1618–1629.Google ScholarCross Ref
- Michael Heilman and Noah A. Smith. 2010. Good question! statistical ranking for question generation. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 609–617.Google Scholar
- Nasrin Mostafazadeh, Ishan Misra, Jacob Devlin, Margaret Mitchell, Xiaodong He, and Lucy Vanderwende. 2016. Generating natural questions about an image. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1802–1813.Google ScholarCross Ref
- David Lindberg, Fred Popowich, John Nesbit, and PhilWinne. 2013. Generating natural language questions to support learning on-line. In Proceedings of the 14th European Workshop on Natural Language Generation, pages 105–114.Google Scholar
- Xinya Du, Junru Shao, and Claire Cardie. 2017. Learning to ask: Neural question generation for reading comprehension. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 1342–1352.Google ScholarCross Ref
- Qingyu Zhou, Nan Yang, Furu Wei, Chuanqi Tan, Hangbo Bao, and Ming Zhou. 2017. Neural question generation from text: A preliminary study. In CCF International Conference of Natural Language Processing and Chinese Computing (NLPCC), pages 662–671.Google Scholar
- Jingjing Li, Yifan Gao, Lidong Bing, Irwin King, and Michael R. Lyu. 2019. Improving question generation with to the point context. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3216–3226.Google Scholar
- Valentina Pyatkin, Paul Roit, Julian Michael, Yoav Goldberg, Reut Tsarfaty, and Ido Dagan. 2021. Asking it all: Generating contextualized questions for any semantic role. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1429–1441.Google ScholarCross Ref
- Yanghoon Kim, Hwanhee Lee, Joongbo Shin, and Kyomin Jung. 2019. Improving neural question generation using answer separation. In AAAI Conference on Artificial Intelligence (AAAI), pages 6602–6609.Google ScholarDigital Library
- Xin Jia, Wenjie Zhou, Xu Sun, and Yunfang Wu. 2020. How to Ask Good Questions? Try to Leverage Paraphrases. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6130–6140.Google ScholarCross Ref
- Diederik P. Kingma and Max Welling. 2014. Autoencoding variational bayes. In 2nd International Conference on Learning Representations, ICLR 2014.Google Scholar
- Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard S. Zemel. 2016. Gated graph sequence neural networks. In International Conference on Learning Representations (ICLR).Google Scholar
- Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392.Google ScholarCross Ref
- Xingdi Yuan, Tong Wang, Caglar Gulcehre, Alessandro Sordoni, Philip Bachman, Sandeep Subramanian, Saizheng Zhang, and Adam Trischler. 2017. Machine Comprehension by Text-to-Text Neural Question Generation. ArXiv preprint arXiv:1705.02012.Google Scholar
- Bang Liu, Mingjun Zhao, Di Niu, Kunfeng Lai, Yancheng He, Haojie Wei, and Yu Xu. 2019. Learning to Generate Questions by Learning What not to Generate. In The World Wide Web Conference, pages 1106–1118.Google Scholar
- Thomas N Kipf and Max Welling. 2017. Semi-supervised Classification with Graph Convolutional Networks. In 5th International Conference on Learning Representations, ICLR 2017.Google Scholar
- Yu Chen, Lingfei Wu, and Mohammed J. Zaki. 2020. Reinforcement learning based graph-to-sequence model for natural question generation. In Proceedings of the 8th International Conference on Learning Representations.Google Scholar
- Xiyao Ma, Qile Zhu, Y. Zhou, Xiaolin Li, and Dapeng Wu. 2020. Asking complex questions with multi-hop answerfocused reasoning. ArXiv preprint arXiv:2009.07402.Google Scholar
- Weichao Wang, Shi Feng, Daling Wang, and Yifei Zhang. 2019. Answer-guided and Semantic Coherent Question Generation in Open-domain Conversation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5066–5076.Google ScholarCross Ref
- Dong Bok Lee, Seanie Lee, Woo Tae Jeong, Donghwan Kim, and Sung Ju Hwang. 2020. Generating Diverse and Consistent QA pairs from Contexts with Information-Maximizing Hierarchical Conditional VAEs. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 208–224.Google ScholarCross Ref
- Lei Shen, Fandong Meng, Jinchao Zhang, Yang Feng, and Jie Zhou. 2021. GTM: A Generative Triple-wise Model for Conversational Question Generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3495–3506.Google ScholarCross Ref
- Wenjie Zhou, Minghua Zhang, and Yunfang Wu. 2019. Multi-Task Learning with Language Modeling for Question Generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3394–3399.Google ScholarCross Ref
- Xiyao Ma, Qile Zhu, Yanlin Zhou, Xiaolin Li, and Dapeng Wu. 2020. Improving Question Generation with Sentence-level Semantic Matching and Answer Position Inferring. In AAAI Conference on Artificial Intelligence.Google Scholar
- Bodhisattwa Prasad Majumder, Sudha Rao, Michel Galley, and Julian McAuley. 2021. Ask what's missing and what's useful: Improving Clarification Question Generation using Global Knowledge. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4300–4312.Google ScholarCross Ref
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems 30 (2017).Google Scholar
- Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1724–1734.Google ScholarCross Ref
- Diederik P. Kingma and Max Welling. 2014. Auto-encoding variational bayes. In 2nd International Conference on Learning Representations, ICLR 2014.Google Scholar
- Chris J. Maddison, Andriy Mnih, and Yee Whye Teh. 2017. The concrete distribution: A continuous relaxation of discrete random variables. In 5th International Conference on Learning Representations, ICLR 2017.Google Scholar
- Eric Jang, Shixiang Gu, and Ben Poole. 2017. Categorical reparameterization with gumbel-softmax. In 5th International Conference on Learning Representations, ICLR 2017.Google Scholar
- Yao Zhao, Xiaochuan Ni, Yuanyuan Ding, and Qifa Ke. 2018. Paragraph-level Neural Question Generation with Maxout Pointer and Gated Self-attention Networks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3901–3910.Google ScholarCross Ref
- Dorottya Demszky, Kelvin Guu, and Percy Liang. 2018. Transforming question answering datasets into natural language inference datasets. ArXiv preprint arXiv:1809.02922.Google Scholar
- Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318.Google Scholar
- Alon Lavie and Abhaya Agarwal. 2007. METEOR: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 228–231.Google ScholarCross Ref
- Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, pages 74–81.Google Scholar
- Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke Zettlemoyer. 2018. AllenNLP: A Deep Semantic Natural Language Processing Platform. In Proceedings of Workshop for NLP Open Source Software (NLP-OSS), pages 1–6, Melbourne, Australia. Association for Computational Linguistics.Google ScholarCross Ref
Index Terms
- Graph-Enhanced Question Generation with Question Type Prediction and Statement Recovery
Index terms have been assigned to the content through auto-classification.
Recommendations
Improving Neural Question Generation using Deep Linguistic Representation
WWW '21: Proceedings of the Web Conference 2021Question Generation (QG) is a challenging Natural Language Processing (NLP) task which aims at generating questions with given answers and context. There are many works incorporating linguistic features to improve the performance of QG. However, ...
Neural Question Generation with Semantics of Question Type
Natural Language Processing and Chinese ComputingAbstractThis paper focuses on automatic question generation (QG) that transforms a narrative sentence into an interrogative sentence. Recently, neural networks have been used in this task due to its extraordinary ability of semantics encoding and ...
Syntax-guided question generation using prompt learning
AbstractQuestion generation (QG) aims to generate natural questions from relevant input. Existing state-of-the-art QG approaches primarily leverage pre-trained language models (PLMs) to encode the deep semantics within the input. Meanwhile, studies show ...
Comments