Jun He
ABSTRACT
The difficulty of test questions is an important indicator for educational examination and recommendation of personalized learning resources. Its evaluation mainly depends on the experience of experts, which is subjective. In recent years, question difficulty prediction (QDP) using neural networks has attracted more and more attention. Although these methods improve the QDP efficiency, it works ill for questions involving abstract concepts, such as numerical calculation, date, and questions whose answers require background knowledge. Therefore, we propose a difficulty prediction model based on rich knowledge fusion (RKF+), which solves the problem that the difficulty prediction models cannot obtain conceptual knowledge and background knowledge. The key is to introduce the attentional mechanism with a sentry vector, which can dynamically obtain the text representation and external knowledge representation of test questions. To further fusion the acquired external knowledge, our model added a bi-interaction layer. Finally, the validity of this model is verified on three different datasets. Besides, the importance of attentional mechanism and external knowledge representation is further analyzed by ablation experiment. In addition, based on a real English reading comprehension test dataset, we explore the influence of two kinds of external knowledge on the question difficulty prediction model.
- Fuchs LS, Fuchs D. Effects of Expert System Consultation Within Curriculum-based Measurement, Using a Reading Maze Task[J]. Exceptional Children, 1992, 58(5): 436-450.Google Scholar
Cross Ref
- Mao, J.F. (2008) .Gaokao mingti zhong QDP fangfa tansuo[J].Educational Sciences(06),22-26.Google Scholar
- Boopathiraj C, Chellamani K. Analysis of Test Items on Difficulty Level and Discrimination Index in the Test for Research in Education[J]. International Journal of Social Sciences & Interdisciplinary Research, 2013.Google Scholar
- Flesch R. Journal of Applied Psychology[J]. Journal of Applied Psychology. 1948(3): 450-463. J. Clerk Maxwell, A Treatise on Electricity and Magnetism, 3rd ed., vol. 2. Oxford: Clarendon, 1892, pp.68–73.Google Scholar
- Zhenya Huang, Qi Liu, Enhong Chen, Question difficulty prediction for reading problems in standard tests[C]//31st AAAI Conference on Artificial Intelligence. San Francisco, 2017:1352–1359.Google Scholar
- Yaneva V, Ha LA, Baldwin P, Predicting Item Survival for Multiple Choice Questions in a High-stakes Medical Exam[C]//12th International Conference on Language Resources and Evaluation.Marseille, 2020:6812- 6818.Google Scholar
- He J, Peng L, Sun B, Automatically Predict Question Difficulty for Reading Comprehension Exercises[C]//2021 IEEE 33rd International Conference on Tools with Artificial Intelligence (ICTAI). IEEE, 2021: 1398-1402.Google Scholar
- Alagumalai S, Curtis D D. Classical test theory[M]//Applied Rasch measurement: A book of exemplars: Papers in honour of John P. Keeves, 2005:1-14.Google Scholar
- Dibello L V, Roussos L A, Stout W. 31A Review of Cognitively Diagnostic Assessment and a Summary of Psychometric Models[J]. Handbook of Statistics,,,2006(6):979-1030.Google Scholar
- Liu Y, Qi L, Wu R, Collaborative Learning Team Formation: A Cognitive Modeling Perspective[C]//21st International Conference on Database Systems for Advanced Applications. Dallas, 2016:383-400. I. S. Jacobs and C. P. Bean, “Fine particles, thin films and exchange anisotropy,” in Magnetism, vol. III, G. T. Rado and H. Suhl, Eds. New York: Academic, 1963, pp. 271–350.Google Scholar
- Hermann K M, Koisk, Tomá, Grefenstette E, Teaching Machines to Read and Comprehend[C]//29th Annual Conference on Neural Information Processing Systems, Montreal, 2015:1693-1701.Google Scholar
- Wen H, Wang Z, Wang H, Short text understanding through lexical-semantic analysis[C]//31st IEEE International Conference on Data Engineering. Seoul, 2015:495-506.Google Scholar
- I. S. Jacobs and C. P. Bean, “Fine particles, thin films and exchange anisotropy,” in Magnetism, vol. III, G. T. Rado and H. Suhl, Eds. New York: Academic, 1963, pp. 271–350.Google Scholar
- Tong, W. (2019) .Shuju qudong de shuxueshiti nanduyuce[J]. Computer Development and Research(5). :1007-1019.Google Scholar
- Liu Q, Huang Z, Huang Z, Finding Similar Exercises in Online Education Systems[C]//24th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. London, 2018:1821-1830.Google Scholar
- Le, Y. (2020) . Ronghe waibuzhishi de jiqiyuedulijie fangfa yanjiu[D]. Harbin Institute of Technology.Google Scholar
- Xu J, Cai Y, Wu X, Incorporating Context-relevant Concepts Into Convolutional Neural Networks for Short Text Classification[J]. Neurocomputing, 2020, 386: 42-53.Google ScholarCross Ref
- Xu J, Cai Y. Incorporating Context-relevant Knowledge Into Convolutional Neural Networks for Short Text Classification[C]//Proceedings of the Aaai Conference on Artificial Intelligence, 2019: 10067-10068.Google Scholar
- Yang A, Wang Q, Liu J, Enhancing Pre-trained Language Representations with Rich Knowledge for Machine Reading Comprehension[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2346-2357.Google Scholar
- Pan X, Sun K, Yu D, Improving Question Answering with External Knowledge[J]. Arxiv Preprint Arxiv:1902.00993, 2019.Google Scholar
- Wang H, Bansal M, Gimpel K, Machine Comprehension with Syntax, Frames, and Semantics[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (volume 2: Short Papers), 2015: 700-706.Google Scholar
- Qiu D, Zhang Y, Feng X, Machine Reading Comprehension Using Structural Knowledge Graph-aware Network[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (emnlp-ijcnlp), 2019: 5896-5901.Google Scholar
- Margatina K, Baziotis C, Potamianos A. Attention-based Conditioning Methods for External Knowledge Integration[J]. Arxiv Preprint Arxiv:1906.03674, 2019.Google Scholar
- Sun K, Yu D, Yu D, Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehension[J]. Transactions of the Association for Computational Linguistics, 2020, 8: 141-155.Google ScholarCross Ref
- Wang Z, Wang H, Wen J, An Inference Approach to Basic Level of Categorization[C]//Proceedings of the 24th Acm International on Conference on Information and Knowledge Management, 2015: 653-662.Google Scholar
- Yang B, Yih W, He X, Embedding Entities and Relations for Learning and Inference in Knowledge Bases[C]: International Conference on Learning Representations, Iclr, 2015.Google Scholar
- Yang B, Mitchell T. Leveraging Knowledge Bases in Lstms for Improving Machine Reading[C]: Association for Computational Linguistics (acl), 2017: 1436-1446.Google ScholarCross Ref
- Yang A, Wang Q, Liu J, Enhancing Pre-trained Language Representations with Rich Knowledge for Machine Reading Comprehension[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2346-2357.Google Scholar
- Seo M, Kembhavi A, Farhadi A, Bidirectional Attention Flow for Machine Comprehension[J]. Arxiv Preprint Arxiv:1611.01603, 2016.Google Scholar
- Yang B, Mitchell T. Leveraging Knowledge Bases in Lstms for Improving Machine Reading[C]: Association for Computational Linguistics (acl), 2017: 1436-1446.Google ScholarCross Ref
- Bird S, Loper E. Nltk: the Natural Language Toolkit[C]: Association for Computational Linguistics, 2004.Google Scholar
- Desai T, Moldovan DI. Towards Predicting Difficulty of Reading Comprehension Questions[C]//The Thirty-second International Flairs Conference, 2019.Google Scholar
- Song H, Yoon S, Park S. Question Difficulty Estimation Based on Attention Model for Question Answering[J]. Applied Sciences, 2021, 11(24): 12023.Google ScholarCross Ref
Index Terms
- Question Difficulty Prediction with External Knowledge
Recommendations
Employing Internal and External Knowledge to Factuality-Oriented Abstractive Summarization
Natural Language Processing and Chinese ComputingAbstractAbstractive summarization models based on neural network have successfully generated human-readable and fluent summaries. However, the generated summary often has factual errors: it is inconsistent with the facts included in the source document (...
A Study on External Knowledge, Knowledge Management Capability and Corporate Performance
SRII '12: Proceedings of the 2012 Annual SRII Global ConferenceIn general, it is difficult for enterprises to establish adequate knowledge transparency in their internal organization and to cope with the knowledge development occurring in the external environment. Furthermore, when the external knowledge is ...
KGAnet: a knowledge graph attention network for enhancing natural language inference
AbstractNatural language inference (NLI) is the basic task of many applications such as question answering and paraphrase recognition. Existing methods have solved the key issue of how the NLI model can benefit from external knowledge. Inspired by this, ...
Comments