Neil L. Zimmerman
ABSTRACT
An ongoing study is being run to ensure that the McGraw-Hill Education LearnSmart platform teaches students as efficiently as possible. The first step in doing so is to identify what Knowledge Components (KCs) exist in the content; while the content is tagged by experts, these tags need to be re-calibrated periodically.
LearnSmart courses are organized into chapters corresponding to those found in a textbook; each chapter can have anywhere from about a hundred to a few thousand questions. The KC extraction algorithms proposed by Barnes [1] and Desmarais et al [3] are applied on a chapter-by-chapter basis. To assess the ability of each mined q matrix to describe the observed learning, the PFA model of Pavlik et al [4] is fitted to it and a cross-validated AUC is calculated. The models are assessed based on whether PFA's predictions of student correctness are accurate.
Early results show that both algorithms do a reasonable job of describing student progress, but q matrices with very different numbers of KCs fit observed data similarly well. Consequently, further consideration is required before automated extraction is practical in this context.
- T. Barnes. The q-matrix method: Mining student response data for knowledge. In Proceedings of the AAAI-2005 Workshop on Educational Data Mining, pages 39--46. Assocation for Advancement of Artificial Intelligence, Jul 2005.Google Scholar
- A. T. Corbett and J. R. Anderson. Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user-adapted interaction, 4(4):253--278, 1994.Google Scholar
- M. C. Desmarais, B. Beheshti, and R. Naceur. Item to skills mapping: Deriving a conjunctive q-matrix from data. In International Conference on Intelligent Tutoring Systems, pages 454--463. Springer Science & Business Media, 2012. Google ScholarDigital Library
- P. I. Pavlik, H. Cen, and K. R. Koedinger. Performance factors analysis-a new alternative to knowledge tracing. In Proceedings of the 2009 Conference on Artificial Intelligence in Education: Building Learning Systems That Care: From Knowledge Representation to Affective Modelling, pages 531--538, Amsterdam, The Netherlands, The Netherlands, 2009. IOS Press. Google ScholarDigital Library
- F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825--2830, 2011. Google ScholarDigital Library
- K. K. Tatsuoka. Rule space: An approach for dealing with misconceptions based on item response theory. Journal of educational measurement, 20(4):345--354, 1983.Google Scholar
- Z. Zhang, T. Li, C. Ding, and X. Zhang. Binary matrix factorization with applications. In Seventh IEEE International Conference on Data Mining (ICDM 2007), pages 391--400. Institute of Electrical and Electronics Engineers, Oct 2007. Google ScholarDigital Library
Index Terms
- Mining knowledge components from many untagged questions
Recommendations
Knowledge component suggestion for untagged content in an intelligent tutoring system
ITS'12: Proceedings of the 11th international conference on Intelligent Tutoring SystemsTagging educational content with knowledge components (KC) is key to providing useable reports to teachers and for use by assessment algorithms to determine knowledge component mastery. With many systems using fine-grained KC models that range from ...
Cooperation between expert knowledge and data mining discovered knowledge: Lessons learned
Expert systems are built from knowledge traditionally elicited from the human expert. It is precisely knowledge elicitation from the expert that is the bottleneck in expert system construction. On the other hand, a data mining system, which ...
Mining negative generalized knowledge from relational databases
Attribute-oriented induction (AOI) is a useful data mining method for extracting generalized knowledge from relational data and users' background knowledge. Concept hierarchies can be integrated with the AOI method to induce multi-level generalized ...
Comments