Abstract
Construction of a mapping between educational content and skills is an important part of development of adaptive educational systems. This task is difficult, requires a domain expert, and any mistakes in the mapping may hinder the potential of an educational system. In this work we study techniques for improving a problem-skill mapping constructed by a domain expert using student data, particularly problem solving times. We describe and compare different techniques for the task – a multidimensional model of problem solving times and supervised classification techniques. In the evaluation we focus on surveying situations where the combination of expert opinion with student data is most useful.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Jarušek, P., Pelánek, R.: Analysis of a simple model of problem solving times. In: Cerri, S.A., Clancey, W.J., Papadourakis, G., Panourgia, K. (eds.) ITS 2012. LNCS, vol. 7315, pp. 379–388. Springer, Heidelberg (2012)
Jarušek, P., Pelánek, R.: A web-based problem solving tool for introductory computer science. In: Proc. of Innovation and technology in computer science education, pp. 371–371. ACM (2012)
Barnes, T.: The q-matrix method: mining student response data for knowledge. In: American Association for Artificial Intelligence 2005 Educational Data Mining Workshop (2005)
Desmarais, M.C., Beheshti, B., Naceur, R.: Item to skills mapping: deriving a conjunctive q-matrix from data. In: Cerri, S.A., Clancey, W.J., Papadourakis, G., Panourgia, K. (eds.) ITS 2012. LNCS, vol. 7315, pp. 454–463. Springer, Heidelberg (2012)
Tatsuoka, K.: Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement 20(4), 345–354 (1983)
Barnes, T.: Novel derivation and application of skill matrices: The q-matrix method. In: Handbook on Educational Data Mining (2010)
Boroš, P., Nižnan, J., Pelánek, R., Řihák, J.: Automatic detection of concepts from problem solving times. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 595–598. Springer, Heidelberg (2013)
De La Torre, J.: An empirically based method of q-matrix validation for the dina model: Development and applications. Journal of Educational Measurement 45(4), 343–362 (2008)
DeCarlo, L.T.: Recognizing uncertainty in the q-matrix via a bayesian extension of the dina model. Applied Psychological Measurement 36(6), 447–468 (2012)
Desmarais, M.C., Naceur, R.: A matrix factorization method for mapping items to skills and for enhancing expert-based Q-matrices. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 441–450. Springer, Heidelberg (2013)
Rupp, A., Templin, J.: The effects of q-matrix misspecification on parameter estimates and classification accuracy in the dina model. Educational and Psychological Measurement 68(1), 78–96 (2008)
Desmarais, M.C., Beheshti, B., Xu, P.: The refinement of a q-matrix: assessing methods to validate tasks to skills mapping. In: Proceedings of the 7th International Conference on Educational Data Mining, pp. 308–311 (2014)
Kantor, P., Ricci, F., Rokach, L., Shapira, B.: Recommender systems handbook. Springer (2010)
Waters, A., Lan, A., Studer, C.: Sparse probit factor analysis for learning analytics. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8776–8780 (2013)
Cen, H., Koedinger, K.R., Junker, B.: Learning factors analysis – a general method for cognitive model evaluation and improvement. In: Ikeda, M., Ashley, K.D., Chan, T.-W. (eds.) ITS 2006. LNCS, vol. 4053, pp. 164–175. Springer, Heidelberg (2006)
Lindsey, R.V., Khajah, M., Mozer, M.C.: Automatic discovery of cognitive skills to improve the prediction of student learning (2014), submitted for publication
Bootkrajang, J., Kabán, A.: Label-noise robust logistic regression and its applications. In: Flach, P.A., De Bie, T., Cristianini, N. (eds.) ECML PKDD 2012, Part I. LNCS, vol. 7523, pp. 143–158. Springer, Heidelberg (2012)
Natarajan, N., Dhillon, I., Ravikumar, P., Tewari, A.: Learning with noisy labels. In: Advances in Neural Information Processing Systems, pp. 1196–1204 (2013)
Nižnan, J., Pelánek, R., Řihák, J.: Using problem solving times and expert opinion to detect skills. In: Proceedings of the 7th International Conference on Educational Data Mining, pp. 433–434 (2014)
Van der Linden, W.: A lognormal model for response times on test items. Journal of Educational and Behavioral Statistics 31(2), 181 (2006)
Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009)
Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: Liblinear: A library for large linear classification. The Journal of Machine Learning Research 9, 1871–1874 (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Nižnan, J., Pelánek, R., Řihák, J. (2014). Mapping Problems to Skills Combining Expert Opinion and Student Data. In: Hliněný, P., et al. Mathematical and Engineering Methods in Computer Science. MEMICS 2014. Lecture Notes in Computer Science(), vol 8934. Springer, Cham. https://doi.org/10.1007/978-3-319-14896-0_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-14896-0_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-14895-3
Online ISBN: 978-3-319-14896-0
eBook Packages: Computer ScienceComputer Science (R0)