Abstract
Educational interventions are often evaluated with randomized control trials, which can be very expensive to conduct. One of the promises of “Big Data” in education is to use non-experimental data to discover insights. We focus on studying the impact of example usage in a Java programming tutoring system using observational data. For this, we compare different formulations of a recently proposed generalized Knowledge Tracing framework called FAST. We discover that different formulations can have the same predictive performance; yet their coefficients may have opposite signs, which may lead researchers to contradictory conclusions. We discuss implications of using fully data-driven approaches to study non-experimental data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Atkinson, R.K., Derry, S.J., Renkl, A., Wortham, D.: Learning from examples: Instructional principles from the worked examples research. Review of Educational Research 70(2), 181–214 (2000)
Beck, J.E., Chang, K., Mostow, J., Corbett, A.T.: Does Help Help? Introducing the Bayesian Evaluation and Assessment Methodology. In: Woolf, B.P., A\"ımeur, E., Nkambou, R., Lajoie, S. (eds.) ITS 2008. LNCS, vol. 5091, pp. 383–394. Springer, Heidelberg (2008)
Corbett, A., Anderson, J.: Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction (1995)
González-Brenes, J., Huang, Y., Brusilovsky, P.: General features in knowledge tracing: Applications to multiple subskills, temporal item response theory, and expert knowledge. In: Educational Data Mining (2014)
Gu, J., Wang, Y., Heffernan, N.T.: Personalizing Knowledge Tracing: Should We Individualize Slip, Guess, Prior or Learn Rate? In: Trausan-Matu, S., Boyer, K.E., Crosby, M., Panourgia, K. (eds.) ITS 2014. LNCS, vol. 8474, pp. 647–648. Springer, Heidelberg (2014)
Hsiao, I.-H., Sosnovsky, S., Brusilovsky, P.: Guiding students to the right questions: adaptive navigation support in an e-learning system for java programming. Journal of Computer Assisted Learning 26(4), 270–283 (2010)
Huang, Y., González-Brenes, J., Kumar, R., Brusilovsky, P.: A framework for multifaceted evaluation of student models. In: Educational Data Mining (2015)
Najar, A.S., Mitrovic, A., McLaren, B.M.: Adaptive Support versus Alternating Worked Examples and Tutored Problems: Which Leads to Better Learning? In: Dimitrova, V., Kuflik, T., Chin, D., Ricci, F., Dolog, P., Houben, G.-J. (eds.) UMAP 2014. LNCS, vol. 8538, pp. 171–182. Springer, Heidelberg (2014)
Sao Pedro, M., Baker, R., Gobert, J.: Incorporating scaffolding and tutor context into bayesian knowledge tracing to predict inquiry skill acquisition. In: Educational Data Mining, Memphis, TN, pp. 185–192 (2013)
Schwonke, R., Wittwer, J., Aleven, V., Salden, R., Krieg, C., Renkl, A.: Can tutored problem solving benefit from faded worked-out examples. Proceedings of EuroCogSci 7, 59–64 (2007)
Velasquez, N.F., Goldin, I., Martin, T., Maughan, J.: Learning aid use patterns and their impact on exam performance in online developmental mathematics (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Huang, Y., González-Brenes, J.P., Brusilovsky, P. (2015). Challenges of Using Observational Data to Determine the Importance of Example Usage. In: Conati, C., Heffernan, N., Mitrovic, A., Verdejo, M. (eds) Artificial Intelligence in Education. AIED 2015. Lecture Notes in Computer Science(), vol 9112. Springer, Cham. https://doi.org/10.1007/978-3-319-19773-9_79
Download citation
DOI: https://doi.org/10.1007/978-3-319-19773-9_79
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-19772-2
Online ISBN: 978-3-319-19773-9
eBook Packages: Computer ScienceComputer Science (R0)