skip to main content
10.1145/2723576.2723616acmotherconferencesArticle/Chapter ViewAbstractPublication PageslakConference Proceedingsconference-collections
short-paper

An analysis of the impact of action order on future performance: the fine-grain action model

Published: 16 March 2015 Publication History

Abstract

To better model students' learning, user modelling should be able to use the detailed sequence of student actions to model student knowledge, not just their right/wrong scores. Our goal is to analyze the question: "Does it matter when a hint is used?". We look at students who use identical attempt counts to get the right answer and look for the impact of help use and action order on future performance. We conclude that students who use hints too early do worse than students who use hints later. However, students who use hints, at times, may perform as well as students who do not use hints. This paper makes a novel contribution showing for the first time that paying attention to the precise sequence of hints and attempts allows better prediction of students' performance, as well as to definitively show that, when we control for the number of attempts and hints, students that attempt problems before asking for hints show higher performance on the next question. This analysis shows that the pattern of hints and attempts, not just their numbers, is important.

References

[1]
Attali, Y., & Powers, D. (2010). Immediate feedback and opportunity to revise answers to open-ended questions. Educational and Psychological Measurement, 70(1), 22--35.
[2]
Beck, J. E., Chang, K., Mostow, J., & Corbett, A. (2008). Does help help? Introducing the Bayesian Evaluation and Assessment methodology. Intelligent Tutoring Systems. Springer Berlin Heidelberg.
[3]
Corbett, A. T., & Anderson, J. R. (1994). Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user-adapted interaction, 4(4), 253--278.
[4]
Duong, H. D., Zhu, L., Wang, Y., & Heffernan, N. T. (2013). A Prediction Model Uses the Sequence of Attempts and Hints to Better Predict Knowledge: Better to Attempt the Problem First, Rather Than Ask for a Hint. Paper Submitted to EDM.
[5]
Feng, M., Heffernan, N. T., & Koedinger, K. R. (2006). Predicting state test scores better with intelligent tutoring systems: developing metrics to measure assistance required. Intelligent Tutoring Systems. Springer Berlin Heidelberg.
[6]
Hawkins, W., Heffernan, N., Wang, Y., & Baker, R. S. Extending the Assistance Model: Analyzing the Use of Assistance over Time.
[7]
Murphy, K.: Bayes Net Toolbox for Matlab. < https://code.google.com/p/bnt/ > Accessed 4 September, 2014
[8]
Pardos, Z. A., & Heffernan, N. T. (2010). Modeling individualization in a bayesian networks implementation of knowledge tracing. User Modeling, Adaptation, and Personalization, 255--266.
[9]
Pavlik Jr, P. I., Cen, H., & Koedinger, K. R. (2009). Performance Factors Analysis--A New Alternative to Knowledge Tracing. Online Submission.
[10]
Qiu, Y., Pardos, Z. A., & Heffernan, N. T. (2012). Towards Data Driven Model Improvement. FLAIRS Conference.
[11]
Qiu, Y., Qi, Y., Lu, H., Pardos, Z. A., & Heffernan, N. T. (2011). Does Time Matter? Modeling the Effect of Time with Bayesian Knowledge Tracing. EDM.
[12]
Wang, Y., & Heffernan, N. T. (2012). Leveraging First Response Time into the Knowledge Tracing Model. International Educational Data Mining Society.
[13]
Wang, Y., & Heffernan, N. T. (2011). The "Assistance" Model: Leveraging How Many Hints and Attempts a Student Needs. FLAIRS Conference.
[14]
Zhu, L., Wang, Y., & Heffernan, N. T. The Sequence of Action Model: Leveraging the Sequence of Attempts and Hints.

Cited By

View all
  • (2023)Aktuelle Erkenntnisse aus der Literatur zu Data Mining und Learning Analytics im BildungsbereichEducational Data Mining und Learning Analytics10.1007/978-3-658-39607-7_1(1-39)Online publication date: 10-Jun-2023
  • (2022)A decade of learning analytics: Structural topic modeling based bibliometric analysisEducation and Information Technologies10.1007/s10639-022-11046-z27:8(10517-10561)Online publication date: 18-Apr-2022
  • (2020)Student performance analysis and prediction in classroom learning: A review of educational data mining studiesEducation and Information Technologies10.1007/s10639-020-10230-3Online publication date: 1-Jul-2020
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
LAK '15: Proceedings of the Fifth International Conference on Learning Analytics And Knowledge
March 2015
448 pages
ISBN:9781450334174
DOI:10.1145/2723576
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 March 2015

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. action order
  2. binning
  3. data mining
  4. hint use
  5. prediction of future success
  6. tabling

Qualifiers

  • Short-paper

Funding Sources

Conference

LAK '15

Acceptance Rates

LAK '15 Paper Acceptance Rate 20 of 74 submissions, 27%;
Overall Acceptance Rate 236 of 782 submissions, 30%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Aktuelle Erkenntnisse aus der Literatur zu Data Mining und Learning Analytics im BildungsbereichEducational Data Mining und Learning Analytics10.1007/978-3-658-39607-7_1(1-39)Online publication date: 10-Jun-2023
  • (2022)A decade of learning analytics: Structural topic modeling based bibliometric analysisEducation and Information Technologies10.1007/s10639-022-11046-z27:8(10517-10561)Online publication date: 18-Apr-2022
  • (2020)Student performance analysis and prediction in classroom learning: A review of educational data mining studiesEducation and Information Technologies10.1007/s10639-020-10230-3Online publication date: 1-Jul-2020
  • (2018)Predicting academic performance: a systematic literature reviewProceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education10.1145/3293881.3295783(175-199)Online publication date: 2-Jul-2018
  • (2018)The details matterUser Modeling and User-Adapted Interaction10.1007/s11257-018-9204-y28:3(207-235)Online publication date: 1-Aug-2018
  • (2016)Impact of data collection on interpretation and evaluation of student modelsProceedings of the Sixth International Conference on Learning Analytics & Knowledge10.1145/2883851.2883868(40-47)Online publication date: 25-Apr-2016

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media