skip to main content
10.1145/2876034.2893425acmconferencesArticle/Chapter ViewAbstractPublication Pagesl-at-sConference Proceedingsconference-collections
Work in Progress

Assessing Problem-Solving Process At Scale

Published: 25 April 2016 Publication History

Abstract

Authentic problem solving tasks in digital environments are often open-ended with ill-defined pathways to a goal state. Scaffolds and formative feedback during this process help learners develop the requisite skills and understanding, but require assessing the problem-solving process. This paper describes a hybrid approach to assessing process at scale in the context of the use of computational thinking practices during programming. Our approach combines hypothesis-driven analysis, using an evidence-centered design framework, with discovery-driven data analytics. We report on work-in-progress involving novices and expert programmers working on Blockly games.

References

[1]
Blikstein, P., Worsley, M., Piech, C., Sahami, M., Cooper, S., & Koller, D. 2014. Programming pluralism: Using learning analytics to detect patterns in the learning of computer programming. Journal of Learning Sciences, 23(4), 561--599.
[2]
Mislevy, R. J., Behrens, J. T., Dicerbo, K. E., & Levy, R. 2012. Design and discovery in educational assessment: Evidence-centered design, psychometrics, and educational data mining. JEDM, 4(1), 11--48.
[3]
Piech, C., Sahami, M., Huang, J., & Guibas, L. 2015. Autonomously Generating Hints by Inferring Problem Solving Policies. In Proc. of the 2nd ACM Conference on Learning@Scale. 195--204.
[4]
Rupp, A. A., Levy, R., Dicerbo, K. E., Sweet, S., Crawford, A. V., Caliço, T., et al. 2012. Putting ECD into Practice: The Interplay of Theory and Data in Evidence Models within a Digital Learning Environment. JEDM, 4(1), 49--110.
[5]
Segedy, J. R., Loretz, K. M., & Biswas, G. 2013. Model-driven assessment of learners in open-ended learning environments. In Proceedings of the 3rd Annual Conference on Learning Analytics and Knowledge (LAK '13). ACM, 202--204.

Cited By

View all
  • (2024)Analyzing Students Collaborative Problem-Solving Behaviors in Synergistic STEM+C LearningProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636912(540-550)Online publication date: 18-Mar-2024
  • (2024)Assessing implicit computational thinking in game‐based learning: A logical puzzle game studyBritish Journal of Educational Technology10.1111/bjet.1344355:5(2357-2382)Online publication date: 23-Feb-2024
  • (2021)A Systematic Literature Review of Gameful Feedback in Computer Science EducationInternational Journal of Information and Education Technology10.18178/ijiet.2021.11.10.155111:10(464-470)Online publication date: 2021
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
L@S '16: Proceedings of the Third (2016) ACM Conference on Learning @ Scale
April 2016
446 pages
ISBN:9781450337267
DOI:10.1145/2876034
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2016

Check for updates

Author Tags

  1. assessment
  2. evidence-centered design
  3. k-12 computing
  4. problem solving
  5. programming process

Qualifiers

  • Work in progress

Funding Sources

  • SRI International internal R&D grant.

Conference

L@S 2016
Sponsor:
L@S 2016: Third (2016) ACM Conference on Learning @ Scale
April 25 - 26, 2016
Scotland, Edinburgh, UK

Acceptance Rates

L@S '16 Paper Acceptance Rate 18 of 79 submissions, 23%;
Overall Acceptance Rate 117 of 440 submissions, 27%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)28
  • Downloads (Last 6 weeks)2
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Analyzing Students Collaborative Problem-Solving Behaviors in Synergistic STEM+C LearningProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636912(540-550)Online publication date: 18-Mar-2024
  • (2024)Assessing implicit computational thinking in game‐based learning: A logical puzzle game studyBritish Journal of Educational Technology10.1111/bjet.1344355:5(2357-2382)Online publication date: 23-Feb-2024
  • (2021)A Systematic Literature Review of Gameful Feedback in Computer Science EducationInternational Journal of Information and Education Technology10.18178/ijiet.2021.11.10.155111:10(464-470)Online publication date: 2021
  • (2021)Applying Game Learning Analytics to a Voluntary Video Game: Intrinsic Motivation, Persistence, and Rewards in Learning to Program at an Early AgeIEEE Access10.1109/ACCESS.2021.31104759(123588-123602)Online publication date: 2021
  • (2021)How do students develop computational thinking? Assessing early programmers in a maze-based online gameComputer Science Education10.1080/08993408.2021.1903248(1-31)Online publication date: 13-Apr-2021
  • (2021)Measuring and Analyzing Students’ Strategic Learning Behaviors in Open-Ended Learning EnvironmentsInternational Journal of Artificial Intelligence in Education10.1007/s40593-021-00275-x32:4(931-970)Online publication date: 24-Sep-2021
  • (2019)Computational Thinking and Robotics: A Teaching Experience in Compulsory Secondary Education with Students with High Degree of Apathy and DemotivationSustainability10.3390/su1118510911:18(5109)Online publication date: 18-Sep-2019
  • (2019)Combining Assessment Tools for a Comprehensive Evaluation of Computational Thinking InterventionsComputational Thinking Education10.1007/978-981-13-6528-7_6(79-98)Online publication date: 3-May-2019
  • (2018)Improving Assessment of Computational Thinking Through a Comprehensive FrameworkProceedings of the 18th Koli Calling International Conference on Computing Education Research10.1145/3279720.3279735(1-5)Online publication date: 22-Nov-2018
  • (2017)A Framework for Using Hypothesis-Driven Approaches to Support Data-Driven Learning Analytics in Measuring Computational Thinking in Block-Based Programming EnvironmentsACM Transactions on Computing Education10.1145/310591017:3(1-25)Online publication date: 28-Aug-2017

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media