Skip to main content

Advertisement

SpringerLink
Log in

Cognitive Demand of Model Tracing Tutor Tasks: Conceptualizing and Predicting How Deeply Students Engage

  • Work-in-Progress
  • Published:
Technology, Knowledge and Learning Aims and scope Submit manuscript

Abstract

Model tracing tutors represent a technology designed to mimic key elements of one-on-one human tutoring. We examine the situations in which such supportive computer technologies may devolve into mindless student work with little conceptual understanding or student development. To analyze the support of student intellectual work in the model tracing tutor case, we adapt a cognitive demand framework that has been previously applied with success to teacher-guided mathematics classrooms. This framework is then tested against think-aloud data from students using a model tracing tutor designed to teach proportional reasoning skills in the context of robotics movement planning problems. Individual tutor tasks are coded for designed level of cognitive demand and compared to students’ enacted level of cognitive demand. In general, designed levels predicted how students enacted the tasks. However, just as in classrooms, student enactment was often at lower levels of demand than designed. Several contextual design features were associated with this decline. Implications for intelligent tutoring system design and research are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Alibali, M. W., & Nathan, M. J. (2012). Embodiment in mathematics teaching and learning: Evidence from learners’ and teachers’ gestures. Journal of the Learning Sciences, 21(2), 247–286.

    Article  Google Scholar 

  • Anderson, J. R. (1996). ACT: A simple theory of complex cognition. American Psychologist, 51(4), 355–365.

    Article  Google Scholar 

  • Baumert, J., Kunter, M., Blum, W., Voss, T., Jordan, A., Klusmann, U., et al. (2010). Teachers’ mathematical knowledge, cognitive activation in the classroom, and student progress. American Educational Research Journal, 47(1), 133–180.

    Article  Google Scholar 

  • Boaler, J., & Brodie, K. (2004). The importance of depth and breadth in the analysis of teaching: A framework for analyzing teacher questions. In Proceedings of the 26th meeting of the North American chapter of the international group for the psychology of mathematics education. Toronto, Ontario, Canada.

  • Boaler, J., & Staples, M. (2008). Creating mathematical futures through an equitable teaching approach: The case of Railside School. The Teachers College Record, 110(3), 608–645.

    Google Scholar 

  • Bosch, M., Chevallard, Y., & Gascón, J. (2006). Science or magic? The use of models and theories in didactics of mathematics. In M. Bosch (Eds.), Proceedings of the fourth congress of the European Society for research in mathematics education (pp. 1254–1263). Barcelona.

  • Boston, M. D., & Smith, M. S. (2009). Transforming secondary mathematics teaching: Increasing the cognitive demands of instructional tasks used in teachers’ classrooms. Journal for Research in Mathematics Education, 40(2),119–156.

  • Brophy, J. (2000). Teaching. Brussels: International Academy of Education.

    Google Scholar 

  • Carmel, E., Crawford, S., & Chen, H. (1992). Browsing in hypertext: A cognitive study. IEEE Transactions on Systems, Man, and Cybernetics, 22, 865–884.

    Article  Google Scholar 

  • Doyle, W. (1983). Academic work. Review of Educational Research, 53, 159–199.

    Article  Google Scholar 

  • Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (revised ed.). Cambridge, MA: MIT Press.

    Google Scholar 

  • Grouws, D., Smith, M. S., & Sztajn, P. (2004). The preparation an detaching practices of U.S. mathematics teachers: 1990–2000 mathematics assessments of the National Assessment of Educational Progress; Results and interpretations (pp. 221–269). Reston, VA: National Council of Teachers of Mathematics.

  • Harel, G., Behr, M., Lesh, R., & Post, T. (1994). Invariance of ratio: The case of children’s anticipatory scheme for constancy of taste. Journal for Research in Mathematics Education, 25, 324–345.

    Article  Google Scholar 

  • Henningsen, M., & Stein, M. K. (1997). Mathematical tasks and student cognition: Classroom-based factors that support and inhibit high-level mathematical thinking and reasoning. Journal for Research in Mathematics Education, 524–549.

  • Hiebert, J. (Ed.). (2013). Conceptual and procedural knowledge: The case of mathematics. London: Routledge.

    Google Scholar 

  • Hiebert, J., & Carpenter, T. (1992). Learning and teaching with understanding. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp. 65–97). New York: Macmillan.

    Google Scholar 

  • Hiebert, J., & Grouws, D. (2007). The effects of classroom mathematics teaching on students’ learning. In F. Lester (Ed.), Handbook of research on mathematics teaching and learning (pp. 371–404). Charlotte, NC: Information Age.

    Google Scholar 

  • Hiebert, J., & Wearne, D. (1993). Instructional tasks, classroom discourse, and students’ learning in second-grade arithmetic. American Educational Research Journal, 30(2), 393–425.

    Article  Google Scholar 

  • Huntley, M. A., Rasmussen, R. S., Villarubi, J., Sangtong, J., & Fey, J. T. (2000). Effects of standards-based mathematics education: A study of the Core-plus mathematics project algebra and functions strand. Journal for Research in Mathematics Education, 31(3), 328–361.

    Article  Google Scholar 

  • Jang, J., & Schunn, C. D. (2014). A framework for unpacking cognitive benefits of distributed complex visual displays. Journal of Experimental Psychology Applied 20(3), 260–269.

  • Jang, J., Schunn, C.D., & Nokes, T. J. (2011). Spatially distributed instructions improve learning outcomes and efficiency. Journal of Educational Psychology, 103(1), 60–72.

  • Kessler, A., Boston, M., & Stein, M. K. (2014). Conceptualizing Teacher’s Practices in Supporting Students’ Mathematical Learning in Computer-Directed Learning Environments. Report published in the proceedings of the 11th International Conference of the Learning Sciences, Boulder, CO.

  • Koedinger, K., & Corbett, A. (2006). Cognitive tutors: Technology bringing learning science to the classroom. In K. Sawyer (Ed.), The Cambridge handbook of the learning sciences. Cambridge, MA: Cambridge University Press.

    Google Scholar 

  • Lamon, S. J. (2007). Rational numbers and proportional reasoning: Toward a theoretical framework for research. Second Handbook of Research on Mathematics Teaching and Learning, 1, 629–667.

    Google Scholar 

  • Liu, A., & Schunn, C. D. (2014). Applying math onto mechanism: Investigating the relationship between mechanistic and mathematical understanding. Paper presented at the 36th annual meeting of the cognitive science society. Quebec City, Canada.

  • Lobato, J., Ellis, A. B., Charles, R., & Zbiek, R. (2010). Developing essential understanding of ratios, proportions & proportional reasoning: Grades 6–8. Reston, VA: National Council of Teachers of Mathematics.

    Google Scholar 

  • Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. (2014). Intelligent tutoring systems and learning outcomes: A meta-analysis. Journal of Educational Psychology, 106(4), 901–918.

    Article  Google Scholar 

  • Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis. Thousand Oaks, CA: Sage.

    Google Scholar 

  • National Council of Teachers of Mathematics. (1989). Curriculum and evaluation standards for school mathematics. Reston, VA: National Council of Teachers of Mathematics.

    Google Scholar 

  • National Research Council. (2004). On evaluating curricular effectiveness: Judging the quality of K-12 mathematics evaluations (Committee for a Review of the Evaluation Data on the Effectiveness of NSF-Supported and Commercially Generated Mathematics Curriculum Materials). Washington, DC: The National Academies Press.

  • O’Connor, M. C. (2001). Can any fraction be turned into a decimal? A case study of a mathematical group discussion. Educational Studies in Mathematics, 46, 143–185.

    Article  Google Scholar 

  • Paas, F. G. W. C., Renkl, A., & Sweller, J. (2004). Cognitive load theory: Instructional implications of the interaction between information structures and cognitive architecture. Instructional Science, 32, 1–8.

    Article  Google Scholar 

  • Ritter, S., Anderson, J. R., Keodinger, K. R., & Corbett, A. (2007). Cognitive tutor: Applied research in mathematics education. Psychonomic Bulletin & Review, 14(2), 249–255.

    Article  Google Scholar 

  • Rittle-Johnson, B., & Alibali, M. W. (1999). Conceptual and procedural knowledge of mathematics: Does one lead to the other? Journal of Educational Psychology, 91(1), 175.

    Article  Google Scholar 

  • Rittle-Johnson, B., Siegler, R. S., & Alibali, M. W. (2001). Developing conceptual understanding and procedural skill in mathematics: An iterative process. Journal of Educational Psychology, 93(2), 346.

    Article  Google Scholar 

  • Seidel, T., & Shavelson, R. (2007). Teaching effectiveness research in the past decade: The role of theory and research design in disentangling meta-analysis results. Review of Educational Research, 77(4), 454–499.

    Article  Google Scholar 

  • Senk, S. L., & Thompspn, D. R. (Eds.). (2003). Standards-based school mathematics curricula: What are they? What do students learn?. Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Silk, E. M., Higashi, R., & Schunn, C. D. (2011). Resources for robot competition success: Assessing math use in grade-school-level engineering design. In American Society for Engineering Education. American Society for Engineering Education.

  • Silk, E. M., & Schunn, C. D. (2011). Calculational versus mechanistic mathematics in propelling the development of physical knowledge. Paper presented at the 41st annual meeting of The Jean Piaget Society, Berkeley, CA.

  • Smith, M. S. (2000). Balancing old and new: An experienced middle school teacher’s learning in the context of mathematics instructional reform. The Elementary School Journal, 100(4), 351–375.

  • Stein, M. K., Grover, B. W., & Henningsen, M. (1996). Building student capacity for mathematical thinking and reasoning: An analysis of mathematical tasks used in reform classrooms. American Educational Research Journal, 33(2), 455–488.

    Article  Google Scholar 

  • Stein, M. K., & Lane, S. (1996). Instructional tasks and the development of student capacity to think and reason: An analysis of the relationship between teaching and learning in a reform mathematics project. Educational Research and Evaluation, 2(1), 50–80.

  • Stein, M. K., & Kaufman, J. H. (2010). Selecting and supporting the use of mathematics curricula at scale. American Educational Research Journal, 47(3), 663–693.

  • Stein, M. K., Smith, M. S., Henningsen, M. A., & Silver, E. A. (2009). Implementing standards-based mathematics instruction: A casebook for professional development. New York, NY: Teachers College Press, Columbia University.

  • Stigler, J. W., & Hiebert, J. (2004). Improving mathematics teaching. Educational Leadership, 61(5), 12–17.

    Google Scholar 

  • Tarr, J. E., Reys, R. E., Reys, B. J., Chávez, Ó., Shih, J., & Osterlind, S. J. (2008). The impact of middle-grades mathematics curricula and the classroom learning environment on student achievement. Journal for Research in Mathematics Education, 39, 247–280.

  • Thompson, P. W., & Saldanha, L. (2003). Fractions and multiplicative reasoning. In J. Kilpatrick, W. G. Martin, & D. Schifter (Eds.), A research companion to principles and standards for school mathematics. Reston, VA: National Council of Teachers of Mathematics.

    Google Scholar 

  • van Merrienboer, J. J. G., & Sweller, J. (2005). Cognitive load theory and complex learning: Recent developments and future directions. Educational Psychology Review, 17(2), 147–177.

    Article  Google Scholar 

  • VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46, 197–221.

    Article  Google Scholar 

  • VanLehn, K., Freedman, R., Jordan, P., Murray, C., Osan, R., Ringenberg, M., et al. (2000). Fading and deepening: The next steps for Andes and other model-tracing tutors. In G. Guathier, C. Frasson, & K. VanLehn (Eds.), Intelligent tutoring systems (pp. 474–483). Berlin: Springer.

    Chapter  Google Scholar 

  • VanLehn, K., Lynch, C., Schulze, K., Shapiro, J. A., Shelby, R., Taylor, L., et al. (2005). The Andes physics tutoring system: Lessons learned. International Journal of Artificial Intelligence in Education, 15(3), 147–204.

    Google Scholar 

  • Venezky, R. (1992). Textbooks in school and society. In P. Jackson. (Ed.), Handbook of research on curriculum (pp. 436–462). New York: Macmillan.

  • Weaver, R., & Junker, B. W. (2004). Model specification for cognitive assessment of proportional reasoning (No. 777). Department of Statistics Technical Report.

  • Wilson, M., & Lloyd, G. (2000). The challenge to share mathematical authority with students: High school teachers’ experiences reforming classroom roles and activities through curriculum implementation. Journal of Curriculum and Supervision, 15, 146–169.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Pittsburgh, Pittsburgh, PA, USA

    Aaron M. Kessler, Mary Kay Stein & Christian D. Schunn

Authors
  1. Aaron M. Kessler
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mary Kay Stein
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christian D. Schunn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aaron M. Kessler.

Additional information

This work was funded by grant DRL-1029404 from the National Science Foundation to the second and third authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kessler, A.M., Stein, M.K. & Schunn, C.D. Cognitive Demand of Model Tracing Tutor Tasks: Conceptualizing and Predicting How Deeply Students Engage. Tech Know Learn 20, 317–337 (2015). https://doi.org/10.1007/s10758-015-9248-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10758-015-9248-6

Keywords

Search

Navigation