Skip to main content

Advertisement

SpringerLink
Log in

An Integrated Look at Middle School Engagement and Learning in Digital Environments as Precursors to College Attendance

  • Original Research
  • Published:
Technology, Knowledge and Learning Aims and scope Submit manuscript

Abstract

Middle school is an important phase in the academic trajectory, which plays a major role in the path to successful post-secondary outcomes such as going to college. Despite this, research on factors leading to college-going choices do not yet utilize the extensive fine-grained data now becoming available on middle school learning and engagement. This paper uses interaction-based data-mined assessments of student behavior, academic emotions and knowledge from a middle school online learning environment, and evaluates their relationships with different outcomes in high school and college. The data-mined measures of student behavior, emotions, and knowledge are used in three analyses: (1) to develop a prediction model of college attendance; (2) to evaluate their relationships to intermediate outcomes on the path to college attendance such as math and science course-taking during high school; (3) to develop an overall path model between the educational experiences students have during middle school, their high school experiences, and their eventual college attendance. This gives a richer picture of the cognitive and non-cognitive mechanisms that students experience throughout varied phases in their years in school, and how they may be related to one another. Such understanding may provide educators with information about students’ trajectories within the college pipeline.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

References

  • Aleven, V., Mclaren, B., Roll, I., & Koedinger, K. (2006). Toward meta-cognitive tutoring: A model of help seeking with a Cognitive Tutor. International Journal of Artificial Intelligence in Education, 16(2), 101–128.

    Google Scholar 

  • Arnold, K. E., & Pistilli, M. D. (2012, April). Course signals at Purdue: Using learning analytics to increase student success. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 267–270). ACM.

  • Arroyo, I., Ferguson, K., Johns, J., Dragon, T., Meheranian, H., Fisher, D., et al. (2007) Repairing disengagement with non-invasive interventions. In Proceedings of AIED 2007 (pp. 195–202).

  • Baker, R., & Siemens, G. (2014). Educational data mining and learning analytics. In K. Sawyer (Ed.), Cambridge handbook of the learning sciences (2nd ed., pp. 253–274).

  • Baker, R., Walonoski, J., Heffernan, N., Roll, I., Corbett, A., & Koedinger, K. (2008b). Why students engage in “gaming the system” behavior in interactive learning environments. Journal of Interactive Learning Research, 19(2), 185–224.

    Google Scholar 

  • Baker, R. S., Corbett, A. T., & Aleven, V. (2008a). More accurate student modeling through contextual estimation of slip and guess probabilities in bayesian knowledge tracing. In ITS 2008 (pp. 406–415).

  • Baker, R. S., Corbett, A. T., Gowda, S. M., Wagner, A. Z., MacLaren, B. A., Kauffman, L. R., et al. (2010a). Contextual slip and prediction of student performance after use of an intelligent tutor. In Proceedings of UMAP 2010 (pp. 52–63).

  • Baker, R. S., Corbett, A. T., Koedinger, K. R., Evenson, S. E., & Beck, J. (2006). Adapting to when students game an intelligent tutoring system. In Proceedings ITS 2006 (pp. 392–401).

  • Baker, R. S., Corbett, A. T., Koedinger, K. R., & Wagner, A. Z. (2004). Off-task behavior in the cognitive tutor classroom: When students game the system. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 383–390).

  • Baker, R. S., D'Mello, S. K., Rodrigo, M. M. T., & Graesser, A. C. (2010b). Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive–affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies, 68(4), 223–241.

    Article  Google Scholar 

  • Balfanz, R. (2009). Putting middle grades students on the graduation path: A policy and practice brief. Everyone Graduates Center & Talent Development Middle Grades Program.

  • Bandura, A. (1986). Social foundations of thought and action (pp. 5–107). Englewood Cliffs, NJ: Prentice Hall.

    Google Scholar 

  • Bowers, A. J. (2010). Grades and graduation: A longitudinal risk perspective to identify student dropouts. The Journal of Educational Research, 103(3), 191–207.

    Article  Google Scholar 

  • Cabrera, A. F. (1994). Logistic regression analysis in higher education: An applied perspective. Higher Education: Handbook of Theory and Research, 10, 225–256.

    Google Scholar 

  • Cabrera, A. F., & La Nasa, S. M. (2000). Understanding the college-choice process. New Directions for Institutional Research, 2000(107), 5–22.

    Article  Google Scholar 

  • Cabrera, A. F., La Nasa, S. M., & Burkum, K. R. (2001). Pathways to a four-year degree: The higher education story of one generation. Center for the Study of Higher Education.

  • Camblin, S. (2003). The middle grades: Putting all students on track for college. Honolulu, HI: Pacific Resources for Education and Learning.

    Google Scholar 

  • Canfield, W. (2001). ALEKS: A Web-based intelligent tutoring system. Mathematics and Computer Education, 35(2), 152–158.

    Google Scholar 

  • Carnevale, A. P., & Rose, S. J. (2003). Socioeconomic status, race/ethnicity, and selective college admissions. New York: Century Foundation.

    Google Scholar 

  • Clarke-Midura, J., & Dede, C. (2010). Assessment, technology, and change. Journal of Research on Technology in Education, 42(3), 309–328.

    Article  Google Scholar 

  • Clements, M. A. (1982). Careless errors made by sixth-grade children on written mathematical tasks. Journal for Research in Mathematics Education, 13, 136–144.

    Article  Google Scholar 

  • Cocea, M., Hershkovitz, A., & Baker, R. S. (2009). The impact of off-task and gaming behaviors on learning: Immediate or aggregate? In Proceedings of AIED 2009 (pp. 507–514).

  • Cohen, J. (1960). A coefficient of agreement for nominal scale. Educational and Psychological Measurement, 20, 37–46.

    Article  Google Scholar 

  • Conley, D. T. (2007). Redefining college readiness. Eugene: Educational Policy Improvement Center.

    Google Scholar 

  • Conley, D. T. (2008). College knowledge: What it really takes for students to succeed and what we can do to get them ready. Hoboken: Wiley.

    Google Scholar 

  • Conley, D., Lombardi, A., Seburn, M., & McGaughy, C. (2009). Formative assessment for college readiness: Measuring skill and growth in five key cognitive strategies associated with postsecondary success. Paper presented at the annual conference of the American Educational Research Association, San Diego, CA.

  • Corbett, A. T., & Anderson, J. R. (1995). Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4), 253–278.

    Article  Google Scholar 

  • Craig, S., Graesser, A., Sullins, J., & Gholson, B. (2004). Affect and learning: An exploratory look into the role of affect in learning with AutoTutor. Journal of Educational Media, 29(3), 241–250.

    Article  Google Scholar 

  • Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. New York: Harper-Row.

    Google Scholar 

  • D’Mello, S. K., Craig, S. D., Witherspoon, A., Mcdaniel, B., & Graesser, A. (2008). Automatic detection of learner’s affect from conversational cues. User Modeling and User-Adapted Interaction, 18(1–2), 45–80.

    Article  Google Scholar 

  • D’Mello, S. K., Lehman, B., Pekrun, R., & Graesser, A. C. (2014). Confusion can be beneficial for learning. Learning & Instruction, 29(1), 153–170.

    Article  Google Scholar 

  • DeFalco, J. A., Baker, R. S., Paquette, L., Georgoulas, V., Rowe, J., Mott, B., et al. (2015). Motivational feedback designs for frustration in a simulation-based combat medic training environment. In Generalized Intelligent Framework for Tutoring (GIFT) users symposium (p. 81).

  • Drummond, J., & Litman, D. (2010). In the zone: Towards detecting student zoning out using supervised machine learning. In Intelligent tutoring systems (pp. 306–308).

  • Eccles, J. S., & Jacobs, J. E. (1986). Social forces shape math attitudes and performance. Signs, 11(2), 367–380.

    Article  Google Scholar 

  • Eccles, J. S., Vida, M. N., & Barber, B. (2004). The relation of early adolescents’ college plans and both academic ability and task-value beliefs to subsequent college enrollment. The Journal of Early Adolescence, 24(1), 63–77.

    Article  Google Scholar 

  • Fancsali, S. (2014, July). Causal discovery with models: Behavior, affect, and learning in cognitive tutor algebra. In Educational data mining 2014.

  • Farrington, C. A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T. S., Johnson, D. W., et al. (2012). Teaching adolescents to become learners: The role of noncognitive factors in shaping school performance—A critical literature review. Consortium on Chicago School Research.

  • Feng, S., D’Mello, S., & Graesser, A. C. (2013). Mind wandering while reading easy and difficult texts. Psychonomic Bulletin & Review, 20(3), 586–592.

    Article  Google Scholar 

  • Feng, M., Heffernan, N., & Koedinger, K. (2009). Addressing the assessment challenge with an online system that tutors as it assesses. User Modeling and User-Adapted Interaction, 19(3), 243–266.

    Article  Google Scholar 

  • Finkelstein, N., Fong, A., Tiffany-Morales, J., Shields, P., & Huang, M. (2012). College bound in middle school & high school? How Math course sequences matter. Center for the Future of Teaching and Learning at WestEd.

  • Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of educational research, 74(1), 59–109.

    Article  Google Scholar 

  • Griffith, A. L., & Rothstein, D. S. (2009). Can’t get there from here: The decision to apply to a selective college. Economics of Education Review, 28(5), 620–628.

    Article  Google Scholar 

  • Hanley, J., & McNeil, B. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143, 29–36.

    Article  Google Scholar 

  • Hawn, A. (2015, March). The bridge report: Bringing learning analytics to low-income, urban schools. In Proceedings of the fifth international conference on learning analytics and knowledge (pp. 410–411). ACM.

  • Hershkovitz, A., Baker, R. S., Gowda, S. M., & Corbett, A. T. (2013). Predicting future learning better using quantitative analysis of moment-by-moment learning. In EDM (Vol. 13, pp. 74–81).

  • Hosmer, D. W., Jr., Lemeshow, S., & Sturdivant, R. X. (2013). Applied logistic regression (Vol. 398). Hoboken: Wiley.

    Book  Google Scholar 

  • Hossler, D., Braxton, J., & Coopersmith, G. (1989). Understanding student college choice. Higher education: Handbook of theory and research, 5, 231–288.

    Google Scholar 

  • Karweit, N., & Slavin, R. E. (1982). Time-on-task—Issues of timing, sampling, and definition. Journal of Educational Psychology, 74(6), 844–851.

    Article  Google Scholar 

  • Kellam, S. G., Ling, X., Merisca, R., Brown, C. H., & Ialongo, N. (1998). The effect of the level of aggression in the first grade classroom on the course and malleability of aggressive behavior into middle school. Development and Psychopathology, 10, 165–185.

    Article  Google Scholar 

  • Khajah, M., Lindsey, R. V., & Mozer, M. C. (2016). How deep is knowledge tracing? In T. Barnes, M. Chi, & M. Feng (Eds.), Proceedings of the ninth international conference on educational data mining (pp. 94–101). Educational Data Mining Society Press.

  • Koedinger, K. R., & Corbett, A. T. (2006). Cognitive tutors: Technology bringing learning science to the classroom. In K. Sawyer (Ed.), The Cambridge handbook of the learning sciences. Cambridge: Cambridge University Press.

    Google Scholar 

  • Kort, B., Reilly, R., Picard, R. (2001). An affective model of interplay between emotions and learning: reengineering educational pedagogy—Building a learning companion. In Proceedings IEEE ICALT 2001 (pp. 43–48).

  • Lehman, B., D’Mello, S., & Graesser, A. (2012). Interventions to regulate confusion during learning. In Intelligent tutoring systems (pp. 576–578). Berlin/Heidelberg: Springer.

  • Lent, R. W., Brown, S. D., & Hackett, G. (1994). Toward a unifying social cognitive theory of career and academic interest, choice and performance. Journal of Vocational Behavior, 45(1), 79–122.

    Article  Google Scholar 

  • Lent, R. W., Brown, S. D., & Hackett, G. (2000). Contextual supports and barriers to career choice: A social cognitive analysis. Journal of Counseling Psychology, 47(1), 36–49.

    Article  Google Scholar 

  • Madigan, T. (1997). Science proficiency and course taking in high school: The relationship of science course-taking patterns to increases in science proficiency between 8th and 12th grades. Washington, DC: National Center for Education Statistics (ED).

  • McQuiggan, S. W., Mott, B. W., & Lester, J. C. (2008). Modeling self-efficacy in intelligent tutoring systems: An inductive approach. User Modeling and User-Adapted Interaction, 18(1–2), 81–123.

    Article  Google Scholar 

  • Milliron, M. D., Malcolm, L., & Kil, D. (2014). Insight and action analytics: Three case studies to consider. Research & Practice in Assessment, 9, 70–89.

    Google Scholar 

  • Núñez, A. M., & Bowers, A. J. (2011). Exploring what leads high school students to enroll in hispanic-serving institutions a multilevel analysis. American Educational Research Journal, 48(6), 1286–1313.

    Article  Google Scholar 

  • Ocumpaugh, J., Baker, R., Gowda, S., Heffernan, N., & Heffernan, C. (2014). Population validity for educational data mining models: A case study in affect detection. British Journal of Educational Technology, 45(3), 487–501.

    Article  Google Scholar 

  • Ocumpaugh, J., Baker, R. S., Rodrigo, M. M. T., Salvi, A. van Velsen, M., Aghababyan, A., et al. (2015). HART: The human affect recording tool. In Proceedings of SIGDOC 2015.

  • Pardos, Z. A., Baker, R. S., San Pedro, M. O., Gowda, S. M., & Gowda, S. M. (2013). Affective states and state tests: Investigating how affect throughout the school year predicts end of year learning outcomes. In Proceedings of LAK 2013 (pp. 117–124).

  • Pekrun, R., Goetz, T., Daniels, L. M., Stupnisky, R. H., & Perry, R. P. (2010). Boredom in achievement settings: Exploring control-value antecedents and performance outcomes of a neglected emotion. Journal of Educational Psychology, 102(3), 531.

    Article  Google Scholar 

  • Pekrun, R., & Linnenbrink-Garcia, L. (2012). Academic emotions and student engagement. In Handbook of research on student engagement (pp. 259–282).

  • Razzaq, L., Feng, M., Nuzzo-Jones, G., & Rasmussen, K. P. (2005). The Assistment project: Blending assessment and assisting. In Proceedings of AIED 2015 (pp. 555–562).

  • Reinke, W. M., & Herman, K. C. (2002). Creating school environments that deter antisocial behaviours in youth. Psychology in the Schools, 39, 549–559.

    Article  Google Scholar 

  • Rock, D. A., Owings, J. A., & Lee, R. (1994). Changes in math proficiency between 8th and 10th grades. US Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics.

  • Roderick, M., Coca, V., & Nagaoka, J. (2011). Potholes on the road to college high school effects in shaping urban students’ participation in college application, four-year college enrollment, and college match. Sociology of Education, 84(3), 178–211.

    Article  Google Scholar 

  • Roderick, M., Nagaoka, J., & Coca, V. (2009). College readiness for all: The challenge for urban high schools. The Future of Children, 19(1), 185–210.

    Article  Google Scholar 

  • Rottinghaus, P. J., & Eshelman, A. J. (2015). Integrative approaches to career intervention. In APA handbook of career intervention, Vol 2: Applications (pp. 25–39).

  • Rowe, J. P., McQuiggan, S. W., Robison, J. L., & Lester, J. C. (2009). Off-task behavior in narrative-centered learning environments. In AIED 2009 (pp. 99–106).

  • Rozin, P., & Cohen, A. B. (2003). High frequency of facial expressions corresponding to confusion, concentration, and worry in an analysis of naturally occurring facial expressions of Americans. Emotion, 3(1), 68.

    Article  Google Scholar 

  • Sabourin, J., Rowe, J., Mott, B., & Lester, J. (2011). When off-task is on-task: The affective role of off-task behavior in narrative-centered learning environments. In Proceedings of artificial intelligence in education 2011 (pp. 534–536).

  • San Pedro, M. O., Baker, R. S., Bowers, A. J., & Heffernan, N. T. (2013). Predicting college enrollment from student interaction with an intelligent tutoring system in middle school. In Proceedings of EDM 2013 (pp. 177–184).

  • San Pedro, M. O., Baker, R. S., & Rodrigo, M. M. T. (2011). Detecting carelessness through contextual estimation of slip probabilities among students using an intelligent tutor for mathematics. In Proceedings of AIED 2011 (pp. 304–311).

  • Tze, V. M., Daniels, L. M., & Klassen, R. M. (2016). Evaluating the relationship between boredom and academic outcomes: A meta-analysis. Educational Psychology Review, 28(1), 119–144.

    Article  Google Scholar 

  • Vaessen, B. E., Prins, F. J., & Jeuring, J. (2014). University students’ achievement goals and help-seeking strategies in an intelligent tutoring system. Computers & Education, 72, 196–208.

    Article  Google Scholar 

  • Wentzel, K. R. (1993). Does being good make the grade? Social behavior and academic competence in middle school. Journal of Educational Psychology, 85(2), 357–364.

    Article  Google Scholar 

  • Witten, I. H., & Frank, E. (2005). Data mining: Practical machine learning tools and techniques. San Francisco: Morgan Kaufmann.

    Google Scholar 

Download references

Acknowledgements

This research was supported by Grants NSF #DRL-1031398, NSF #SBE-0836012, and Grant #OPP1048577 from the Bill and Melinda Gates Foundation.

Author information

Authors and Affiliations

  1. Research on Assessment and Learning, ACT, Inc., 500 Act Dr, Iowa City, IA, USA

    Maria Ofelia Z. San Pedro

  2. Teaching, Learning, and Leadership Division, University of Pennsylvania, Philadelphia, PA, USA

    Ryan S. Baker

  3. Department of Computer Science, Worcester Polytechnic Institute, Worcester, MA, USA

    Neil T. Heffernan

Authors
  1. Maria Ofelia Z. San Pedro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ryan S. Baker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Neil T. Heffernan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maria Ofelia Z. San Pedro.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

San Pedro, M.O.Z., Baker, R.S. & Heffernan, N.T. An Integrated Look at Middle School Engagement and Learning in Digital Environments as Precursors to College Attendance. Tech Know Learn 22, 243–270 (2017). https://doi.org/10.1007/s10758-017-9318-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10758-017-9318-z

Keywords

Search

Navigation