skip to main content
10.1145/2883851.2883932acmotherconferencesArticle/Chapter ViewAbstractPublication PageslakConference Proceedingsconference-collections
short-paper

Forecasting student achievement in MOOCs with natural language processing

Published: 25 April 2016 Publication History

Abstract

Student intention and motivation are among the strongest predictors of persistence and completion in Massive Open Online Courses (MOOCs), but these factors are typically measured through fixed-response items that constrain student expression. We use natural language processing techniques to evaluate whether text analysis of open responses questions about motivation and utility value can offer additional capacity to predict persistence and completion over and above information obtained from fixed-response items. Compared to simple benchmarks based on demographics, we find that a machine learning prediction model can learn from unstructured text to predict which students will complete an online course. We show that the model performs well out-of-sample, compared to a standard array of demographics. These results demonstrate the potential for natural language processing to contribute to predicting student success in MOOCs and other forms of open online learning.

References

[1]
Bong, M. (2001). Role of self-efficacy and task-value in predicting college students' course performance and future enrollment intentions. Contemporary educational psychology, 26(4), 553--570.
[2]
Brooks, C., Thompson, C., & Teasley, S. (2015). A time series interaction analysis method for building predictive models of learners using log data. Paper presented at the Proceedings of the Fifth International Conference on Learning Analytics And Knowledge.
[3]
Eccles, J. S., Wigfield, A., & Schiefele, U. (1998). Motivation to succeed. In N. Eisenberg (Ed.), Handbook of child psychology (Vol. 4, pp. 1017--1095). New York: John Wiley & Sons.
[4]
Gilbert, D. T., & Wilson, T. D. (2007). Prospection: Experiencing the future. Science, 317(5843), 1351--1354.
[5]
Greene, J. A., Oswald, C. A., & Pomerantz, J. (2015). Predictors of Retention and Achievement in a Massive Open Online Course. American Educational Research Journal, 0002831215584621.
[6]
Halawa, S., Greene, D., & Mitchell, J. (2014). Dropout prediction in MOOCs using learner activity features. Experiences and best practices in and around MOOCs, 7.
[7]
Ho, A. D., Chuang, I., Reich, J., Coleman, C. A., Whitehill, J., Northcutt, C. G., ... Petersen, R. (2015). HarvardX and MITx: Two Years of Open Online Courses Fall 2012-Summer 2014.
[8]
Ho, A. D., Reich, J., Nesterko, S. O., Seaton, D. T., Mullaney, T., Waldo, J., & Chuang, I. (2014). HarvardX and MITx: The first year of open online courses, fall 2012-summer 2013. (HarvardX and MITx Working Paper No. 1).
[9]
Hulleman, C. S., Durik, A. M., Schweigert, S. B., & Harackiewicz, J. M. (2008). Task values, achievement goals, and interest: An integrative analysis. Journal of Educational Psychology, 100(2), 398.
[10]
Hulleman, C. S., Godes, O., Hendricks, B. L., & Harackiewicz, J. M. (2010). Enhancing interest and performance with a utility value intervention. Journal of Educational Psychology, 102(4), 880.
[11]
Joksimović, S., Dowell, N., Skrypnyk, O., Kovanović, V., Gašević, D., Dawson, S., & Graesser, A. C. (2015, March). How do you connect?: Analysis of social capital accumulation in connectivist MOOCs. In Proceedings of the Fifth International Conference on Learning Analytics And Knowledge (pp. 64--68). ACM.
[12]
Kizilcec, R., & Halawa, S. (2015). Attrition and Achievement Gaps in Online Learning. Proc. of ACM Learning at Scale, 15, 14--15.
[13]
Kloft, M., Stiehler, F., Zheng, Z., & Pinkwart, N. (2014). Predicting MOOC dropout over weeks using machine learning methods. Empirical Methods on Natural Language Processing (EMNLP) 2014, 60.
[14]
MET Project. (2012). Asking Student about Teaching.
[15]
Pennebaker, J. W., Francis, M. E., & Booth, R. J. (2001). Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates, 71, 2001.
[16]
Reich, J. (2014). MOOC completion and retention in the context of student intent. EDUCAUSE Review Online.
[17]
Reich, J., Tingley, D. H., Leder-Luis, J., Roberts, M. E., & Stewart, B. (2014). Computer-Assisted Reading and Discovery for Student Generated Text in Massive Open Online Courses.
[18]
Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., & Müller, M. (2011). pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC bioinformatics, 12(1), 77.
[19]
Taddy, M. (2013). Multinomial inverse regression for text analysis. Journal of the American Statistical Association, 108(503), 755--770.
[20]
Varma, S., & Simon, R. (2006). Bias in error estimation when using cross-validation for model selection. BMC bioinformatics, 7(1), 91.
[21]
Wen, M., Yang, D., & Rose, C. (2014). Sentiment Analysis in MOOC Discussion Forums: What does it tell us? Paper presented at the Educational Data Mining 2014.
[22]
Whitehill, J., Williams, J. J., Lopez, G., Coleman, C. A., & Reich, J. (2015). Beyond Prediction: First Steps Toward Automatic Intervention in MOOC Student Stopout.
[23]
Yang, D., Sinha, T., Adamson, D., & Rose, C. P. (2013). Turn on, tune in, drop out: Anticipating student dropouts in massive open online courses. Paper presented at the Proceedings of the 2013 NIPS Data-Driven Education Workshop.

Cited By

View all
  • (2025)Predicting learning performance using NLP: an exploratory study using two semantic textual similarity methodsKnowledge and Information Systems10.1007/s10115-024-02293-2Online publication date: 13-Feb-2025
  • (2024) Evaluating Sentence‐BERT ‐powered learning analytics for automated assessment of students' causal diagrams Journal of Computer Assisted Learning10.1111/jcal.1299240:6(2667-2680)Online publication date: 23-Apr-2024
  • (2024)Knowledge Navigator – Guiding You Through Your Learning Journey Using AI2024 International Conference on Computational Intelligence and Computing Applications (ICCICA)10.1109/ICCICA60014.2024.10584841(309-316)Online publication date: 23-May-2024
  • Show More Cited By

Index Terms

  1. Forecasting student achievement in MOOCs with natural language processing

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    LAK '16: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge
    April 2016
    567 pages
    ISBN:9781450341905
    DOI:10.1145/2883851
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 25 April 2016

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. MOOCS
    2. learning analytics
    3. motivation

    Qualifiers

    • Short-paper

    Conference

    LAK '16

    Acceptance Rates

    LAK '16 Paper Acceptance Rate 36 of 116 submissions, 31%;
    Overall Acceptance Rate 236 of 782 submissions, 30%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)51
    • Downloads (Last 6 weeks)6
    Reflects downloads up to 12 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Predicting learning performance using NLP: an exploratory study using two semantic textual similarity methodsKnowledge and Information Systems10.1007/s10115-024-02293-2Online publication date: 13-Feb-2025
    • (2024) Evaluating Sentence‐BERT ‐powered learning analytics for automated assessment of students' causal diagrams Journal of Computer Assisted Learning10.1111/jcal.1299240:6(2667-2680)Online publication date: 23-Apr-2024
    • (2024)Knowledge Navigator – Guiding You Through Your Learning Journey Using AI2024 International Conference on Computational Intelligence and Computing Applications (ICCICA)10.1109/ICCICA60014.2024.10584841(309-316)Online publication date: 23-May-2024
    • (2024)Innovations in Online Learning Analytics: A Review of Recent Research and Emerging TrendsIEEE Access10.1109/ACCESS.2024.349362112(166761-166775)Online publication date: 2024
    • (2024)Trends in NLP for personalized learning: LDA and sentiment analysis insightsEducation and Information Technologies10.1007/s10639-024-12988-2Online publication date: 28-Aug-2024
    • (2024)Educators’ motivations in massive open online courses for professional developmentEducation and Information Technologies10.1007/s10639-024-12590-629:14(1-25)Online publication date: 1-Oct-2024
    • (2024)Investigating engineering undergraduates' agentic and communal career values in writing responsesJournal of Engineering Education10.1002/jee.20584113:2(308-329)Online publication date: Feb-2024
    • (2023)Automated short answer grading using long short-term memory optimized with particle swarm optimizationi-manager's Journal on Data Science & Big Data Analytics10.26634/jds.1.2.203341:2(12)Online publication date: 2023
    • (2023)A Temporal Sequence Framework based on Self-Attention for Student Dropout Prediction in MOOCs2023 Eleventh International Conference on Advanced Cloud and Big Data (CBD)10.1109/CBD63341.2023.00045(211-216)Online publication date: 18-Dec-2023
    • (2023)Positive Artificial Intelligence in Education (P-AIED): A RoadmapInternational Journal of Artificial Intelligence in Education10.1007/s40593-023-00357-y34:3(732-792)Online publication date: 3-Aug-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media