Skip to main content

Advertisement

SpringerLink
Log in

Detecting the Depth and Progression of Learning in Massive Open Online Courses by Mining Discussion Data

  • Original research
  • Published:
Technology, Knowledge and Learning Aims and scope Submit manuscript

Abstract

In massive open online courses (MOOCs), learners can interact with each other using discussion boards. Automatically inferring the states or needs of learners from their posts is of interest to instructors, who are faced with a high attrition in MOOCs. Machine learning has previously been successfully used to identify states such as confusion or posting questions, but no solution has yet been provided so that instructors can track the progress of the learners using a validated framework from education research. In this paper, we develop a model to automatically label a post based on the first phase of the interaction analysis model (IAM). This allows instructors to automatically identify whether students are stating opinions, clarifying details, or engaging in activities such as providing examples to peers. Our model is tested on a Coursera MOOC devoted to Chemistry, for which we are able to correctly categorize the IAM status in 4 out of 5 posts. Our approach thus provides instructors with an intelligent system that generates actionable learning assessment data and can cope with large enrollment. Using the system, instructors can quickly identify and remedy learning issues, thus supporting learners in attaining their intended outcomes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Agrawal, A., et al. (2015). YouEDU: Addressing confusion in MOOC discussion forums by recommending instructional video clips. Stanford InfoLab: In National Science Foundation.

    Google Scholar 

  • Almatrafi, O., Johri, A., & Rangwala, H. (2018). Needle in a haystack: identifying learner posts that require urgent response in MOOC discussion forums. Computers and Education, 118, 1–9.

    Article  Google Scholar 

  • Amemado, D., & Manca, S. (2017). Learning from decades of online distance education: MOOCs and the Community of Inquiry Framework. Journal of E-Learning and Knowledge Society, 13(2). https://www.learntechlib.org/p/180225/.

  • Arguello, J., & Shaffer, K. (2015). Predicting speech acts in MOOC forum posts. In Ninth international AAAI conference on web and social media.

  • Barak, M., Watted, A., & Haick, H. (2016). Motivation to learn in massive open online courses: Examining aspects of language and social engagement. Computers and Education, 94, 49–60.

    Article  Google Scholar 

  • Bonk, C. J., Zhu, M., Kim, M., Xu, S., Sabir, N., & Sari, A. R. (2018). Pushing toward a more personalized MOOC: Exploring instructor selected activities, resources, and technologies for MOOC design and implementation. International Review of Research in Open and Distributed Learning, 19(4). https://www.erudit.org/en/journals/irrodl/1900-v1-n1-irrodl04233/1055530ar/abstract/.

  • Chapman, S. A., Goodman, S., Jawitz, J., & Deacon, A. (2016). A strategy for monitoring and evaluating massive open online courses. Evaluation and Program Planning, 57, 55–63.

    Article  Google Scholar 

  • Chawla, N. V., et al. (2002). SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16, 321–357.

    Article  Google Scholar 

  • Cohen, A., Shimony, U., Nachmias, R., & Soffer, T. (2019). Active learners’ characterization in MOOC forums and their generated knowledge. British Journal of Educational Technology, 50(1), 177–198.

    Article  Google Scholar 

  • Crutzen, R., & Giabbanelli, P. J. (2014). Using classifiers to identify binge drinkers based on drinking motives. Substance Use and Misuse, 49(1–2), 110–115.

    Article  Google Scholar 

  • Darling-Hammond, L. (2013). Inequality and school resources: What it will take to close the opportunity gap. In P. Carter & K. Welner (Eds.), Closing the opportunity gap: What America must do to give every child an even chance (pp. 77–97). New York: Oxford University Press.

    Chapter  Google Scholar 

  • Fan, J., & Fan, Y. (2008). High dimensional classification using features annealed independence rules. Annals of Statistics, 36(6), 2605.

    Article  Google Scholar 

  • Giabbanelli, P. J., & Adams, J. (2016). Identifying small groups of foods that can predict achievement of key dietary recommendations: data mining of the UK National Diet and Nutrition Survey, 2008–2012. Public Health Nutrition, 19(9), 1543–1551.

    Article  Google Scholar 

  • Goggins, S., Galyen, K., Petakovic, E., & Laffey, J. (2016). Connecting performance to social structure and pedagogy as a pathway to scaling learning analytics in MOOCs: An exploratory study. Journal of Computer Assisted learning, 32(3), 244–266.

    Article  Google Scholar 

  • Gunawardena, C. N., et al. (1997). Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397–431.

    Article  Google Scholar 

  • He, J., Bailey, J., Rubinstein, B. I. P., & Zhang, R. (2015). Identifying at-risk students in massive open online courses. In Proceedings of the twenty-ninth AAAI conference on artificial intelligence (pp. 1749–1755).

  • Hood, N., Littlejohn, A., & Milligan, C. (2015). Context counts: How learners’ contexts influence learning in a MOOC. Computers and Education, 91, 83–91.

    Article  Google Scholar 

  • Japkowicz, N. (2003). Class imbalances: are we focusing on the right issue. In Workshop on learning from imbalanced data sets II (vol. 1723, p. 63).

  • Joachims, T. (1998). Text categorization with support vector machines: Learning with many relevant features. In European conference on machine learning (pp. 137–142). Springer.

  • Joo, Y. J., So, H. J., & Kim, N. H. (2018). Examination of relationships among students’ self-determination, technology acceptance, satisfaction, and continuance intention to use K-MOOCs. Computers and Education, 122, 260–272.

    Article  Google Scholar 

  • Jordan, K. (2014). Initial trends in enrolment and completion of massive open online courses. The International Review of Research in Open and Distributed Learning, 15(1), 133–160.

    Article  Google Scholar 

  • Kasch, J., Van Rosmalen, P., & Kalz, M. (2017). A framework towards educational scalability of open online courses. Journal of Universal Computer Science, 23(9), 845–867.

    Google Scholar 

  • Kennedy, G., Coffrin, C., De Barba, P., & Corrin, L. (2015). Predicting success: how learners’ prior knowledge, skills and activities predict MOOC performance. In Proceedings of the fifth international conference on learning analytics and knowledge (pp. 136–140). ACM.

  • Kizilcec, R. F., Pérez-Sanagustín, M., & Maldonado, J. J. (2017). Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses. Computers and Education, 104, 18–33.

    Article  Google Scholar 

  • Klusener, M., & Fortenbacher, A. (2015). Predicting students’ success based on forum activities in MOOCs. In 2015 IEEE 8th international conference on intelligent data acquisition and advanced computing systems (vol. 2, pp. 925–928).

  • Kuncheva, L. I. (2004). Combining pattern classifiers: methods and algorithms. Hoboken: Wiley.

    Book  Google Scholar 

  • Liu, W., Kidzinski, L., & Dillenbourg, P. (2016). Semiautomatic annotation of mooc forum posts. In State-of-the-art and future directions of smart learning (pp. 399–408). Singapore: Springer.

    Chapter  Google Scholar 

  • Lucas, M., Gunawardena, C., & Moreira, A. (2014). Assessing social construction of knowledge online: A critique of the interaction analysis model. Computers in Human Behavior, 30, 574–582.

    Article  Google Scholar 

  • Margaryan, A., Bianco, M., & Littlejohn, A. (2015). Instructional quality of massive open online courses (MOOCs). Computers and Education, 80, 77–83.

    Article  Google Scholar 

  • Nagrecha, S., Dillon, J. Z., & Chawla, N. V. (2017). MOOC dropout prediction: Lessons learned from making pipelines interpretable. In Proceedings of the 26th international conference on world wide web companion, WWW ‘17 Companion (pp. 351–359).

  • Reeves, T., & Bonk, C. (2015). MOOCs: Redirecting the quest for quality higher education for all. International Journal on E-Learning, 14(3), 385–399.

    Google Scholar 

  • Robinson, S. (2014). Simulation: the practice of model development and use. London: Palgrave Macmillan.

    Book  Google Scholar 

  • Rose, C.P., & Siemens, G. (2014) Shared task on prediction and dropout over time in massively open online courses. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 39–41. Doha, Qatar, October 25–29.

  • Rossi, L. A., & Gnawali, O. (2014). Language independent analysis and classification of discussion threads in coursera MOOC forums. In 15th international conference on information reuse and integration (IRI), 2014 IEEE (pp. 654–661). IEEE.

  • Rosso, N., & Giabbanelli, P.J. (2018) Accurately inferring compliance to five major food guidelines through simplified surveys: applying data mining to the UK National Diet and Nutrition Survey. JMIR Public Health and Surveillance, 4(2). https://publichealth.jmir.org/2018/2/e56/.

  • Shao, Z. (2018). Examining the impact mechanism of social psychological motivations on individuals’ continuance intention of MOOCs. Internet Research. https://www.emerald.com/insight/content/doi/10.1108/IntR-11-2016-0335/full/html.

  • Shapiro, H. B., Lee, C. H., Roth, N. E. W., Li, K., Çetinkaya-Rundel, M., & Canelas, D. A. (2017). Understanding the massive open online course (MOOC) student experience: An examination of attitudes, motivations, and barriers. Computers and Education, 110, 35–50.

    Article  Google Scholar 

  • Spector, J. (2014). Remarks on MOOCs and mini-MOOCs. Educational Technology Research and Development, 62(3), 385–392.

    Article  Google Scholar 

  • Spector, J. M. (2017). A critical look at MOOCs. In M. J. Kinshuk & M. K. Khribi (Eds.), Open education: From OERs to MOOCs (pp. 135–147). Berlin: Springer.

    Chapter  Google Scholar 

  • Tawfik, A. A., Reeves, T. D., Stich, A. E., Gill, A., Hong, C., McDade, J., et al. (2017). The nature and level of learner-learner inter-action in a chemistry massive open online course (MOOC). Journal of Computing in Higher Education, 29(3), 411–431.

    Article  Google Scholar 

  • Torres, W. J., & Beier, M. E. (2018). Adult development in the wild: The determinants of autonomous learning in a massive open online course. Learning and Individual Differences, 65, 207–217.

    Article  Google Scholar 

  • Veletsianos, G., & Shepherdson, P. (2016). A systematic analysis and synthesis of the empirical MOOC literature published in 2013–2015. The International Review of Research in Open and Distributed Learning. https://doi.org/10.19173/irrodl.v17i2.2448.

    Article  Google Scholar 

  • Wang, Y., & Baker, R. (2015). Content or platform: Why do students complete MOOCs. Journal of Online Learning and Teaching, 11(1), 17–30.

    Google Scholar 

  • Watted, A., & Barak, M. (2018). Motivating factors of MOOC completers: Comparing between university-affiliated students and general participants. The Internet and Higher Education, 37, 11–20.

    Article  Google Scholar 

  • Wise, A. F., Cui, Y., Jin, W. Q., & Vytasek, J. (2017). Mining for gold: Identifying content-related MOOC discussion threads across domains through linguistic modeling. The Internet and Higher Education, 32, 11–28.

    Article  Google Scholar 

  • Xing, W., Chen, X., Stein, J., & Marcinkowski, M. (2016). Temporal predication of dropouts in MOOCs: Reaching the low hanging fruit through stacking generalization. Computers in Human Behavior, 58, 119–129.

    Article  Google Scholar 

  • Zawacki-Richter, O., Bozkurt, A., Alturki, U., & Aldraiweesh, A. (2018). What research says about MOOCs–An explorative content analysis. International Review of Research in Open and Distributed Learning, 19(1).

Download references

Author information

Authors and Affiliations

  1. Loven Systems, 22260 Haggerty Rd, Northville, MI, 48167, USA

    Venkata Sai Pillutla

  2. Department of Instructional Design and Technology, University of Memphis, 406 E.C. Ball Hall, Memphis, TN, 37152-3570, USA

    Andrew A. Tawfik

  3. Department of Computer Science and Software Engineering, Miami University, 205 Benton Hall, 510 E High St, Oxford, OH, 45056, USA

    Philippe J. Giabbanelli

Authors
  1. Venkata Sai Pillutla
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrew A. Tawfik
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Philippe J. Giabbanelli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Philippe J. Giabbanelli.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pillutla, V.S., Tawfik, A.A. & Giabbanelli, P.J. Detecting the Depth and Progression of Learning in Massive Open Online Courses by Mining Discussion Data. Tech Know Learn 25, 881–898 (2020). https://doi.org/10.1007/s10758-020-09434-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10758-020-09434-w

Keywords

Search

Navigation