Elsevier

Computers & Education

Volume 110, July 2017, Pages 35-50
Computers & Education

Understanding the massive open online course (MOOC) student experience: An examination of attitudes, motivations, and barriers

https://doi.org/10.1016/j.compedu.2017.03.003Get rights and content

Highlights

  • Sentiment analysis revealed that participants were generally positive in statements about the courses and MOOCs in general.

  • Bachelor's degree learners were the most positive: a highly statistically significant result.

  • Knowledge, work, convenience, and personal interest were the most prevalent motivations.

  • Most interviewees were ambivalent about the certificates.

  • Lack of time was most common barrier; others were previous bad experience & inadequate background.

Abstract

During the widespread development of open access online course materials in the last two decades, advances have been made in understanding the impact of instructional design on quantitative outcomes. Much less is known about the experiences of learners that affect their engagement with the course content. Through a case study employing text analysis of interview transcripts, we revealed the authentic voices of participants and gained a deeper understanding of motivations for and barriers to course engagements experienced by students participating in Massive Open Online Courses (MOOCs). We sought to understand why learners take the courses, specifically Introduction to Chemistry or Data Analysis and Statistical Inference, and to identify factors both inside and outside of the course setting that impacted engagement and learning. Thirty-six participants in the courses were interviewed, and these students varied in age, experience with the subject matter, and worldwide geographical location. Most of the interviewee statements were neutral in attitude; sentiment analysis of the interview transcripts revealed that 80 percent of the statements that were either extremely positive or negative were found to be positive rather than negative, and this is important because an overall positive climate is known to correlate with higher academic achievement in traditional education settings. When demographic data was added to the sentiment analysis, students who have already earned bachelor's degrees were found to be more positive about the courses than students with either more or less formal education, and this was a highly statistically significant result. In general, students from America were more critical than students from Africa and Asia, and the sentiments of female participants' comments were generally less positive than those of male participants. An examination of student statements related to motivations revealed that knowledge, work, convenience, and personal interest were the most frequently coded nodes (more generally referred to as “codes”). On the other hand, lack of time was the most prevalently coded barrier for students. Other barriers and challenges cited by the interviewed learners included previous bad classroom experiences with the subject matter, inadequate background, and lack of resources such as money, infrastructure, and internet access. These results are enriched by illustrative quotes from interview transcripts and compared and contrasted with previous findings reported in the literature, and thus this study enhances the field by providing the voices of the learners.

Introduction

Massive Open Online Courses (MOOCs) are large-scale, open-access classes taught by university faculty via the internet using a variety of techniques such as weekly lecture videos/webcasts, online assessments, discussion forums, and even live video chat discussions and help sessions. Since the coinage of the term MOOC in 2008, this new avenue for education has raised a great deal of excitement and controversy in academia; these discussions have been recently chronicled and reviewed (Ebben and Murphy, 2014, Rhoads et al., 2015). Some assert that these online courses carry the potential to revolutionize the boundaries of modern learning by making high-quality education available to a much more vast pool of students (Waldrop, 2013). To this end, early instructors using this format enthusiastically proclaimed “…there's no reason to limit the geographical boundaries. Anywhere there is an Internet connection, students can log-on to learn and get help” (Moore & Janowicz, 2009, p. 4). MOOCs can also serve as an outlet for worldwide university outreach, expanding avenues for providing free, credible information to the general public. For example, some suggest that MOOCs will improve science and health literacy awareness and discussion among the public (Goldberg et al., 2015, Leontyev and Baranov, 2013). The power of a MOOC can be realized “especially on taboo subjects such as acquired immunodeficiency syndrome (AIDS), tuberculosis, and contraception” (Liyanagunawardena & Williams, 2014, p. 11). In cases where rapidly emerging fields are led by a small group of specialized experts, such as pharmacogenomics, MOOCs offer a route to efficiently improve in depth education for a larger group of students than can be handled by individual faculty at all institutions (Ma, Lee, & Kuo, 2013). In professions where continuing education is desirable or required, MOOCs also provide an efficient venue for adult working professionals to gain new skills or stay current with new developments in their field. To this end, several MOOCs has been certified for the purpose of providing continuing education to adults in professional fields such as K-12 teaching (Vivian, Falkner, & Falkner, 2014), and physicians are beginning to promote MOOCs for continuing medical education, particularly for medical practitioners in remote locations (Murphy & Monk, 2013). Clearly MOOCs have tremendous potential for the promotion of life-long learning beyond the traditional classroom.

On the other hand, not all academics welcome MOOCs with open arms and view the possible future scenarios through such rose-colored glasses. Some educators fear that a rapid proliferation of MOOCs could compromise the quality of learning and lead to a deterioration of the post-secondary education system. These critics point to the importance of face-to-face classroom engagement, laboratory, clinical, or fieldwork, and other aspects of the college experience outside of the purview of formal coursework that would be difficult or impossible to replicate online (Cooper and Sahami, 2013, Harder, 2013, Martin, 2012, McNutt, 2013). While most will now admit that online resources and online learning have a rapidly expanding place in higher education, the current common platform configurations (Fidalgo-Blanco, Sein-Echaluce Lacleta, & García-Peñalvo, 2015) and other limitations to synchronous, hands-on, and face-to-face experiences mean that MOOCs cannot so easily replace higher education as we know it today. Others point to the relatively advanced education levels (Emanuel, 2013) or high socioeconomic status (Hansen & Reich, 2015) of a large percentage of early participants to dampen the claim that MOOCs are the solution to widening access to education. Moreover, since the overwhelming majority of MOOCs constructed to date are in English, language access can be added to the technology access barrier for many populations (Liyanagunawardena & Williams, 2014), although the landscape in this regard is rapidly evolving. Another set of concerns cited revolves around the potential impact of MOOCs on academic life. Opponents of the MOOC movement point out that they contribute to the casualization of academic labor and threaten current institutions of higher education (Kolowich, 2013, Rhoads et al., 2015). Certainly the impact of MOOCs on the economic models of higher education constitutes a subject of intense interest (Hollands and Tirthali, 2014, Hoxby, 2014), and some find the uncertainty in that arena unsettling. These and other ethical implications of MOOCs, including cheating/plagiarism or research ethics involved for the large datasets produced by human subjects, have been recently explored (Marshall, 2014).

Distance, online, and other forms of e-learning have a rich history of research, reviews of that research, and development of models and frameworks (Arbaugh et al., 2009, Bernard et al., 2004, Childs et al., 2005, Gikandi et al., 2011, Leacock and Nesbit, 2007, Lee and Choi, 2011, Roca et al., 2006, U.S. Department of Education, 2010, Zawacki-Richter et al., 2009). However, MOOCs and research specifically related to MOOC pedagogies and learner outcomes are relatively recent developments in this arena. The MOOC research literature prior to 2012 has been reviewed elsewhere (Liyanagunawardena, Adams, & Williams, 2013).

Researchers from multiple disciplines have studied students' demographics, their performances, MOOC retention rates, best practices for course design and pedagogy, etc. Most studies about MOOCs to date have employed primarily quantitative or mixed-methods, such as analysis of course statistics and student survey data. For example, some of the early published research examined student learning in edX's first MOOC by evaluating course component access and completion, time spent on each resource online, scores on assignments, persistence, and some student demographic and survey data such as location, age, and selected reasons for enrolling (Breslow et al., 2013, DeBoer et al., 2013). More recent studies have used survey data to explore education research topics such as students' self-regulated learning behaviors in the context of MOOCs (Hood, Littlejohn, & Milligan, 2015). Research into innovative course designs is beginning to show promise in increasing course completion rates (Fidalgo-Blanco et al., 2015, Fidalgo-Blanco et al., 2016).

Researchers examining the effects of MOOCs on participants who cannot afford formal post-secondary education found that these learners were much less likely to have a college degree and were much more likely than a comparison group to enroll in a MOOC due to reasons of geographic isolation (Dillahunt, Wang, & Teasley, 2014). Students who self-reported that they could not afford a formal college education were more likely than the comparison group to be using a MOOC to see if they wanted to enroll in a more formal college course and were also more likely to be awarded a certificate of achievement (Dillahunt et al., 2014).

Researchers examining the pedagogies employed in a cross-section of MOOCs determined that an objectivist-individual approach (Arbaugh & Benbunan-Fich, 2006) was the most common framework and was used in all 24 MOOCs examined (Toven-Lindsey, Rhoads, & Lozano, 2015). They conclude that MOOC creators should “ …strive for more creative and empowering forms of open online learning” (Toven-Lindsey et al., 2015, p. 1).

Qualitative work in MOOC research is just beginning. The affective domain has been investigated through qualitative methods examining student writing in assignments and on discussion forums in MOOCs (Comer, Clark, & Canelas, 2014). Motivation to learn in MOOCs has recently been studied from various perspectives, such as by examining aspects of language and social engagement (Barak, Watted, & Haick, 2016). Finally, novel research methods such as blog mining have been developed to gauge the tenor of recent online discussions of MOOCs in various nontraditional publication outlets (Chen, 2014).

Motivation plays a key role in persistence and the extent of learning in all education environments, and there is a rich body of literature on the complex relationships between students' motivations, attitudes, and levels of engagement in a variety of learning contexts. Motivation theory is often invoked to explain why individuals choose to participate in certain tasks and their related effort level (Bandura, 1989a, Bandura, 1989b, Graham and Weiner, 1996, Keller, 1979). Participation in MOOCs currently falls heavily into the category of “voluntary learning,” so therefore it follows that motivation is especially important in determining individual differences in both total time spent on learning and effort intensity (Lei, 2010). While reviewing the rich history of the development of motivation theory in various contexts is beyond the scope of this contribution, interested readers are referred to reviews of motivation studies in online learning, in general (Bekele, 2010, Hart, 2012), and a recent detailed discussion of motivation research and its application to MOOCs, in particular (Barak et al., 2016, Ferguson and Clow, 2015, Kizilcec and Schneider, 2015).

Herein, we probe the future of online education through mixed-methods research. This involved analyzing surveys and interview transcript data from semi-structured interviews of participants in two Coursera courses, Introduction to Chemistry and Data Analysis and Statistical Inference, to gain a deeper understanding of the MOOC student experience. Our initial research questions were very broad:

  • 1.

    Who are the students taking MOOCs?

  • 2.

    What motivates students to take MOOCs?

  • 3.

    What can we learn from students that may help improve the MOOC experience?

We aimed to use semi-structured interviews and qualitative coding techniques to probe more deeply into the relationship between motivation and engagement and explore emerging themes that to date have been only examined in a cursory way through surveys. For example, what challenges do participants face in the effort to have full engagement with and deep learning of the course materials? By understanding the motivations for students to take MOOCs, as well as the challenges one might face throughout such courses, we aim to provide insight into a poorly studied topic, providing a foundation for further research. Ultimately, such research could lead to platforms and pedagogies that maximize the number of positive student experiences and provide a better learning environment for those who choose to enroll.

This project examines students' motivations and perceived barriers and challenges by interviewing students in the two courses. The project can contribute to the literature on MOOC students' motivation and provide deeper understandings of how MOOC designers can help learners overcome their perceived barriers in taking the course.

The Introduction to Chemistry course is taught by the sixth listed author of one of the submitting institutions. The session-based course seeks to reach students with little to no background in the subject in order to prepare the students for further study in chemistry, which is needed for many science, health, and policy professions. Students are introduced to chemical problem solving involving topics such as atoms, molecules, ions, the periodic table, stoichiometry, chemical reactions, bonding, thermochemistry, and gas laws. The course features videos, discussion forums, problem sets, quizzes, and exams as well as an optional writing project assessed by peers. It was adapted from a campus-based course described in detail elsewhere (Canelas, 2015, Hall et al., 2014). Since the completion of the data acquisition for this study, the course has been split into two shorter courses: Introduction to Chemistry: Reactions and Ratios and Introduction to Chemistry: Structures and Solutions.

The fifth listed author of one of the submitting institutions teaches the Data Analysis and Statistical Inference course. The session-based course aims to introduce students with little to no experience to the discipline of statistics while learning how to collect data, how to analyze data, and how to use data to make inferences and conclusions about real world phenomena. The course features videos, quizzes, and exams as well as data analysis labs in the R statistical language and an independent data analysis project assessed by peers. Since the completion of the data collection for this study, this course has also been split into a series of courses that make up the “Statistics with R” Coursera specialization.

Section snippets

Theoretical framework

This work is comprised of mixed-methods research guided by the literature and scholarship in motivation theory, e-learning, and distance education theory (Bernard et al., 2004, Sun et al., 2008). We draw inspiration from the e-learning theoretical framework outlined by Aparicio and coworkers, who use an overlapping domains conceptual framework consisting of people (stakeholders), e-learning technologies including those designed for content and communication, and e-learning activities including

Data collection and description of sample

Participants were recruited from both courses through announcements on the Coursera course sites, email messages, and posts on the courses' Facebook pages (Appendix A). The first stage of the interview subject recruitment was a survey in which students were asked about their spoken English fluency and whether or not they were willing to participate in an interview (Appendix B, document 1). At the time the survey was sent out, the Statistics course had roughly 10,000 students participating on a

Sentiment analysis

The sentiment analysis extracted 10,954 total interviewee statements: 6183 from participants in the chemistry course, and 3576 from participants in the statistics course. This analysis of the students' transcripts showed that about 80 percent of the comments were neutral (8993 neutral comments, see Fig. 1). That means that, in a particular statement, there were just as many positive phrases as there were negative. Overall, the average sentiment score was about 0.162, which means that the

Conclusions

The authentic voice of the learner is extremely important in education research, and our study recognizes this value and delves into an analysis of learner statements about their experiences in MOOCs. Quantitative analysis of learner interview transcripts revealed that the attitude of the interviewee statements was more positive than negative. This result indicates that the MOOCs could offer a constructive learning environment with manageable levels of frustration. Learners who had already

Acknowledgements

We would like to acknowledge the work of the coding team: in addition to the study authors, Maria Elena Carvajal, Joel Herndon, and Anthony Weishampel are thanked for their coding work on the project and participation in the coding workshop. Charlotte Clark is thanked for providing feedback to the second author on the development of the semi-structured interview guide and for suggesting expert consultants for organizing the coding workshop. We thank Lynne O'Brien for her leadership in Duke's

References (79)

  • B. Toven-Lindsey et al.

    Virtually unlimited classrooms: Pedagogical practices in massive open online courses

    Internet and Higher Education

    (2015)
  • T. Anderson

    Towards a theory of online learning

  • M. Aparicio et al.

    An e-learning theoretical framework

    Educational Technology & Society

    (2016)
  • J. Arbaugh et al.

    An investigation of epistemological and social dimensions of teaching in online learning environments

    Academy of Management Learning and Education

    (2006)
  • J.B. Arbaugh et al.

    Technological and structural characteristics, student learning and satisfaction with web-based courses: An exploratory study of two on-line MBA programs

    Management Learning

    (2002)
  • A. Bandura

    Social cognitive theory

  • A. Bandura

    A social cognitive theory of action

  • T.A. Bekele

    Motivation and satisfaction in internet-supported learning environments: A review

    Journal of Educational Technology & Society

    (2010)
  • V. Belanger et al.

    Bioelectricity: A quantitative approach

    (2013)
  • R. Benbunan-Fich et al.

    Mediators of the effectiveness of online courses

    IEEE Transactions on Professional Communication

    (2003)
  • R.M. Bernard et al.

    How does distance education compare with classroom instruction? A meta-analysis of the empirical literature

    Review of Educational Research

    (2004)
  • Breen, J. (2014)....
  • L. Breslow et al.

    Studying learning in the worldwide classroom: Research into edX's first MOOC

    Research and Practice in Assessment

    (2013)
  • D.A. Canelas

    Teaching college chemistry to the edges rather than to the average: Implications for less experienced science students

  • Y. Chen

    Investigating MOOCs through blog mining

    International Review of Research in Open and Distributed Learning

    (2014)
  • S. Childs et al.

    Effective e-learning for health professionals and students—barriers and their solutions. A systematic review of the literature—findings from the HeXL project

    Health Information & Libraries Journal

    (2005)
  • J. Cohen et al.

    School Climate: Research, policy, practice, and teacher education

    Teachers College Record

    (2009)
  • D.K. Comer et al.

    Writing to learn and learning to write across the disciplines: Peer-to-peer writing in introductory level MOOCs

    International Review of Research in Open and Distributed Learning

    (2014)
  • S. Cooper et al.

    Reflections on Stanford's MOOCs

    Communications of the ACM

    (2013)
  • DeBoer, J., Stump, G. S., Seaton, D., Breslow, L. (2013). Diversity in MOOC students’ backgrounds and behaviors in...
  • J. DeBoer et al.

    Changing “course”: Reconceptualizing educational variables for massive open online courses

    Educational Researcher

    (2014)
  • T. Dillahunt et al.

    Democratizing higher Education: Exploring MOOC use among those who cannot afford a formal education

    International Review of Research in Open and Distributed Learning

    (2014)
  • P. Diver et al.

    MOOCs as a massive research laboratory: Opportunities and challenges

    Distance Education

    (2015)
  • M. Ebben et al.

    Unpacking MOOC scholarly discourse: A review of nascent MOOC scholarship

    Learning, Media, and Technology

    (2014)
  • E.J. Emanuel

    Online education: MOOCs taken by educated few

    Nature

    (2013)
  • D. Engle et al.

    Coursera's introductory human physiology course: Factors that characterize successful completion of a MOOC

    International Review of Research in Open and Distributed Learning

    (2015)
  • R. Ferguson et al.

    Examining engagement: Analysing learner subpopulations in massive open online courses (MOOCs)

  • Á. Fidalgo-Blanco et al.

    Methodological approach and technological framework to break the current limitations of MOOC model

    Journal of Universal Computer Science

    (2015)
  • Á. Fidalgo-Blanco et al.

    From massive access to cooperation: Lessons learned and proven results of a hybrid xMOOC/cMOOC pedagogical approach to MOOCs

    International Journal of Educational Technology in Higher Education

    (2016)
  • Cited by (232)

    View all citing articles on Scopus
    1

    Current address: QSR International, 55 Cambridge Street, Burlington, MA 01803, United States.

    View full text