skip to main content
10.1145/2724660.2724681acmconferencesArticle/Chapter ViewAbstractPublication Pagesl-at-sConference Proceedingsconference-collections
research-article
Open access

Learning is Not a Spectator Sport: Doing is Better than Watching for Learning from a MOOC

Published: 14 March 2015 Publication History

Abstract

The printing press long ago and the computer today have made widespread access to information possible. Learning theorists have suggested, however, that mere information is a poor way to learn. Instead, more effective learning comes through doing. While the most popularized element of today's MOOCs are the video lectures, many MOOCs also include interactive activities that can afford learning by doing. This paper explores the learning benefits of the use of informational assets (e.g., videos and text) in MOOCs, versus the learning by doing opportunities that interactive activities provide. We find that students doing more activities learn more than students watching more videos or reading more pages. We estimate the learning benefit from extra doing (1 SD increase) to be more than six times that of extra watching or reading. Our data, from a psychology MOOC, is correlational in character, however we employ causal inference mechanisms to lend support for the claim that the associations we find are causal.

References

[1]
Aleven, V., Beal, C. R., & Graesser, A. C. (2013). Introduction to the special issue on advanced learning technologies. Journal of Educational Psychology, 105(4), 929--931.
[2]
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995) . Cognitive tutors:Lessons learned. The Journal of the Learning Sciences, 4, 167--207.
[3]
Anzai, Y., & Simon, H. A. (1979). The theory of learning by doing. Psychological Review, 86, 124--140.
[4]
Barab, S.A. & Squire, K.D. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Science, 13 (1) 1--14.
[5]
Bichsel, J. (2013). The State of E-Learning in Higher Education: An Eye toward Growth and Increased Access (Research Report), Louisville, CO: EDUCAUSE Center for Analysis and Research. Available online: http://www.educause.edu/ecar.
[6]
Bowen, W. G., Chingos, M. M., Lack, K. A., & Nygren, T. I. (2013). Interactive learning online at public universities: Evidence from a six-campus randomized trial. Journal of Policy Analysis and Management, 33(1), 94--111.
[7]
Champaign, J., Fredericks, C., Colvin, K., Seaton, D., Liu, A. & Pritchard, D. Correlating skill and improvement in 2 MOOCs with a student's time on task. In Proc. Learning@Scale 2014, Retrieved from: http://dx.doi.org/10.1145/2556325.2566250.
[8]
Clark, R.E., Feldon, D., van Merriënboer, J., Yates, K., & Early, S. (2007). Cognitive task analysis. In J.M. Spector, M.D. Merrill, J.J.G. van Merriënboer, & M.P. Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 577--593). Mahwah, NJ: Lawrence Erlbaum Associates.
[9]
Collins, E.D. (2013) "SJSU Plus augmented online learning environment: Pilot project report." The Research and Planning Group for California Community Colleges. Available online:http://www.sjsu.edu/chemistry/People/Faculty/Collins_Research_Page/AOLE%20Report%20Final%20Version_Jan%201_2014.pdf
[10]
Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics. In L. B. Resnick. Knowing, Learning, and Instruction: Essays in Honor of Robert Glaser (pp. 453--494). Hillsdale, NJ: Erlbaum.
[11]
Dewey, J. (1916), (2007 edition). Democracy and Education, Teddington: Echo Library.
[12]
Hill, P. (2013). Emerging Student Patterns in MOOCs: A (Revised) Graphical View. e-literate. Available online: http://mfeldstein.com/emerging-student-patterns-in-moocs-a-revised-graphical-view/
[13]
Kaufman, J.; Ryan, R.; Thille, C. and Bier, N. (2013) Open Learning Initiative Courses in Community Colleges: Evidence on Use and Effectiveness. Mellon University, Pittsburgh, PA. Available online: http://www.hewlett.org/sites/default/files/CCOLI_Report_Final_1.pdf
[14]
Koedinger, K.R., & Aleven V. (2007). Exploring the assistance dilemma in experiments with Cognitive Tutors. Educational Psychology Review, 19(3), 239--264.
[15]
Koedinger, K.R., Baker, R.S.J.d., Cunningham, K., Skogsholm, A., Leber, B., Stamper, J. (2010). A Data Repository for the EDM community: The PSLC DataShop. In Romero, C., Ventura, S., Pechenizkiy, M., Baker, R.S.J.d. (Eds.) Handbook of Educational Data Mining. Boca Raton, FL: CRC Press.
[16]
Koedinger, K.R., Booth, J. L., & Klahr, D. (2013). Instructional Complexity and the Science to Constrain It. Science, 342(6161), 935--937.
[17]
Koedinger, K. R., Corbett, A. C., & Perfetti, C. (2012). The Knowledge-Learning-Instruction (KLI) framework: Bridging the science-practice chasm to enhance robust student learning. Cognitive Science, 36 (5), 757--798.
[18]
Koedinger, K. R., McLaughlin, E. A., & Stamper, J. C. (2014). Ubiquity symposium: MOOCs and technology to advance learning and learning research: Data-driven learner modeling to understand and improve online learning. Ubiquity, Volume 2014, Number May (2014), Pages 1--13. ACM New York, NY, USA.
[19]
Koedinger, K. R., & Sueker, E. L. F. (2014). Monitored design of an effective learning environment for algebraic problem solving. Technical report Carnegie Mellon University-HCII-14-102.
[20]
Lovett, M., Meyer, O., Thille, C. (2008). The Open Learning Initiative: Measuring the effectiveness of the OLI learning course in accelerating student learning. Journal of Interactive Media in Education. http://jime.open.ac.uk/2008/14/.
[21]
Meyer, O., & Lovett, M. C. (2002). Implementing a computerized tutor in a statistical reasoning Course: Getting the big picture. In B. Phillips (Ed.) Proc. of the Sixth International Conference on Teaching Statistics.
[22]
Morote, E. & Pritchard, D. E. (2002). What Course Elements Correlate with Improvement on Tests in Introductory Newtonian Mechanics? National Association for Research in Science Teaching -- NARST- 2002 Conference.
[23]
Ng, A. and Widom J. (2014). Origins of the Modern MOOC. http://www.cs.stanford.edu/people/ang/papers/mooc14-OriginsOfModernMOOC.pdf
[24]
Pane, J.F., Griffin, B., McCaffrey, D.F. & Karam, R. (2014). Effectiveness of Cognitive Tutor Algebra I at Scale. Educational Evaluation and Policy Analysis, 36 (2), 127--144.
[25]
Pappano, L. (2012, November 2). The year of the MOOC. The New York Times.
[26]
Pashler, H., Bain, P., Bottge, B., Graesser, A., Koedinger, K., McDaniel, M., & Metcalfe, J. (2007). Organizing Instruction and Study to Improve Student Learning (NCER 2007-2004). Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education.
[27]
Ritter S., Anderson, J. R., Koedinger, K. R., & Corbett, A. (2007). Cognitive tutor: Applied research in mathematics education. Psychonomic Bulletin & Review, 14 (2):249--255.
[28]
Schank, R. C., Berman, T. R. & Macperson, K. A. (1999). Learning by doing. In C. M. Reigeluth (Ed.), Instructional Design Theories and Models: A New Paradigm of Instructional Theory (Vol. II) (pp.161--181). Mahwah, NJ: Lawrence Erlbaum Associates.
[29]
Singer, S.R. & Bonvillian, W.B. (2013). Two Revolutions in Learning. Science 22, Vol. 339 no. 6126, p.1359.
[30]
Sinha, T., Jermann, P., Li, N., Dillenbourg, P. (2014). Your click decides your fate: Inferring Information Processing and Attrition Behavior from MOOC Video Clickstream Interactions. Proc. of the 2014 Empirical Methods in Natural Language Processing Workshop on Modeling Large Scale Social Interaction in Massively Open Online Courses.
[31]
Strader, R. & Thille, C. (2012). The Open Learning Initiative: Enacting Instruction Online. In Oblinger, D.G. (Ed.) Game Changers: Education and Information Technologies (201--213). Educause.
[32]
Tetrad IV. http://www.phil.cmu.edu/tetrad.
[33]
Thille, C. (2014). Ubiquity symposium: MOOCs and technology to advance learning and learning research: opening statement. Ubiquity, Volume 2014, Number April (2014), Pages 1--7. ACM New York, NY, USA.
[34]
VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227--265.
[35]
VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197--221.
[36]
Zhu X., Lee Y., Simon H.A., & Zhu, D. (1996). Cue recognition and cue elaboration in learning from examples. In Proc. of the National Academy of Sciences 93, (pp. 1346--1351).

Cited By

View all
  • (2025)Rationale, design, and methodology of My Body Is Fit and Fabulous at school (MyBFF@school) study: a multi-pronged intervention program to combat obesity among Malaysian schoolchildrenBMC Public Health10.1186/s12889-024-20726-z24:S1Online publication date: 10-Jan-2025
  • (2025)Do Tutors Learn from Equity Training and Can Generative AI Assess It?Proceedings of the 15th International Learning Analytics and Knowledge Conference10.1145/3706468.3706531(505-515)Online publication date: 3-Mar-2025
  • (2025)Does Multiple Choice Have a Future in the Age of Generative AI? A Posttest-only RCTProceedings of the 15th International Learning Analytics and Knowledge Conference10.1145/3706468.3706530(494-504)Online publication date: 3-Mar-2025
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
L@S '15: Proceedings of the Second (2015) ACM Conference on Learning @ Scale
March 2015
438 pages
ISBN:9781450334112
DOI:10.1145/2724660
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 March 2015

Check for updates

Author Tags

  1. course effectiveness
  2. learning by doing
  3. learning prediction
  4. moocs
  5. oer
  6. open education

Qualifiers

  • Research-article

Funding Sources

  • National Science Foundation

Conference

L@S 2015
Sponsor:
L@S 2015: Second (2015) ACM Conference on Learning @ Scale
March 14 - 18, 2015
BC, Vancouver, Canada

Acceptance Rates

L@S '15 Paper Acceptance Rate 23 of 90 submissions, 26%;
Overall Acceptance Rate 117 of 440 submissions, 27%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)686
  • Downloads (Last 6 weeks)82
Reflects downloads up to 03 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Rationale, design, and methodology of My Body Is Fit and Fabulous at school (MyBFF@school) study: a multi-pronged intervention program to combat obesity among Malaysian schoolchildrenBMC Public Health10.1186/s12889-024-20726-z24:S1Online publication date: 10-Jan-2025
  • (2025)Do Tutors Learn from Equity Training and Can Generative AI Assess It?Proceedings of the 15th International Learning Analytics and Knowledge Conference10.1145/3706468.3706531(505-515)Online publication date: 3-Mar-2025
  • (2025)Does Multiple Choice Have a Future in the Age of Generative AI? A Posttest-only RCTProceedings of the 15th International Learning Analytics and Knowledge Conference10.1145/3706468.3706530(494-504)Online publication date: 3-Mar-2025
  • (2025)Does the Doer Effect Generalize To Non-WEIRD Populations? Toward Analytics in Radio and Phone-Based LearningProceedings of the 15th International Learning Analytics and Knowledge Conference10.1145/3706468.3706505(844-850)Online publication date: 3-Mar-2025
  • (2025)NeuRL: A Standalone No-Code Web-Based Agent Environment to Explore Neural Networks and Reinforcement LearningProceedings of the 56th ACM Technical Symposium on Computer Science Education V. 110.1145/3641554.3701850(1064-1070)Online publication date: 12-Feb-2025
  • (2025)Is anybody watchingComputers & Education10.1016/j.compedu.2024.105148222:COnline publication date: 7-Jan-2025
  • (2024)Empowering Teachers With Generative AI Tools and SupportTransforming Education With Generative AI10.4018/979-8-3693-1351-0.ch011(214-238)Online publication date: 29-Mar-2024
  • (2024)The Future of Postsecondary Education in the Age of AIEducation Sciences10.3390/educsci1403032614:3(326)Online publication date: 19-Mar-2024
  • (2024)Individual learning paths mastering teachers’ professional visionFrontiers in Education10.3389/feduc.2024.13050739Online publication date: 1-Feb-2024
  • (2024)Vocational school students’ use and opinions of voluntary-based online learning solutions presented in a mathematics courseEuropean Journal of Science and Mathematics Education10.30935/scimath/1475112:3(394-410)Online publication date: 2024
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media