Tiffany Wenting Li
Skip Abstract Section
Abstract
As colleges and universities continue their commitment to increasing access to higher education through offering education online and at scale, attention on teaching open-ended subjects online and at scale, mainly the arts, humanities, and the social sciences, remains limited. While existing work in scaling open-ended courses primarily focuses on the evaluation and feedback of open-ended assignments, there is a lack of understanding of how to effectively teach open-ended, university-level courses at scale. To better understand the needs of teaching large-scale, open-ended courses online effectively in a university setting, we conducted a mixed-methods study with university instructors and students, using surveys and interviews, and identified five critical pedagogical elements that distinguish the teaching and learning experiences in an open-ended course from that in a non-open-ended course. An overarching theme for the five elements was the need to support students' self-expression. We further uncovered open challenges and opportunities when incorporating the five critical pedagogical elements into large-scale, open-ended courses online in a university setting, and suggested six future research directions: (1) facilitate in-depth conversations, (2) create a studio-friendly environment, (3) adapt to open-ended assessment, (4) scale individual open-ended feedback, (5) establish trust for self-expression, and (6) personalize instruction and harness the benefits of student diversity.
- Carlos Alario-Hoyos, Mar Pérez-Sanagustín, Carlos Delgado-Kloos, Mario Muñoz-Organero, Antonio Rodríguez-delas-Heras, et al. 2013. Analysing the impact of built-in and external social tools in a mooc on educational technologies. In European Conference on Technology Enhanced Learning. Springer, 5--18.Google Scholar
Digital Library
- Alaa Althubaiti. 2016. Information bias in health research: definition, pitfalls, and adjustment methods. Journal of multidisciplinary healthcare, 9, 211.Google ScholarCross Ref
- Ernesto Arroyo, Valeria Righi, Roger Tarrago, Patricia Santos, Davinia Hernández-Leo, and Josep Blat. 2011. Remote collaborative multi-user informal learning experiences: design and evaluation. In European Conference on Technology Enhanced Learning. Springer, 43--56.Google ScholarCross Ref
- Stephen P Balfour. 2013. Assessing writing in moocs: automated essay scoring and calibrated peer review?. Research & Practice in Assessment, 8, 40--48.Google Scholar
- William Bianchi. 2002. The wisconsin school of the air: success story with implications. Journal of Educational Technology & Society, 5, 1, 141--147.Google Scholar
- Patrick Blessinger and John M Carfora. 2014. Inquiry-based learning for the arts, humanities and social sciences: A conceptual and practical resource for educators. Emerald Group Publishing.Google Scholar
- Benjamin S Bloom et al. 1956. Taxonomy of educational objectives. vol. 1: cognitive domain. New York: McKay, 20--24.Google Scholar
- College Board. 2020. Quick guide: types of college courses. https://bigfuture.collegeboard.org/find-colleges/academiclife/quick-guide-types-of-college-courses. (2020).Google Scholar
- Sorana Bolboac-, JÃ Lorentz, et al. 2007. Computer-based testing on physical chemistry topic: a case study. International Journal of Education and Development using ICT, 3, 1.Google Scholar
- Michael Brooks, Sumit Basu, Charles Jacobs, and Lucy Vanderwende. 2014. Divide and correct: using clusters to grade short answers at scale. In Proceedings of the first ACM conference on Learning@ scale conference. ACM, 89--98.Google ScholarDigital Library
- Julia Cambre, Scott Klemmer, and Chinmay Kulkarni. 2018. Juxtapeer: comparative peer review yields higher quality feedback and promotes deeper reflection. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 294.Google ScholarDigital Library
- David Carless. 2009. Trust, distrust and their impact on assessment reform. Assessment & Evaluation in Higher Education, 34, 1, 79--89.Google ScholarCross Ref
- Muthu Kumar Chandrasekaran, Min-Yen Kan, Bernard CY Tan, and Kiruthika Ragupathi. 2015. Learning instructor intervention from mooc forums: early results and issues. arXiv preprint arXiv:1504.07206.Google Scholar
- Jyoti Chauhan and Anita Goel. 2016. An analysis of quiz in mooc. In 2016 Ninth International Conference on Contemporary Computing (IC3). IEEE, 1--6.Google ScholarCross Ref
- Mary M Christopher, Julie A Thomas, and Mary K Tallent-Runnels. 2004. Raising the bar: encouraging high level thinking in online discussion forums. Roeper Review, 26, 3, 166--171.Google ScholarCross Ref
- Derrick Coetzee, Armando Fox, Marti A Hearst, and Bjoern Hartmann. 2014. Chatrooms in moocs: all talk and no action. In Proceedings of the first ACM conference on Learning@ scale conference. ACM, 127--136.Google ScholarDigital Library
- R Cook. 2000. Technology trends: teleconferencing, distance education, and future technologies. (2000).Google Scholar
- Claude Cookman. 2009. Using jitt to foster active learning in a humanities course. Just-in-Time Teaching, 163--178.Google Scholar
- Yi Cui and Alyssa Friend Wise. 2015. Identifying content-related threads in mooc discussion forums. In L@ S, 299--303.Google Scholar
- Thanasis Daradoumis, Roxana Bassi, Fatos Xhafa, and Santi Caballé. 2013. A review on massive e-learning (mooc) design, delivery and assessment. In 2013 eighth international conference on P2P, parallel, grid, cloud and internet computing. IEEE, 208--213.Google Scholar
- Brian Dear. 2017. The friendly orange glow: The untold story of the PLATO System and the Dawn of Cyberculture. Pantheon.Google Scholar
- Vanessa Paz Dennen*. 2005. From message posting to learning dialogues: factors affecting learner participation in asynchronous discussion. Distance Education, 26, 1, 127--148.Google ScholarCross Ref
- Louis Deslauriers, Logan S McCarty, Kelly Miller, Kristina Callaghan, and Greg Kestin. 2019. Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 116, 39, 19251--19257.Google ScholarCross Ref
- Stephen Downes. 2008. Places to go: connectivism & connective knowledge. Innovate: Journal of Online Education, 5, 1, 6.Google Scholar
- Suzannah Evans and Karen McIntyre. 2016. Moocs in the humanities: can they reach underprivileged students? Convergence, 22, 3, 313--323.Google ScholarCross Ref
- Suzannah Evans and Jessica Gall Myrick. 2015. How mooc instructors view the pedagogy and purposes of massive open online courses. Distance Education, 36, 3, 295--311.Google ScholarCross Ref
- Helene Fournier and Rita Kop. 2015. Mooc learning experience design: issues and challenges. International Journal on E-Learning, 14, 3, 289--304.Google Scholar
- C Ailie Fraser, Tricia J Ngoon, Ariel S Weingarten, Mira Dontcheva, and Scott Klemmer. 2017. Critiquekit: a mixedinitiative, real-time interface for improving feedback. In Adjunct Publication of the 30th Annual ACM Symposium on User Interface Software and Technology. ACM, 7--9.Google Scholar
- Nicola K Gale, Gemma Heath, Elaine Cameron, Sabina Rashid, and Sabi Redwood. 2013. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC medical research methodology, 13, 1, 117.Google Scholar
- Chase Geigle, ChengXiang Zhai, and Duncan C Ferguson. 2016. An exploration of automated grading of complex assignments. In Proceedings of the Third (2016) ACM Conference on Learning@ Scale. ACM, 351--360.Google ScholarDigital Library
- Graham R Gibbs. 2007. Thematic coding and categorizing. Analyzing qualitative data. London: Sage, 38--56.Google Scholar
- Joshua Goodman, Julia Melkers, and Amanda Pallais. 2019. Can online delivery increase access to education? Journal of Labor Economics, 37, 1, 1--34.Google ScholarCross Ref
- James Grimmelmann. 2015. The virtues of moderation. Yale JL & Tech., 17, 42.Google Scholar
- Greg Guest, Arwen Bunce, and Laura Johnson. 2006. How many interviews are enough? an experiment with data saturation and variability. Field methods, 18, 1, 59--82.Google Scholar
- Kathleen Harting and Margaret J Erthal. 2005. History of distance learning. Information technology, learning, and performance journal, 23, 1, 35.Google Scholar
- John Hattie and Helen Timperley. 2007. The power of feedback. Review of educational research, 77, 1, 81--112.Google Scholar
- Donald E Heller. 2001. The states and public higher education policy: Affordability, access, and accountability. JHU Press.Google Scholar
- Khe Foon Hew and Wing Sum Cheung. 2014. Students? and instructors? use of massive open online courses (moocs): motivations and challenges. Educational research review, 12, 45--58.Google Scholar
- Starr R Hiltz and Murray Turoff. 1985. Structuring computer-mediated communication systems to avoid information overload. Communications of the ACM, 28, 7, 680--689.Google ScholarDigital Library
- Malinka Ivanova. 2016. Technology landscape in moocs platforms. In 2016 19th International Symposium on Electrical Apparatus and Technologies (SIELA). IEEE, 1--4.Google ScholarCross Ref
- Hyeonsu B Kang, Gabriel Amoako, Neil Sengupta, and Steven P Dow. 2018. Paragon: an online gallery for enhancing design feedback with visual examples. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 606.Google ScholarDigital Library
- Desmond Keegan. 2013. Foundations of distance education. Routledge.Google Scholar
- Hanan Khalil and Martin Ebner. 2013. "how satisfied are you with your mooc?"-a research study on interaction in huge online courses. In EdMedia Innovate Learning. Association for the Advancement of Computing in Education (AACE), 830--839.Google Scholar
- Andrea Kienle and Carsten Ritterskamp. 2007. Facilitating asynchronous discussions in learning communities: the impact of moderation strategies. Behaviour & Information Technology, 26, 1, 73--80.Google ScholarDigital Library
- Sara Kiesler, Robert Kraut, Paul Resnick, and Aniket Kittur. 2012. Regulating behavior in online communities. Building successful online communities: Evidence-based social design, 125--178.Google Scholar
- Chinmay Kulkarni, Julia Cambre, Yasmine Kotturi, Michael S Bernstein, and Scott R Klemmer. 2015. Talkabout: making distance matter with small groups in massive classes. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, 1116--1128.Google ScholarDigital Library
- Chinmay E Kulkarni, Michael S Bernstein, and Scott R Klemmer. 2015. Peerstudio: rapid peer feedback emphasizes revision and improves performance. In Proceedings of the second (2015) ACM conference on learning@ scale. ACM, 75--84.Google ScholarDigital Library
- Fabrizio Lamberti, Andrea Sanna, Gianluca Paravati, and Gilles Carlevaris. 2014. Automatic grading of 3d computer animation laboratory assignments. IEEE Transactions on Learning Technologies, 7, 3, 280--290.Google ScholarCross Ref
- N Levenburg. 2000. A brief history of curriculum reform in the united states. Distance Learning Dynamics.Google Scholar
- Ou Lydia Liu, Chris Brew, John Blackmore, Libby Gerard, Jacquie Madhok, and Marcia C Linn. 2014. Automated scoring of constructed-response science items: prospects and obstacles. Educational Measurement: Issues and Practice, 33, 2, 19--28.Google ScholarCross Ref
- Karen D Locke. 2000. Grounded theory in management research. Sage.Google Scholar
- Jan Machotka, Zorica Nedic, and Özdemir Göl. 2008. Collaborative learning in the remote laboratory NetLab. PhD thesis. International Institute of Informatics and Systemics.Google Scholar
- Philip W Martin. 2003. Key aspects of teaching and learning in arts, humanities and social sciences. A handbook for teaching and learning in higher education, 300.Google Scholar
- Michael J McInerney and L Dee Fink. 2003. Team-based learning enhances long-term retention and critical thinking in an undergraduate microbial physiology course. Microbiology Education, 4, 3.Google ScholarCross Ref
- John A Michael. 1980. Studio art experience: the heart of art education. Art Education, 33, 2, 15--19.Google ScholarCross Ref
- Michael Mohler, Razvan Bunescu, and Rada Mihalcea. 2011. Learning to grade short answer questions using semantic similarity measures and dependency graph alignments. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. Association for Computational Linguistics, 752--762.Google ScholarDigital Library
- Michael G Moore. 1989. Three types of interaction. (1989).Google Scholar
- National Association of Schools of Art and Design. 2018. Nasad handbook 2018--19. https://nasad.arts-accredit.org/ accreditation/standards-guidelines/handbook/. (Dec. 2018).Google Scholar
- Ebba Ossiannilsson, Fahriye Altinay, and Zehra Altinay. 2015. Analysis of moocs practices from the perspective of learner experiences and quality culture. Educational Media International, 52, 4, 272--283.Google ScholarCross Ref
- Tina Overton. 2003. Key aspects of teaching and learning in experimental sciences and engineering. A handbook for teaching and learning in higher education: Enhancing academic practice, 2, 255--277.Google Scholar
- Wanqi Peng. 2016. How can mooc providers create an interactive learning experience in the arts. (2016).Google Scholar
- Justin Reich and José A Ruipérez-Valiente. 2019. The mooc pivot. Science, 363, 6423, 130--131.Google Scholar
- Yuqing Ren, Robert Kraut, Sara Kiesler, and Paul Resnick. 2010. Regulating behavior in online communities.Google Scholar
- Frederico Menine Schaf, Dieter Müller, F Wilhelm Bruns, Carlos Eduardo Pereira, and H-H Erbe. 2009. Collaborative learning and engineering workspaces. Annual Reviews in Control, 33, 2, 246--252.Google ScholarCross Ref
- D Shah. 2018. Kadenze programs: a modern day credential for artists and creatives. (2018).Google Scholar
- Dhawal Shah. 2019. By the numbers: moocs in 2019. https://www.classcentral.com/report/mooc-stats-2019/. (Dec. 2019).Google Scholar
- Peter Shea, Alexandra Pickett, and Chun Sau Li. 2005. Increasing access to higher education: a study of the diffusion of online teaching among 913 college faculty. The International review of research in open and distributed learning, 6, 2.Google Scholar
- Ali Sher. 2009. Assessing the relationship of student-instructor and student-student interaction to student learning and satisfaction in web-based online learning environment. Journal of Interactive Online Learning, 8, 2.Google Scholar
- Scott Simkins and Mark Maier. 2010. Just-in-time teaching: Across the disciplines, across the academy. Stylus Publishing, LLC.Google Scholar
- Hoi K Suen. 2014. Peer assessment for massive open online courses (moocs). International Review of Research in Open and Distributed Learning, 15, 3, 312--327.Google ScholarCross Ref
- Ayse Saliha Sunar, Nor Aniza Abdullah, Su White, and Hugh C Davis. 2015. Personalisation of moocs: the state of the art.Google Scholar
- Karen Swan, Scott Day, Leonard Bogle, and Traci van Prooyen. 2015. Amp 1: a tool for characterizing the pedagogical approaches of moocs. In MOOCs and open education around the world. Routledge, 105--118.Google Scholar
- Michael Sweet and Larry K Michaelsen. 2012. Team-based learning in the social sciences and humanities: Group work that works to generate critical thinking and engagement. Stylus Publishing, LLC.Google Scholar
- Joan Thormann and Patricia Fidalgo. 2014. Guidelines for online course moderation and community building from a student's perspective. Journal of Online Learning & Teaching, 10, 3, 374--388.Google Scholar
- Michael B Twidale and Karen Ruhleder. 2004. Over-the-shoulder-learning in a distance education environment. Learning, culture, and community in online education: research and practice, Caroline Haythornthwaite & Michelle M. Kazmer, eds. New York, NY: Lang, 177--194.Google Scholar
- Arizona State University. 2019. Defining course components. https://provost.asu.edu/sites/default/files/page/1585/ defining-course-components_6-17-19.pdf. (June 2019).Google Scholar
- Beth Walker. 2009. New twists on an old problem: preventing plagiarism and enforcing academic integrity in an art and design school. Art Documentation: Journal of the Art Libraries Society of North America, 28, 1, 48--51.Google ScholarCross Ref
- Xu Wang, Srinivasa Teja Talluri, Carolyn Rose, and Kenneth Koedinger. 2019. Upgrade: sourcing student open-ended solutions to create scalable learning opportunities. In Proceedings of the Sixth (2019) ACM Conference on Learning@ Scale, 1--10.Google ScholarDigital Library
- Etienne Wenger, Nancy White, and John D Smith. 2009. Digital habitats: Stewarding technology for communities. CPsquare.Google Scholar
- Joseph Jay Williams, Juho Kim, Anna Rafferty, Samuel Maldonado, Krzysztof Z Gajos, Walter S Lasecki, and Neil Heffernan. 2016. Axis: generating explanations at scale with learnersourcing and machine learning. In Proceedings of the Third (2016) ACM Conference on Learning@ Scale, 379--388.Google ScholarDigital Library
- Pete Williams, David Nicholas, and Barrie Gunter. 2005. E-learning: what the literature tells us about distance education. In Aslib Proceedings. Emerald Group Publishing Limited.Google ScholarCross Ref
- Zhen Xiong, Lin Zhi, and Jie Jiang. 2019. Research on art education digital platform based on big data. In 2019 IEEE 4th International Conference on Big Data Analytics (ICBDA). IEEE, 208--211.Google ScholarCross Ref
- Diyi Yang, Mario Piergallini, Iris Howley, and Carolyn Rose. 2014. Forum thread recommendation for massive open online courses. In Educational Data Mining 2014. Citeseer.Google Scholar
- Yu-Chun Grace Yen, Joy O Kim, and Brian P Bailey. 2020. Decipher: an interactive visualization tool for interpreting unstructured design feedback from multiple providers. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1--13.Google Scholar
- Jeff Young. 2020. Sustaining higher education in the coronavirus crisis. Edsurge. Accessed 6/1/2020 at: https://www. edsurge.com/research/guides/sustaining-higher-education-in-the-coronavirus-crisis.Google Scholar
- Yuan Zhenming, Zhang Liang, and Zhan Guohua. 2003. A novel web-based online examination system for computer science education. In 33rd ASEE/IEEE Frontiers in Education Conference, 5--8.Google ScholarCross Ref
- Craig B. Zilles, Matthew West, Geoffrey L. Herman, and Timothy Bretl. 2019. Every university should have a computer-based testing facility. In CSEDU.Google Scholar
Index Terms
- "It's all about conversation": Challenges and Concerns of Faculty and Students in the Arts, Humanities, and the Social Sciences about Education at Scale
Recommendations
Connecting across campus
SIGCSE '10: Proceedings of the 41st ACM technical symposium on Computer science educationComputer science holds a unique position to craft multidisciplinary curricula for the new generation of faculty and students across the academy who increasingly rely on computing for their scholarship. We propose that computer science programs cease ...
Five Years of Graduate CS Education Online and at Scale
CompEd '19: Proceedings of the ACM Conference on Global Computing EducationIn 2014, Georgia Tech launched an online campus for its Master of Science in Computer Science program. The degree, equal in stature and accreditation to its on-campus counterpart, offered a notably lower cost of attendance. Its design emphasized ...
Early Exploration of MOOCs in the U.S. Higher Education: An Absorptive Capacity Perspective
Advanced information technologies have enabled Massive Open Online Courses (MOOCs), which have the potential to transform higher education around the world. Why are some institutions eager to embrace this technology-enabled model of teaching, while others ...
Comments