The impact of online video lecture recordings and automated feedback on student performance

https://doi.org/10.1016/j.compedu.2009.10.002Get rights and content

Abstract

To what extent a blended learning configuration of face-to-face lectures, online on-demand video recordings of the face-to-face lectures and the offering of online quizzes with appropriate feedback has an additional positive impact on the performance of these students compared to the traditional face-to-face course approach? In a between-subjects design in which students were randomly assigned to a group having access to the online lectures including multiple choice quizzes and appropriate feedback or to a group having access to the online lectures only, 474 students (161 men and 313 women) of a course on European Law agreed to participate in the experiment. By using regression analysis we found that the course grade of the students was predicted by their grade point average, their study discipline, their grade goal for the course, the expected difficulty-level of the course, the number of online lectures they viewed, the number of lectures the students attended in person and the interaction between the lectures they viewed online and attended in person. Students who attended few lectures had more benefit from viewing online lectures than students who attended many lectures. In contrast to our expectations, the regression analysis did not show a significant effect of automated feedback on student performance. Offering recordings of face-to-face lectures is an easy extension of a traditional course and is of practical importance, because it enables students who are often absent from the regular face-to-face lectures to be able to improve their course grade by viewing the lectures online.

Introduction

E-learning has recently become one of the fastest-moving trends and aims to provide a configurable infrastructure that integrates learning material, tools, and services into a single solution to create and deliver training or educational content quickly, effectively, and economically (Zhang, Zhou, Briggs, & Nunamaker, 2006). In many studies comparisons have been made between the effectiveness of online (distance) learning versus face-to-face learning. Russell (1999) made an inventory of many of these media comparison studies (MCS) and concluded that generally there is no significant difference between the average performance of learners in the case of face-to-face learning as compared to learners exposed to distance learning methods (i.e. the ‘no significant difference phenomenon’). However, Ross and Bell (2007) indicated that this phenomenon could be dependent on the level of learning. While they found no significant difference in performance at lower levels of abstraction between students in the traditional setting as compared to online students, students in the traditional setting outperformed online students with respect to higher order learning (i.e. applying, analyzing and synthesizing information).

Video is a rich and powerful medium in e-learning because it can present information in an attractive manner. Prior studies have investigated the effect of instructional video on learning outcomes. However, the instructional video used in early studies was primarily either broadcasted through TV programs or stored on CD-ROMs. The linear nature of such video instructions produced inconsistent results. Recent advances in multimedia and communication technologies have resulted in powerful learning systems with instructional video components. The emergence of non-linear, interactive digital video technology allows students to interact with instructional video. This may enhance learner engagement, and so improve learning effectiveness. One of the examples is the Virtual Classroom project which uses asynchronous learning networks plus videotaped lectures to evaluate the effectiveness of online courses required for bachelor’s degrees in information systems and computer science. Students who completed online courses tended to do as well as those in traditional classrooms, even though more online students withdrew or took an incomplete grade. Carnegie Mellon University’s just-in-time lecture project suggests that video-based education and training systems support the same level of teaching and learning effectiveness as face-to-face instruction (Zhang et al., 2006).

Offering online video recordings of lectures after they have been given is useful in allowing students to view lectures they have missed or to re-view difficult lectures again to improve understanding. Chiu, Lee, and Yang (2006) investigated the viewing behavior of students in a Chinese grammar course when online post-class lecture videos were made available. They divided students in two groups based on their viewing activity (top 50% and bottom 50%) and found no difference in course grades between the two groups corrected for their GPA. Additionally they found that students had a preference for recordings of their own lectures as compared to lectures of a parallel group.

Ross and Bell (2007) compared the performance of students in a quality management course who had access to face-to-face lectures as well as the online lecture video recordings to students who only had access to the online lecture recordings. Using a regression analysis they found that the course score of students in the first group with access to the face-to-face lectures was predicted positively by their GPA, negatively by their age, positively by their homework performance and negatively by the number of lectures they viewed online. For the students who did not have access to the face-to-face lectures, the course score was positively predicted by their GPA, negatively by their age, positively by their homework performance and positively by the number of lectures they viewed online.

In this study the instructional method consists of a combination of face-to-face lectures and online on-demand video recordings of the face-to-face lectures, combined with the offering of online quizzes with appropriate feedback.

The general research question is to what extent this blended learning configuration of face-to-face lectures, online on-demand video recordings of the face-to-face lectures and the offering of online quizzes with appropriate feedback has an additional positive impact on the performance of these students compared to the traditional face-to-face course approach.

Creating an effective hybrid course takes a lot of effort; McFarlin (2008) reported a time investment of about 16–20 h for creating a single online lecture in his hybrid course. The required time investment could therefore be an obstacle for many teachers to create a hybrid course. A less time-consuming approach is to supplement the traditional course with online video recordings of the face-to-face lectures, which have been found to reduce dropout rate (Olsen, 2003). Even though Chiu and colleagues (2006) and Ross and Bell (2007) tried to measure the effect of the number of lectures students viewed online, their studies did not include a measure of attended face-to-face lectures, which is also important in explaining course performance (Stanca, 2006, Clark and Mayer, 2008).

Another method to improve the performance of students is the use of formative assessment (Lowry, 2005). When a Virtual Learning Environment is available (e.g., Blackboard), online quizzes consisting of multiple-choice questions and appropriate feedback are relatively easy to construct.

The primary goal of this study is to assess the effect on student performance of offering online on-demand video recordings of the face-to-face lectures, while also taking into account the number of lectures the students have attended in person. Additionally we aim to investigate the effect of offering online quizzes with appropriate feedback on the performance of these students.

We posit the following hypotheses which are tested in this study:

  • (1)

    The number of lectures students view online and attend in person will contribute positively to the course performance. However, the positive effect of viewing lectures online will be weaker for students who attend a large number of lectures in person compared to a small number of lectures.

  • (2)

    Students who have access to multiple choice quizzes including appropriate feedback will perform better than students who do not have access to these quizzes.

Section snippets

Design

To test our hypotheses we used a between-subjects design in which students were randomly assigned to a group having access to the online lectures including multiple choice quizzes and appropriate feedback, or to a group having access to the online lectures only.

Sample

A total of 474 students (161 men and 313 women) of the course European Law finished the course exam and agreed to participate in the experiment. Most participants studied Law (392 students), while a minority (82 students) studied

Scale consistency and descriptives

The consistency of all scales used in this study and discussed in Section 3.3 is shown in Table 1. Scales which were not very consistent (i.e. Cronbach’s alpha less than 0.7) consisted of the extrinsic motivation scale and three out of four ILS scales (i.e. visual–verbal, sequential–global and active–reflective). Felder and Spurlin (2005) indicated that several studies found similar consistency values for the ILS dimensions, but they argued by referring to Tuckman (1999) that a Cronbach’s alpha

Discussion

In our first hypothesis we posited that the number of lectures students viewed online and attended in person contributed positively to course performance. Inspection of Table 3 indeed shows that this holds true, even while controlling for other possibly important variables. Furthermore, the interaction we predicted also showed up in the data. Fig. 1 visualizes the effect of the number of viewed online lecture recordings on performance moderated by the number of attended lectures. Using the

References (27)

  • D. Zhang et al.

    Instructional video in e-learning: Assessing the impact of interactive video on learning effectiveness

    Information and Management

    (2006)
  • M.N. van den Berg et al.

    Student success in university education. A multi-measurement study into the impact of student and faculty factors on study progress

    Higher Education

    (2005)
  • Chiu, C. F., Lee, G. C., & Yang, J. H. (2006). A comparative study of post-class lecture viewing. In Proceedings of the...
  • R.B. Clariana et al.

    Applying a connectionist description of feedback timing

    Educational Technology Research and Development

    (2000)
  • R.C. Clark et al.

    E-learning and the science of instruction

    (2008)
  • M.J. Cox et al.

    Effect of ICT: Do we know what we should know?

    Education and Information Technologies

    (2007)
  • Ding, N. (2009). Computer-Supported Collaborative learning & gender. Ph’d dissertation. University...
  • Å. Diseth

    Validation of a Norwegian version of the approaches and study skills inventory for students (ASSIST): Application of structural equation modelling

    Scandinavian Journal of Educational Research

    (2001)
  • R.M. Felder et al.

    Learning and teaching styles in engineering education

    Engineering Education

    (1988)
  • Felder, R. M., & Soloman, B. A. (1991). Index of learning styles. <http://www.ncsu.edu/felder-public/ILSpage.html>...
  • R.M. Felder et al.

    Applications, reliability and validity of the index of learning styles

    International Journal of Engineering Education

    (2005)
  • J. Hattie et al.

    The power of feedback

    Review of Educational Research

    (2007)
  • M.H. Heijne-Penninga et al.

    Medical Education

    (2008)
  • Cited by (127)

    • Pre-class video watching fosters achievement and knowledge retention in a flipped classroom

      2022, Computers and Education
      Citation Excerpt :

      The few studies documenting (mostly positive) effects of completing pre-class activities on performance have had some methodological shortcomings: (1) Students watched the videos collectively in the seminar room without any possibility for self-determined learning behavior (e.g. Choi & Yang, 2010; Hadie et al., 2019), (2) the timing of video access relative to in-class learning was not taken into account (e. g. Moravec et al., 2010; Geri & Winer, 2015; Mirriahi & Dawson, 2013), or (3) simple recordings from the normal lecture were used instead of professionally designed and edited educational videos within an aligned instructional design for a flipped classroom (e. g. Mirriahi & Dawson, 2013; Murphy & Stewart, 2015; O’Callaghan, Neumann, Jones, & Creed, 2017). Moreover, studies on digital learning are often restricted to data gathered from single experiments that do not appropriately reflect the natural elements of self-guided learning throughout an entire semester (Choi & Yang, 2010; Hadie et al., 2019; Zhang et al., 2006) or from self-reported patterns in usage of course materials and thus lack an objective operationalization of students' video usage patterns and academic achievement (Murphy & Stewart, 2015; O'Callaghan et al., 2017; Wieling & Hofman, 2010). In our study, we address these shortcomings by collecting objective data on students' viewing patterns on sophisticated pre-class videos that we developed in constructive alignment within our specific in-class activities.

    • Electronic Feedback in Large University Statistics Courses: The Longitudinal Effects of Quizzes on Motivation, Emotion, and Cognition

      2023, Electronic Feedback in Large University Statistics Courses: The Longitudinal Effects of Quizzes on Motivation, Emotion, and Cognition
    View all citing articles on Scopus
    View full text