Keywords

1 Introduction

The Flip-Flop instructional methodology is designed to augment the currently increasingly popular teaching concepts that rely on students learning from video recordings of lectures posted online. Inverted or flipped classroom and learning based on MOOCs (Massive Open Online Classes) are prime examples of such educational strategies. Our Flip-Flop method requires the students to construct quizzes that are synchronized with the screencast of the lecture. In essence, Flip-Flop offers a structured approach to the “Learning by Teaching” concept and supports it with several online tools that facilitate creating quizzes, taking quizzes, administrative chores such as scheduling and creating quiz templates. Moreover, our Flip-Flop tools support instructors in evaluation of the quizzes and provide them with a plethora of data that can be used for grading and to ascertain the effectiveness of a students’ learning. In addition, peer evaluation is also an integral component of the Flip-Flop method.

Since Flip-Flop quizzes are tightly interconnected with the lecture videos, students do not just listen to a lecture, but are virtually guaranteed to interact with its content - either while making a quiz from one of its segments, or by taking a quiz from the other segments of the screencast that were created by one of their peers. In particular, creating a quiz is an intrinsically creative endeavor that the vast majority of students have never attempted before. Studies devoted to this uncharted creative activity promise to deliver fundamental insights into students’ emotion and cognitive load, which can provide deep insight into how deeply the information is being processed, the creativity used to create the quizzes, and the effectiveness of the method.

2 Theoretical Background Flip-Flop

At the core of the Flip-Flop methodology is the concept of making and taking quizzes synchronized with video recordings of lectures in order to deepen the students’ understanding of the subject presented in the screencast. On the technology side, this basic strategy is supported by several online tools.

Taking a quiz is straightforward: As the screenshot in Fig. 1 shows, while the students view at the lecture video on the left hand side of the screen, tasks are displayed to the right. A smiley face confirms that a correct choice has been selected at which point a feedback may provide additional information. Similarly, a crying face indicates an incorrect answer. When student selects the hint button, the video stops playing and a short hint text appears. The hint can be accompanied by link to a web page with related information.

Fig. 1.
figure 1

Taking a quiz: selecting a correct choice displays a feedback that offers additional information.

Full size image

But the Flip-Flop methodology goes beyond simply taking a quiz prepared by the instructor: Using the online quiz authoring tool depicted in Fig. 2, students create quiz tasks synchronized with video lectures. When students need to formulate a multiple choice task related to a segment of the video they need to view the segment, understand what the lecturer is presenting within this particular segment and then formulate a related question with numerous possible answers. In addition to entering the question and answers as text and/or image, the Flip-Flop authoring tool supports entering a feedback for each of the correct and incorrect choices, adding a hint text and creating a link button with a web address (URL) that leads to an external online document that offers additional information that may help solve the task. Students are encouraged to formulate feedbacks that point to reasoning why the correct choice is incorrect and what is the likely misconception that leads to answer incorrectly. The screenshot in Fig. 1 shows, for instance, a feedback that explains why the chosen answer is the correct one.

Fig. 2.
figure 2

Creating a quiz: multiple-choice task synchronized with a segment of video and entries for hint text, hint link label, hint link URL, and for choice feedback.

Full size image

Peer evaluation is another essential component of Flip-Flop. Students are asked to have several poll questions at the end of each quiz. These survey questions allow the quiz author’s peers to judge the quality of the questions, the answers, the feedbacks as well as the hints and the resources accessed via the hint links.

The Flip-Flop technology incorporates two other online support tools. The scheduling tool offers extensive support for the students as well as for the instructor: it can be used assign students to groups, it provides the quiz author with an initial quiz template and notifies the other members of the other members in the group once a quiz is ready.

The Instructor Support Tool allows the instructor to view all the quizzes students created and add to the database their evaluation of all the components of a quiz – questions, answers, feedbacks, etc. This data together with the students’ peer evaluations may constitute a valuable component of grading.

2.1 Creativity and Cognitive Load Measured by EEG

When developing hypotheses for this study, it was first necessary to deconstruct the cognitive processes associated with the study task—a text-based creation of a quiz with feedback using a novel tool. Students will creatively generate quiz questions with feedback using this tool. In this study we will use EEG and psychophysiological measures. EEG is a psychophysiological measurement of post-synaptic electrical potentials on the surface of the scalp [1]. Electrodes are placed on specific locations of the scalp which collect the summation of synchronized activity from underlying pyramidal neurons lying near the surface of the cortex. The measure at each electrode location is then compared to a reference electrode located elsewhere on the scalp [2]. The recorded oscillations of brain activity at each electrode are complex waveforms that can be decomposed into simple waveforms of different periodicity at varying amplitudes. EEG researchers often are interested in five frequency bands: delta (<4 Hz), theta (4–8 Hz), alpha (8–13 Hz), beta (13–20 Hz), and gamma (>20) [2]. In this study we focus on the alpha band because of the inverse relationship between alpha frequency amplitude and attention, wherein lower levels of alpha represent higher levels of cognitive processing. This phenomenon is referred to as “alpha blocking” [3–5].

This study focuses on the alpha band because of a plethora of findings relating the alpha wave to creative processes. At this point it is important to delineate different creative processes and their effects on cortical activation and the alpha rhythm. In an extensive review of the EEG and ERP literature, Dietrich and Kanso [6] discuss the findings of many EEG studies examining divergent thinking, artistic creativity and insight. There findings indicate that much of the divergent thinking literature findings relating to the alpha wave is contradictory, but the findings on insight are consistent [6]. In one study of creativity, a coherence analysis of EEG data (i.e., showing correlations of activity among different electrode sites) found that when examining the alpha band in groups of participants, increased coherence is observed throughout the alpha band, exhibiting intra- and interhemispherical long-distance coherences [7] and, more specifically higher levels of alpha in response to creative tasks, such as alternate uses tasks [8, 9]. Other creativity findings have shown increases in alpha power being associated with creativity (both divergent thinking and artistic) [10]. In the realm of divergent thinking, findings have indicated that disinhibition (increases in power within frequency bands) corresponds to increased creativity [11]. Another line of EEG research has found that increases in the alpha rhythm in the frontal cortices during creative ideation [12]. However, these findings note an intriguing nuance. Increases in the alpha rhythm are found only during internally oriented attention that is characterized by the absence of bottom-up stimulation [12]. When creative generation is externally-directed, however, decreases in alpha power have been observed [12]. In the context of Flip-Flop creative generation it is important to note that external stimulation is occurring as the individual interacts with the quiz design tool to generate quiz ideas.

One consistent finding in the EEG creativity literature deals with insight. Insight refers to acquisition of a new understanding of the problem situation after repeated attempts to solve the problem [13]. Studies have shown that creative insight problem solving requires less working memory than other problem solving [14]. Research on insight using EEG has found consistently a decrease in alpha power in frontal, temporal and parietal sites [6]. Idea generation is process that links two or more separate concepts in working memory to produce a new idea [15]. There are two different processes to this linking, search and insight, with search being a more methodical process and insight being more creative [16, 17]. Methodical thinking is commonly associated with activation of the frontal cortex [18, 19]. Creativity and insight have been studied in a variety of contexts using EEG [6]. Findings on insight have been fairly consistent, with several EEG studies finding decreases in alpha power in the frontal cortices to be connected to insight [20–22]. In generating a quiz question within the Flip-Flop environment, it is expected that the students will generate insightful thinking.

In addition to creativity, the current research will examine the cognitive load of the students while they generate quiz questions in the Flip-Flop environment. Working memory plays a central role in cognition and cognitive load [23]. It encapsulates both what many consider “short-term memory” and attention. Therefore, working memory is pivotal for both information processing and decision making, responsible for encoding information from the environment and retrieving information from long-term memory in order to make sense of it [23–25]. A useful computer analogy for understanding working memory is that it represents the brain’s RAM, storing of information currently undergoing processing but limited in its capacity [26]. Working memory is located in the frontal areas of cortex, namely Dorsolateral Prefrontal Cortex (DLPFC) [27]. Changes in activity in the DLPFC can indicate changes in working memory load and attention [18, 28]. In EEG, attenuation of the alpha rhythm over DLPFC indicates increases in working memory load [29]. Better performance has been found as a result of increased activity in this region [30]. Initially using the Flip-Flop tool to generate quiz questions should result in increased levels of cognitive load. However, as the student becomes used to the generative process and the tool, it is expected the cognitive load required to generate quizzes will decrease, despite an increase in the difficulty of the questions they are generating.

3 Method

3.1 Experimental Context and Participants

The “Algorithms” course is a key course at Department of Information and Computer Sciences at University of Hawaii - it is the prerequisite for most of the senior level courses and no student will be able to graduate without passing it with a satisfactory grade. The majority of the students consider this course to be the most demanding course in the curriculum.

This course has been taught in the inverted classroom fashion for the last five semesters. The lectures consist of 72 screencasts, mostly 15–20 min long. The students have to view typically 3 or 4 screencasts at home before they come to a classroom session. There are two such classroom sessions per week. Every screencast set is accompanied by a web page with detailed notes and the students are encouraged to view these notes while watching the videos. During the classroom session the students first take a paper quiz, and then solve class problems that practice the topics presented in the corresponding set of screencasts.

The Flip-Flop method is being applied in a pilot fashion in the “Algorithms” course during the current semester. Preliminary results indicate that while student are concerned about the extra time needed to make and take the quizzes, they are more than willing to take on these chores to gain bonus points. Moreover, the quality of the quiz tasks seems to have improved considerably within only a couple of weeks.

The Flip-Flop method will be applied systematically in the aforementioned “Algorithms” course in the upcoming Fall 2016 semester. We expect that 20 to 30 students will enroll in each of the two sections. After an initial introductory classroom session the students will complete a ‘warm-up’ exercise where they will make a quiz from a YouTube video of their choice. Based on our experience from current pilot course, the students a likely to choose a topic related to their hobbies, favorite sports, movie trailers, music videos, etc.

In subsequent weeks, both course sections will be subdivided into groups of up to four students. The Flip-Flop scheduling app will then subdivide each screencast into four segments and assign each student one of the segments to make a quiz from. This tool also allows the student author to download a template that she can then upload within the quiz authoring tool to define a set of default multiple-choice tasks as well as the set of peer evaluation poll questions asked at the end of the quiz. Since the default length of each task is one minute, most screencasts are 15 to 20 min long and each is subdivided among four authors, the template typically defines 4 to 5 tasks. This template greatly simplifies the menial chores of defining the components of a task – question, answers, feedbacks, hints and links to online resources - as well as of synchronizing the tasks with the video segments.

Once the author has created the quiz, she can use this app to send an email to the other three the students in the group announcing that the quiz is ready and they can take it. Note that this way every student creates one quiz and takes three other quizzes from a screencast. As a consequence the quizzes she makes or takes cover the entire length of this video.

We plan to ‘flip-flop’ alternate set of screencasts for each of the sections. I.e., suppose that the classroom sessions will be scheduled for Monday and Wednesday for both sections – then the first section will make and take the quizzes from screencasts for the Monday’s session and the other section from screencasts for the Wednesday’s session. In other words, the sections will use Flip-Flop for different sets of screencasts. Since the paper quizzes as well as the problems on midterms and final exam will be identical for both sections and will be closely related to the topics of the individual screencasts, we will be able to contrast the student’s performance on problems where they either created or took a quiz with the problems related to screencast that they only needed to watch. Our hypothesis is that they will receive significantly more points on problems related to their ‘flip-flop’ topics.

To be able to assess the quality of the quizzes students created, our instructor support tool will integrate evaluation facilities that will allow a rater to give points to all of the individual components of a task: the questions, answers, answer feedbacks, hints and links to related resources. Our hypothesis is that the ratings for all the task components will be significantly better towards the end of the semester.

Furthermore, we plan to augment the quiz authoring software to collect data while the author is creating a quiz. Obviously, it will be interesting to investigate the time needed to create the entire quiz. Moreover, the times needed to create a question, to come up with the correct and incorrect answers, and to formulate a hint will be collected. Our hypothesis is that despite the fact that the course topics are more difficult towards the end of the semester, all these times will become gradually improve, and at the end of the semester they will be significantly shorter than at the beginning of the semester.

Our intuition as instructors lets us believe that creating an incorrect answer is more difficult that creating the correct answer – after all, an incorrect answer should be neither partially correct nor obviously wrong. Furthermore, a good task should have several incorrect choices so that guessing the right choice is less likely and coming up with yet another ‘good’ incorrect answer seems to be increasingly difficult. The analysis of the quiz-making data will show us whether this hypothesis is correct.

In addition to the above data analysis we will let the students fill out a survey questionnaire asking for their assessment of the Flip-Flop method at the beginning, after the second week, and at the end of the semester. Our hypothesis is that they will be initially inclined to give rather positive feedback given that they could freely choose the subject of the warm-up exercise. We expect, however, that the students will be much more critical once they had to make a few quizzes based on the actual course screencasts. (After all, taking quizzes and in particular making quizzes takes time). Finally, their assessment at the end of semester is likely to be more positive as quiz-making and -taking becomes merely another routine.

3.2 Neurophysiological and Psychophysiological Measurement of Flip-Flop

Our dependent variables are cortical alpha wave activity, autonomic arousal, and emotional valence. These are operationalized using neurological and psychophysiological measures. EEG measures will be collected using a 14-channel headset (Emotiv Systems, San Francisco, CA, USA) with electrodes dispersed over the scalp along the 10–20 system [31] (see Fig. 1). The electrodes connect with the scalp surface via felt pads saturated in saline solution. The reference electrodes are located at P3 and P4 over the inferior, posterior parietal lobule [31]. All other channels were measured in relation to the electrical activity present at these locations, sampled at 128 Hz. Impedances were verified and data collected using Emotiv TestBench Software Version 1.5.0.3, which export into comma-delimited format for subsequent analysis (Fig. 3).

Fig. 3.
figure 3

Position of the electrodes on the EEG headset with labels along the 10–20 system

Full size image

Autonomic arousal will be operationalized as skin conductance level measured with disposable electrodes filled with electrically neutral gel and adhered planar surface of the foot. A Biopac MP150 system will be used to collect the skin conductance data at 1000 Hz. Emotional valence will be operationalized as the relative activation of the corrugator supercilli muscle group (facial EMG). Corrugator EMG will be measured using a pair of mini (4 mm) reusable AG/AGCL electrodes filled with electrolyte gel placed above the subject’s left eye after dead skin cells has been removed by a skin prep pad containing rubbing alcohol and pumice. The bipolar corrugator measures were collected using the Biopac MP150 system with high pass filters set at 8 Hz. The full wave signal was rectified and then contour integrated online at a time constant of 100 ms, and then sampled at 1000 Hz by the Biopac MP150 system.

4 Future Directions

The empirical examination of the Flip-Flop tool opens many doors in the educational arena. First, the neurophysiological and psychophysiological correlates of using this method in the classroom will be elucidated. There are many different ways individuals learn and much research in psychology has examined it. However, using EEG and psychophysiology in such an applied setting can provide insight into how to better engage students in the process of learning. Tying these to behavioral outcomes (i.e., surveys and instructor evaluations) will provide invaluable feedback in how to develop tools that help facilitate learning. Furthermore, the use of the tools from cognitive neuroscience could spawn a new area of research that can be used in a variety of academic settings to evaluate pedagogical tools and their outcomes. We refer to this future research area as “NeuroEDU.” NeuroEDU utilizes the tools of cognitive neuroscience (EEG, fMRI, fNIRS, etc) to understand and facilitate the development or improvement of pedagogical methods or tools. Our study will provide the first longitudinal investigation of a developed tool that ties both neurophysiology and psychophysiology to course outcomes. Future NeuroEDU research can examine the effectiveness of Flip-Flop and other emerging educational methodologies for different age groups – in particular at high-school and elementary school level –, different subject areas – such as social sciences and arts – as well as in different educational settings – for instance in commercial training courses.