Keywords

1 Introduction

1.1 Problem-Based Learning (PBL)

Problem based learning (PBL) is a student-centered pedagogy in which students collaborate with each other to solve complex and ill-structured problems (Hmelo-Silver and Barrows 2006). In this pedagogy, the objective of the instruction is to help students learn knowledge and reasoning skills as they learn in context. In it, the teacher’s role switches from providing content knowledge to facilitating and guiding the students’ interactions towards their learning goals (Hung et al. 2008). There are many benefits to this student-centered pedagogical approach, such as improving students’ problem-solving and group-work skills, enriching their higher order thinking, motivation and deep internalization of students’ knowledge. By solving complex real-world problems in a PBL setting, such important skills can be developed to enhance students’ academic and professional competencies.

1.2 What is Involved in Providing PBL?

Context.

Ill-structured real-world problems (often interdisciplinary) provide the context of the PBL environment. Although PBL has been traditionally used in the medical domain, it has been extended to other domains; e.g. mathematics, psychology, business education.

Collaboration spaces.

A Learning Space is a distinguished social feature in PBL (Roscelle and Teasley 1995). It serves as a platform to afford learning materials and tools for learners to discuss key concepts and critical knowledge (Hmelo-Silver 2013). By participating in collaborative activities, learners share thoughts, discuss rationale, negotiate conflicts, and generate recommendations to solve complex problems. According to Hmelo-Silver (2013), a learning space could be divided into: (a) a problem space to discuss general content related with a problem case; and (b) a related conceptual space to discuss specific problems and related concepts at hand.

Scaffolding learners.

Teachers need to take roles as facilitator whose main tasks should be modeling, coaching and fading (Hmelo-Silver and Barrows 2006). Scaffolding is support provided to students based on their personal needs, learning processes and levels of understanding. As students take more responsibilities for their own learning and become experienced and advanced in PBL, instructors fade their support progressively to motivate students to take more control of their learning.

Scaffolding facilitators.

In complex PBL situations (e.g., multiple PBL groups or novel technology-supported PBL environments), facilitators themselves may require additional support. Using a “Wizard of Oz” approach a wizard teacher can support the other facilitators (Lajoie et al. 2014). In particular, wizards support the facilitators by observing the teaching process and noticing issues with the instructional content and student interactions that the facilitator might have overlooked. On such occasions, wizards may provide support by discussing and reminding facilitators to organize their instruction to correspond with students’ dynamic learning processes and the development of learning activities.

1.3 Challenges of PBL

Various limitations of PBL have been discussed for some time. Two much-discussed limitations are: (a) the difficulty of scaling up small-group (b) challenges in assessing individual learning outcomes, etc. (Martinez-Maldonado et al. 2012). One main challenge of traditional PBL is the low student-teacher ratio. Students tackle questions that are significantly more open-ended and ill-structured than those in traditional instruction, and hence success often hinges on facilitation, monitoring and guidance from the instructor. The increased demand on instructing the team has made traditional forms of PBL difficult and costly to implement in large-enrolment courses.

Efforts to scale-up while maintaining the pedagogical approach have included peer tutoring, and facilitators that periodically visit multiple groups (Hmelo-Silver 2013). Other efforts have modified the pedagogical model to reduce the need for a facilitator (Abdelkhalek et al. 2010). However, research to date shows that when fostering multiple small-group PBLs simultaneously, facilitators’ awareness of individual small-group PBL interactions is limited and requires considerable additional support (Martinez Maldonado et al. 2012). Our design aims to structure pedagogy and technology to provide this additional support to PBL facilitators working with multiple collaborative groups.

1.4 Technology-Enhanced PBL

Breakthroughs in technology may empower facilitators and boost their instructional capacity, allowing larger numbers of learners to participate and interact within a scaled-up PBL setting. Many researchers are interested in using technology with opportunities for supporting and scaffolding learners in a PBL context (e.g., Lajoie et al. 2014). Computer-supported PBL broadens the range of application of PBL across cultures and disciplines, and enables learners to interact and connect beyond boundaries of location, time and space. Technology may also support scaling-up PBL by allowing instructors to facilitate multiple groups at the same time. Recent PBL environments are structured and designed in technology-rich contexts (e.g., computer-supported learning environments). New technologies and cognitive tools have been exploited to enhance and empower the development of PBL. The features of technology-rich PBL environments may include: (1) shared collaborative learning spaces, (2) a collection of computer-mediated cognitive tools, and (3) use of visualizations. PBL has also been widely used in online learning environments via different models and media. Online PBL can be delivered in asynchronous or synchronous communications, or a mixture of the two. It has gained great popularity in recent years for its many instructional capabilities and its adaptability to a variety of contexts.

The components of online PBL contribute in different ways: whiteboards can be designed to foster students’ brainstorming activities; chat spaces can promote students’ flow of thought; threaded discussion forums encourage the exchange and discussion of ideas; other interactive and collaborative learning spaces may increase learners’ engagement, creativity, reflection and productivity (Jonassen 1995).

2 Value of the Present Research

This research aims to investigate ways to support PBL facilitators working with multiple small PBL groups. To do so, we designed online tools to boost instructional capacity. These tools were brought together in an online learning environment entitled HOWARD (Helping Others With Argumentation and Reasoning Dashboard). It is expected that the PBL methodology in tandem with the technological competences of the designed learning environment will aid in making PBL-type teaching methods viable option for large courses. The platform may also allow for the implementation of courses across countries and cultures, aiding students to tackle problems of intercultural communication and awareness of context.

3 The Study

This research is situated in the context of an important but minimally attended medical domain: that of effectively breaking bad news to patients (Baile et al. 2000). In the realm of medical instruction and practice, emphasis has been placed in recent years to the relevance of communicational skills in the context of health services, and the impact that these soft skills have in proficient medical practice. One of these is the communication of problematic news. The manner in which this is done can alter a patient’s course of decisions and actions, potentially impacting on his/her relations with the health system and treatment, emotional and mental health and, importantly, his/her immediate social context (family, workplace, friends). The way in which, for example, a person is informed about a venereal disease and whether or not this information should be shared with a spouse or partner has immediate consequences in the health of a whole family group.

The PBL course developed in the context of this research aims to help medical students learn from the critical assessment and group discussion of contrasting video-case based scenarios to foster their clinical decision making, and also enrich their communication skills in challenging situations about emotionally sensitive issues. Third and fourth year medical students will be recruited and take an online multiday workshop, with synchronous and asynchronous capabilities, and with the assessment and facilitation of a group of instructors. Students are organized in groups, which may belong to the same medical school or multi-cultural groups from different parts of the globe. The groups have deadlines and the course is taken in a given time frame (synchronous), but the individual activities each group member performs can be executed asynchronously. All of the relevant activities are conducted online through HOWARD.

4 HOWARD Interface

As described above, The HOWARD system is a web application designed to support asynchronous PBL workshops. This workshop involves three types of users: students, instructors and instructor-facilitators (who aid the instructors). Each user type participates in the workshop differently. Figure 1 provides an overview of the different user types and their interactions.

Fig. 1.
figure 1

User interactions during a HOWARD workshop

Full size image

The overall layout for the site is similar for the different user types. A video column on the left side of the site provides access to video materials. A navigation bar at the bottom allows learners to access the site’s different features, which are loaded dynamically into the main working space in the center of the screen. Hereafter we describe how this structure is used to implement user interfaces for the three user types in our workshops, beginning with students.

4.1 Student Interface

Students who log in for a workshop for the first time are directed to the guide page. The guide provides background information on the aims of the workshop (i.e., giving bad news) and the instruction method (i.e., PBL). Brief introductions to the website include frequently-asked-questions and a video guide providing an overview of the system.

After reading through the guide, learners are expected to visit the “Today’s Tasks” page. This page presents learners with a list of tasks to complete each day of the workshop and serves as pedagogical scaffolding to help students stay engaged, while providing them with a sense of progress. The task list is updated daily by an instructor. The general work-flow would involve students visiting the task list, and then visiting other parts of the site to complete those tasks. At present, the learner is responsible for tracking their progress through the workshop using the list. As they complete tasks, they return back to the task page to mark items complete. To help avoid having learners forget to update the status of their tasks, a reminder prompt is shown when learners log out from the system.

In order to replicate the advantages of traditional PBL in new technological PBL environments face-to-face PBL in HOWARD, we have attempted to provide an environment which adapts this learning approach to an asynchronous on-line environment. Our approach is to combine a discussion space, implemented as a threaded chat, with a collaborative text editor that can be used as a whiteboard.

Figure 2 shows the interface we have developed for group collaboration. Learners access the space using the ‘Home’ item on the navigation bar. The middle area of the screen is the discussion area. This collaborative writing space enables the traditional affordances of group work and discussion, such as sharing thoughts, discussing perspectives, negotiating conflict and designing and generating recommendations, while at the same time allowing the students insights to be available not only for further revision, but for the assessment by the instructing team. Special steps have been taken to facilitate asynchronous communication (to support learners from different time zones). First, learners are notified of new messages or whiteboard edits by means of a small badge-style notification indicator that appears on the navigation bar. Second, color is used to distinguish between old posts that the user has seen (colored white) with new posts that the user has not yet read (colored blue). After a user has read a post, they mark it as ‘read’ by clicking on the provided ‘Read’ link which changes the color to white. Because new posts can appear for both the most recent and older conversations, indicators visible on the bottom left of the discussion area are provided. These show learners how many unread posts exist and where these posts are located relative to the currently visible area (either above or below).

Fig. 2.
figure 2

The video annotation interface. Annotations students create are visible to other group members to promote discussion.

Full size image

A collaborative whiteboard is located to the right of the discussion area. The whiteboard is built on the open-source text-editing platform Etherpad that allows learners to collaboratively write in real-time. Typically, this space is used as a real whiteboard would be, to summarize and record important points from the discussions that take place. Background color is used to attribute regions of text to their authors, with each learner having a unique color within a group.

To approximate the ability in face-to-face PBL to point to things on a real whiteboard, we have developed a group workspace feature that allows learners to link their posts in the discussion area with text located on the whiteboard. This is accomplished by selecting a region of text, typing a message in the discussion area and then posting. For other students who are reading these linked posts, locating the target text is accomplished by clicking a look-up button on the top-right corner of the post. Look-ups can also be performed from the whiteboard. In both cases, the relevant text is highlighted and automatically scrolled into view.

The video materials visible in the video column to the left of the group work space are a second important workshop resource. Videos are the medium used to present students with cases, i.e. examples of doctors providing patients with bad news. In the present course design, we provide two cases situated in different socio-cultural contexts, Montreal and Hong Kong. As a way of increasing engagement, we have leveraged code from the Open Video Annotation Project to implement an interface for annotating the videos. Learners first select a video to annotate from the video column, after which the video annotation tool loads in the main working space (see Fig. 3).

Fig. 3.
figure 3

The student user interface, consisting of navigation bar (1), video column (2) discussion space (3) and collaborative whiteboard (4). New chat messages appear blue (5). Chat messages allow students to link to whiteboard text (6) (Color figure online).

Full size image

The video annotation tool allows learners to select regions of the video and attach a comment or observation. These annotations foster a dialogue among the students across time zones (specifically in cases of international collaboration), encouraging peer-based and intercultural learning, with the students feeding from the reflections and interactions with other members of the group.

A third and final key component of the online workshop is related to assessment. To evaluate progress made during the workshop, learners complete two reflective writing activities on private “Reflection” whiteboards shared only with their instructor. The interface provided by a private whiteboard is identical to a group whiteboard made full-screen without a discussion space, similar to a typical word processing environment. These private whiteboards contain questions for the student to reflect on and answer, and a submission button to notify an instructor when an assignment is ready for review. The system notifies a learner when their assignment has been reviewed using the same badge-style notification employed for group discussion changes. Feedback on the assignments, as well as other general messages from instructors, can be accessed via the ‘Instructor Input’ menu item which launches an email-style inbox.

Before describing the instructor view of the platform, we first briefly mention that the actions that learners perform while using the system are logged. The purpose of this logging is two-fold: first, it allows a learner analytics module to process the log data and generate visualizations and other summaries to provide instructors with information on the activities of the group that help to gauge their progress and participation and flag groups or individuals in need of attention. Second, the log data provides researchers with a description of the learning process and an opportunity to associate behavior patterns during the activities with learning outcomes. This can also be used to investigate whether a design feature has the intended effect. We now present the instructor view of the system.

4.2 Instructor Interface

The instructor interface provides access to information on the participation and progress of the groups and a means to provide feedback on the group discussions and activities. For brevity, we focus on one aspect of the interface: the dashboard which is designed to facilitate the monitoring of group activities. Each group is represented on the dashboard page with four visualizations arranged in a row (see Fig. 4). We describe each of the visualizations below.

Fig. 4.
figure 4

The instructor dashboard provides information on student participation and progress (1), recent activities (2), interaction trends between group members (3) and commonly discussed terms (4) (Color figure online).

Full size image

The left-most area of the dashboard shows information reflecting individual and group ‘health’. A pie chart breaks down the participation levels of each student by analyzing the number of words they type in their group discussions and on the whiteboard. Below the pie chart, progress bars display various group-level information. The first progress bar shows group participation relative to other groups. If groups participate approximately the same amount, then all group bars appear green. However, for groups that deviate from each other, the bars for low-ranking groups are flagged with yellow or red to attract the instructor’s attention. The second progress bar shows the overall progress for the group, measured as the mean level of tasks completed. A more detailed view of individual students’ progress status is available by clicking on the label. The bottom-most progress bars show two related measures of student-instructor interaction. The student-instructor act ratio shows amount of attention the group has received, while instructor focus shows the amount of attention received relative to the other groups.

To the right, we see the ‘Latest Activities’ news feed which lists the various activities of group members in reverse chronological order. To avoid a cluttered display, the details of the acts are hidden until the instructor hovers over an event with the mouse. In some cases, the notification is provided as a hyperlink to provide convenient access to the particular discussion post, video annotation or other written content produced by the learners. An ‘instructor only’ filter can be applied to the notification feed to make it easier for instructors to review the activities they have completed. The visualization located at the top-right of the dashboard is a social network analysis diagram representing the interactions of the group members in the discussion space. Student users are represented by nodes. Larger nodes indicate more output and color indicating their location. Arcs joining the nodes show which given learner has replied to whom in the discussion space, with thicker arcs indicating more words exchanged. Finally, a word cloud shows the frequency of different words being used in the discussions and the whiteboard, with more frequent words appearing larger. Common stop words are excluded.

Besides the dashboard, there are a number of other menu items on the instructor navigation bar. We briefly describe those:

  1. 1.

    Student whiteboards: This menu item provides links to discussion spaces for each group the instructor is managing. A discussion space loads as it would appear to the students, and the instructor can observe what is happening more closely, or contribute to discussions within the chat space or the whiteboard.

  2. 2.

    Send Input: This feature is for sending notifications to a group or to all groups.

  3. 3.

    Assignments: Allows instructors to see which students have submitted what assignments at a glance. When a student submits an assignment, an indicator appears which also links directly to the assignment. When instructors have reviewed the assignment and left feedback, they can click on a button to notify the student.

  4. 4.

    Wizard chat: This item provides access to a private space where instructors and wizards can discuss any concerns that come up during the course of a workshop. The space is identical to a student group’s work space, but access is limited to the instructor and the wizard.

  5. 5.

    Materials: Clicking on the item loads a tabbed page where instructors can perform administrative functions typical of a learning system, such as creating users, managing groups and creating or distributing learning materials.

4.3 The Instructor-Facilitator Interface

In addition to students and instructors that take part in a workshop, there are also ‘wizards’; these are PBL facilitators who monitor the interactions of the instructors and their groups but do not interact directly with the students. Instead, they function as a coach who provides tips and guidance to the instructors during a workshop. This is specifically in order to support facilitators in managing the cognitive load of their multiple group PBL instruction.

Their web interface is much the same as the instructors, i.e. they have access to the dashboard visualizations and the group discussions as well as the other menu items. Two aspects of the wizard interface are different from the other groups:

  1. 1.

    ‘Mark Flag’ notifications: Wizards have the ability to send messages to instructors, via the ‘Mark Flag’ item, to direct their attention to different areas of the site. Creating a ‘Mark Flag’ message is similar to creating a typical email message, with the exception that these messages record page location information (the page the wizard is viewing while typing the message). When an instructor receives a ‘Mark Flag’ message in their notification feed, they can click on it to access it.

  2. 2.

    Instructor Chats: A final menu item specific to the wizard interface is the ‘Instructor Chats’ item, which provides links to the Wizard-Instructor private chat areas for each of the instructors.

5 Value and Future Directions

We conducted a pilot test of the platform in a week-long workshop for medical students, focused on breaking bad news. Instructors used HOWARD to monitor PBL groups and provide support to students as they reasoned through a bad news delivery case. As our aim is to increase the instructional capacity of PBL instructors, our preliminary analysis is focused on their instructors’ felt needs and goals.

Facilitation is a primary concern for instructors. During post-interviews, instructors focused on three values: (1) gaining a sense of students’ understanding to encourage participation and emotional engagement, and (2) managing the multiple interfaces and pathways to respond to students and the instructor-facilitator. (3) the value of and need for finding ways to conduct PBL online.

The asynchronous aspect of the design creates a challenge for instructors in terms of having a sense of what students’ instructional needs. As one instructor noted:

When I am teaching a student live, just by looking at their body language I can tell. Are they in distress, or are they liking this? In the online system, you can’t tell….I lose the insight that I can get from having a human conversation to know, is this teaching on task, on target, or not? In a live setting, I would be able to pick that up in body language very quickly, as a tutor. In a pure online asynchronous setting, I’m completely blind to that…Students are trained to do their medical writing in as dispassionate, and in an unemotional, like, this is science, this is what I observed… So students are writing as if they would be writing medical records, they would deliberately hide any emotions, or deliberately remove any emotions from what it is that they would be writing down.

This finding suggests that future interventions should take a broader consideration of organization-specific cultures when designing for online environments and modes of communication. Beyond national cultures, organizational cultures may impact students’ participation and emotional engagement.

The value of PBL in medical education has been strongly voiced by the medical community for many years. The primary benefit is that students encounter a problem as they would in actual practice: as ill-structured, complex, and often lacking a correct answer. As one of our workshop instructors noted:

“I guess that, part of it is that, medical students so often feel, “I am not good enough, I’m never going to be as good as Dr. Y.” I spend a lot of time as an educator, helping them realize that their opinion counts so much, that sometimes it is more important than what is in the literature. It is important because later on, when you go into practice as a physician, there are many questions in medicine that cannot be answered by the literature. And so then you are still faced with having to come to, what seems to be a reasonable decision with your patient. And people call that non-evidence based. But that’s 99 % of medicine. So that what they are doing in these small groups, is that in a way, they are doing in my opinion, a very authentic medical thinking activity, where their own opinion and their own conceptions and misconceptions form a very important part of what it is they are thinking about and what they are doing.”

HOWARD is designed to help both the student and the teacher by providing asynchronous tools for supporting individual and group interactions to foster PBL discussions about patient cases. The HOWARD tools support instructors through learning analytic metrics about the group process as well as through a ‘Wizard of Oz’ technique to support teacher attention to the instructional context.

The value of our design is in finding ways to make PBL more accessible by working to extend the instructional capacity of a PBL instructor. However, we are finding that an iterative design process is necessary, as each new technological capability brings with it new challenges to address. Focused on preserving the core value of authentic and ill-structured problems, and the challenges of facilitating, our future directions include (1) testing the design in a multi-group PBL instructional session, (2) Reduction of features and pathways, and (3) refining dashboard visualizations to focus on learning-relevant details (rather than activity-relevant details).