Keywords

1 Introduction

In recent years, cooperative work skills among experts in different fields such as science, technology, and sociology have become important. As an example of this, corporate society requests that universities cultivate fundamental competencies for working persons, which consist of three competencies (action, thinking, teamwork) and 12 capacity elements [1]. To develop these skills in a university setting, it is necessary to provide an environment for students to work together outside the frameworks of their individual faculties and departments. For this reason, we are aiming to build a virtual learning environment in which students who have different academic backgrounds and cultures can communicate without limitations due to spatial constraints.

Seminars are held by the education and research communities of universities [2]. Each seminar in Japan generally consists of a few teachers and 10 to 20 students. Seminars are intended to be opportunities for acquiring high level expertise through peer teaching and learning. Collaborative learning methods conducted by students under the support of faculty in such seminars are evaluated as effective human resource development methods beyond mere expert knowledge learning [3]. In recent years, research empirically investigating the learning effect of seminars has shown that activities that are suited to student situations, such as support for group activities and job hunting activities, are useful for improving student generic skills [4].

In this research, we conceive, propose, and develop a system that supports seminar activities by a single community in order to expand the educational function of seminar activities [5,6,7,8]. In addition to this, we consider that promoting exchanges among communities, which was not so active so far, will lead to the development of new educational functions.

Each seminar is focused on its own specific knowledge. Therefore, we believe that communication between a wide variety of seminars would be effective for the development of cooperation skills. In this study, we call the generic activities performed through cooperation among multiple seminars a “virtual seminar.”

We define “Virtual Seminar” as a collaborative learning method implemented by multiple seminars having diverse academic cultures. Joint seminars that connect technically multiple seminars with a TV conference system are becoming familiar. However, many of the implementations are extension of normal seminar function among seminars sharing same expertise. The learning goals of Virtual Seminar are totally different from those of existent joint seminars. The goals are to know future customers and colleagues at university education stage by exchanging completely different expertise associating with diversity of cultures and values on equal terms, and to have a panoramic perspective.

The purpose of this study is to develop a system for supporting the construction and operation of virtual seminars. The design requirements include the combination of seminars, learning tasks, exchange environment, and group composition for discussion. In the operation of a virtual seminar, a mechanism is needed to appropriately assess the collaboration skills training. However, because seminars are diverse, it is difficult to define a unified design model and evaluation model that takes into account the diversity of expertise. Thus, by utilizing behavioral data in practice, it is necessary to consider a descriptive approach for expanding building support and operational support on the basis of data analysis.

To realize the construction and operation support of virtual seminar based on the data, a framework for storing data about the seminar activities is needed. We define the system as a seminar management system (SMS) that has data generation and data analysis functions for seminar activities. This study is thus an attempt to develop an SMS. We have already run a virtual seminar, and the aim of this paper is to elucidate the design guidelines for virtual seminars through the analysis of the data obtained from the SMS.

2 Concepts and Implementation of SMS

Figure 1 shows an SMS compared with other management systems. Learning management systems (LMSs) are responsible for the operation and management of lessons and have been introduced in many universities. Meanwhile, social networking services (SNSs) are systems for supporting the formation of virtual learning communities. This study is intended for data management of a community formed from multiple seminars (community), and it is necessary to employ different management schemes from the past.

Fig. 1.
figure 1

Support by SMS

Full size image

Figure 2 shows an implementation of the virtual seminar in this study. The SMS is responsible for storing the role and action history data that link the different seminars. In this study, this includes the comments in the seminar activities, scale rating, and the ability to accumulate action history data through video recording and other means. To enable communication with learners in remote locations, video conferencing and text chat functions are added to the SMS.

Fig. 2.
figure 2

Model of a virtual seminar mediated by SMS

Full size image

3 Practice

3.1 Overview

Data stored in the SMS obtained by running a virtual seminar are analyzed to reveal the design guidelines for virtual seminars. The virtual seminar was conducted with the cooperation of seminars at Waseda University (seminar A) and Kyoritsu Women’s University (seminar B) from April 2017 to July 2017. Table 1 shows a schedule of the seminars. Adlib speech is an activity in which students give a speech on a specified theme for a specific length of time. To train the ability to summarize a story in a short space of time, the speech time is set to 1 min. Virtual group discussion (VGD) is a discussion conducted in groups between students who belong to different seminars. To communicate with other learners in remote locations, students use text chat.

Table 1. Schedule of virtual seminars
Full size table

Figure 3 shows a scene of the virtual seminar. Seminars at Waseda University, Kyoritsu Women’s University, and the University of Kitakyushu are connected via Skype, an Internet communication tool. Faculty belonging to the University of Kitakyushu serve as facilitators, and the seminars of the other two universities are interacting. At this time, students at Waseda University are giving a speech, whereas students at Kyoritsu Women’s University are inputting assessments/comments using smartphones.

Fig. 3.
figure 3

Scene of a virtual seminar between multiple universities

Full size image

3.2 Adlib Speech

Figure 4 shows the random selection function of the speech theme. This function randomly presents a pre-accumulated speech theme. Students do not know the theme in advance. Therefore, students are required to structure their speech instantaneously.

Fig. 4.
figure 4

Random theme selection function of adlib speech activity

Full size image

Figure 5 shows an input screen for speech evaluation. Assuming various users in the virtual seminar, the user interface of the system in previous research [7, 8] was updated to support input by smartphones. Assessment from each viewpoint, free description of good points and points for improvement, and comprehensive assessment for how good the speech is can be input. Assessments/comments collected by this function are aggregated and are fed back to the speaker instantaneously.

Fig. 5.
figure 5

Input screen for assessment

Full size image

Figure 6 shows the feedback screen. Participants can use smartphones to check assessment results. Assessment can be visualized using a radar chart. Self assessment, peer assessment, and teacher assessment overlap in one chart. Free description data such as good points and points for improvement are also summarized.

Fig. 6.
figure 6

Feedback screen

Full size image

3.3 Virtual Group Discussion

VGD is a method of discussing by groups composed of mixtures of seminars from multiple universities (virtual group composition). Figure 7 shows a schematic diagram of VGD. VGD basically uses text chat to allow students in remote areas to discuss at the same time in multiple groups. The video conference system is used for announcing each discussion result. This method enables discussion among students with different academic cultures, group reconstruction in limited class hours, observation of discussion by other groups asynchronously, and so on.

Fig. 7.
figure 7

Concept of virtual grouping

Full size image

Figure 8 shows the screen of text chat by SMS. Individual rooms are created so that groups can be changed dynamically and each group can share one room.

Fig. 8.
figure 8

Text chat screen on SMS for VGD

Full size image

3.4 Assessment for Each Activity

Users can set evaluation items for each activity using SMS. Table 2 shows the evaluation items used for the speech activity. It consists of nine items on the behavior of the speaker and the impression given to the audience.

Table 2. Assessment Items for adlib speech
Full size table

Table 3 shows the evaluation items for discussion activities. This is a self-evaluation item. Learners evaluate what kinds of difficulties arise during discussion activities. Using these data, it is possible to develop better activities.

Table 3. Assessment items for virtual group discussion
Full size table

4 Analysis of Seminar Activity Using SMS Data

To clarify the design guidelines for virtual seminars, we investigated the following research questions.

  1. 1.

    What points are important in the virtual seminar in a remote environment compared with a face-to-face (FTF) environment?

  2. 2.

    What points are important when making groups in VGD?

The data collected through running the seminars mentioned in Sect. 3 were analyzed in order to clarify these two research questions.

4.1 Comparison Between FTF Environment and Remote Environment

To address research question (1), FTF environment and remote environment were compared based on data about the adlib speech activity in Table 1. Since historical data in an FTF environment were accumulated by the SMS in previous research [7, 8], it is possible to compare the FTF and remote environments.

  1. (i)

    Comparison of average values of assessment items by t-test

Average values in the FTF environment (FTF group) and those in the remote environment (virtual group) were compared by t-test. Table 4 shows the result. We found that the evaluation of non-verbal communication tended to be lower in the virtual group than in the FTF group (Qp1–Qp7). Thus, one point that needs improvement in virtual seminars is to improve non-verbal communication to make the remote learning environment more similar to the FTF learning environment.

Table 4. Comparative analysis by t-test between the FTF and virtual groups
Full size table

However, the difference in assessment regarding the impression felt by the audience was not significant (Qp8–Qp9). We assume that these impressions strongly depend on the content of the speech. This is therefore not a matter of differences in learning environment.

  1. (ii)

    Comparison of determinants based on multiple regression analysis.

To apply multiple regression analysis, the overall rating of speeches (Qp9) is treated as an objective variable, and assessment values of items related to verbal and non-verbal speech techniques are treated as explanatory variables. By analyzing and comparing the determinants between learning environments, we seek to clarify the important factors for improving the remote learning environment.

As a result of the analysis (Table 5), we found commonalities and differences between the two environments. Scores of determinants “speaker confidence (Qp2)” and “consideration of listeners (Qp4)” are commonly high, while scores of the determinant “voice volume (Qp1)” were low. For differences between the learning environments, “eye contact (Qp3)” and “smiling (Qp5)” were significant as determinants in the FTF group but not significant in the virtual group. Since it is difficult to recognize some expressions through a TV conference system, expressions might not be important for learners. Based on these results, we believe that it is necessary to support the audience’s recognition of facial expressions and eyes in a remote environment (Table 6).

Table 5. Comparison of determinants by learning environment based on non-standardized partial regression coefficients
Full size table
Table 6. Inter-group comparison of difficulty of discussion activities using multiple comparison (Tukey honestly significant difference)
Full size table

4.2 Impact of Group Composition on Discussion Activities

To clarify research question (2), the relationship between the group composition and the evaluation of VGD was analyzed. Seven items (shown in Table 3) were designed to ask about the difficulty of learning activities. As a result of running the seminars, four types of group could be compared. Group A was composed of only seminar A members, group B was composed of only seminar B members, group A2B2 was a mixed group consisting of two members from seminar A and two from seminar B, and group A2B1 is a mixed group consisting of two members from seminar A and one from seminar B.

The results of the multiple comparison test (Tukey honest significant difference) found that differences between group A2B1 and group A and between group A2B1 and group B were significant. Thus, differences in the number of participants between seminars in one group led to difficulties in group discussion in a virtual seminar. To correctly assess discussion abilities or ideas through a discussion, group composition should be unified for all groups in VGD.

5 Conclusion

Data collected from running a virtual seminar were analyzed for the purpose of developing an SMS to support the construction and operation of virtual seminars. We found that it is possible to analyze factors that contribute to improving a virtual seminar. To improve the virtual seminar, we found that it is important to consider the following:

  1. (1)

    Support functionality for non-verbal communication is needed

  2. (2)

    In VGD, consideration is necessary to ensure there is no bias in the number ratio among different seminars.

In the future, we intend to carry out functional improvements in the SMS to increase the variety of data acquired in the SMS for a deeper understanding of virtual seminars. For example, it is important to develop functions for seminar rooms suitable for virtual seminars and to configure the automatic grouping function of VGD. In addition, it will be necessary to enhance contents of seminar activities and enhance assessment items by seminar activities in future.