Development of a Seminar Management System

Abstract

The role of universities in imparting of knowledge is declining as e-learning and massive open online courses become widespread, and eventually only seminar activity will remain on university campuses. It is important to make seminars the central component of university education, and to think of them as a “university within a university”. Within this educational philosophy, each seminar should compete as an attractive educational experience that students want to participate in, and should aim to achieve high quality. In this paper, we report the development of an integrated seminar management system to support seminar activities.

1 Seminar Activity for a “University Within a University”

In most advanced nations, the rate of matriculation to higher education now far exceeds 50 %, and according to the American education scholar Martin Trow, higher education is nearly universally available in these countries. Different methods of providing universal access through distance education, such as the OpenCourseWare program and massive open online courses (MOOCs), have rapidly gained prominence since the beginning of this century, and universities around the world have been pressed to change with the times. Although it seems that most lecture-type classes are likely to be offered through distance education to off-campus locations in the near future, discussion- and participatory-type lessons are still performed mainly at university campuses and require in-person attendance at the school. One type of educational model is centered on seminar activities. In particular, each seminar activity offered by a university instructor should fill a role in a larger framework, which defines the curriculum, and these seminar activities should support the particular educational philosophy established for a “university within a university.” In this paper, we report the development of an integrated seminar management system (SMS) to support seminar activities. We propose a function for supporting undergraduate research as one aspect of SMS.

The proposed system is based around an electronic portfolio (e-portfolio). A key concept is that students can discover their strengths and weaknesses through reflection on evidence provided by their studies. In line with this, it is expected that introducing an e-portfolio into seminar activities will result in improved student motivation and attitude toward the seminar activities. The twin purposes of this research are to develop an e-portfolio system that is appropriate for studying evidence gained through seminar activities and to verify that the e-portfolio supports seminar activities. In this paper, we focus on the specific research activity of “creating the research topic” (CRT). We examine the utility of using an e-portfolio for CRT support by analyzing study evidence and the results of a questionnaire.

2 Investigation of Student Attitudes Toward Seminar Activities

To clarify the guidelines for developing the SMS, student attitudes toward seminar activities are investigated. Seminar activities have rarely been compared across different seminars. The policy of active seminar support is clear from collecting, comparing, and considering the viewpoints of students who use seminars.

2.1 Methodology

A questionnaire survey and interview were performed with 30 fourth-year students [1] who attend a seminar in the School of Human Sciences, Waseda University. Questions belonged to categories such as “reason for seminar choice”, “comparison of seminars and lectures”, “usual seminar activity”, “extracurricular activities besides seminar activity”, and “ideal seminar”.

2.2 Results and Discussion

In the questionnaire, 13 of the 30 students mentioned that an “ideal seminar” is a seminar in which a “good relationship between students and teachers” is forged. Students do not regard seminars as a place for advanced research; rather, they think of it as a community for building interpersonal relationships. Here, the definitions of “good” and “ideal” depend on personal values, and they should be similar for the student and the teacher involved in the seminar.

Ten of the 30 students mentioned that seminars managed by students on their own initiative are ideal. This suggests that it is desirable to change the role of teacher to that of a mentor, and that supporting students so that they can act on their own initiative would improve their motivation in seminars.

3 Development of SMS

3.1 Objective of SMS

According to the results in Sect. 2, the objective of SMS is to promote the development of students and teachers as a community. Introducing mentoring should also help support the learning community [2]. Teachers can also learn as they teach [2], contributing to the growth of a community. So we place the teacher in the seminar activity as the primary mentor and supporting mentoring as a main policy for SMS.

Based on the results in Sect. 2, it is also important for the development of individuals and the community that students feel that seminars are relevant. For effective course advice [3], it is useful to accumulate student information, so that new students can compare themselves with other students, and discover their aptitude for courses. Thus, we think it is also important to use e-portfolios in a seminar, and to make it possible to offer guidance when students choose suitable seminars. Therefore, e-portfolios are fundamental to SMS. Initially, we construct the overall design of the e-portfolio for accumulating the learning evidence from the seminar activity.

3.2 Seminar Activities

In designing an integrated e-portfolio for seminar activities, the seminar activities in the authors’ laboratory were used as a prototype model. The content of the seminar activities was organized, and each activity was classified into one of three activity categories (Table 1).

Table 1. Overview of design of an integrated e-portfolio for seminar activities

“Research activities” constitute the core seminar activities. “Interaction inside the laboratory” aims at removing obstacles in the path of seminar students, enhancing knowledge through discussion, and improving motivation to learn through things such as job search assistance and class reunions that include alumni. “Interaction between laboratories” aims at promoting intellectual exchange with new people and other activities that cannot be done through ordinary seminar interactions.

3.3 Design of an e-Portfolio

To promote reflection by students through use of an e-portfolio, it is important to accumulate the content for the e-portfolio in a way that addresses each educational objective [4] and to use different kinds of portfolio structures [5]. In this paper, the design of the e-portfolio for seminar activities is based on the framework proposed by Ueno and Uto [5]. A development portfolio is a portfolio that provides structure for portfolios having various purposes, and using a variety of portfolios provides more support to students than a single portfolio alone can provide. Liqin and Chunhui [4] classified portfolios into four types, with lower types forming a subset of higher types. The relation among the four kinds of portfolios is as follows:

career portfolio ∋ course portfolio ∋ topic portfolio ∋ content portfolio.

Here, “career portfolio ∋ course portfolio” indicates that a course portfolio is one part of a career portfolio, and so on. A career portfolio is built over a long period of time and includes activities from across one’s entire time at university and life after graduation. Below this, a course portfolio is created for a subject course, a topic portfolio is created for each topic within the course, and a content portfolio is created for each unit within a topic and contains the student’s output on that unit. An overview of an integrated portfolio for seminar activities is shown in Table 1. The hierarchical structure of the career portfolio and course portfolio can be seen here. Based on the e-portfolio in Table 1, a design is required to accommodate the extra in-depth data and the support function for which data was used.

4 Support Function for Creating Research Topic

We describe the CRT support function, developed as a function of SMS, based on the portfolio design in Table 1 in this chapter. It is important for students to read articles to suggest research topics. Understanding and explaining research articles allows students to identify gaps in previous research and generate new ideas. However, it is time consuming and difficult for seminar teachers to check how students read articles, and provide individual guidance on literature research. In this chapter, we develop a support system to help students learn how to read articles as a part of seminar activity.

4.1 Learning Method

It is important for students to summarize each article and explain the relationship of the article to the literature as a whole as learning method. We define the learning method as follows.

• (a) Students make a summary of each article.

• (b) Students make a “mixed summary”, which explains the relationship between the articles.

4.2 Support Method

This method offers students useful indexes to help them evaluate and improve their summaries. We define suitable indexes calculated according from the sentences. A good mixed summary should have at least two of the following properties in the sentences of the summary:

• 1. The main points of each article are reflected.

• 2. The mixed summary discusses the relationship between the articles, rather than simply summarizing them one after the other.

We define two indexes, “similarity” and “mixing degree” to help students create mixed summaries that satisfy criteria (1) and (2).

Similarity

We assume that the words in an article summary and a mixed summary are related. Then, we define similarity Sim _i of article i to mixed summary U by the following Eq. (1).

$$ Sim_{i} = \frac{{|W_{i} \mathop \cap \nolimits W_{U} |}}{{|W_{i} |}} $$

(1)

• Here,

• $W_{i}$: Set of nouns in article i

• $W_{U}$: Set of nouns in summary U

This index will provide feedback to students on how they can improve their summaries, for example, “There are no words in the mixed summary from each article” and “Are you using the correct phraseology?” This can motivate students to modify the mixed summary and their individual article summaries.

Mixing degree

The similarity cannot show whether a student is explaining the relationship between articles. So we introduce the “mixing degree” to measure how well the article information is combined and explained. The indexes, M _inter and M _intra, are introduced. M _inter is the mixing degree of the whole mixed summary and M _intra is the mixing degree of each sentence.

To calculate M _inter, the order in which nouns appear in the sentences in the mixed summary (1, 2, …, n) is obtained. M _inter is defined in Eq. (2).

$$ M_{inter} = \frac{{\mathop \sum \nolimits_{i = 1}^{n - 1} c_{{i\{ i + 1\} }} }}{n - 1} $$

(2)

• Here,

• n: The number of the noun in mixed summary U

• $c_{ij} = \left\{ {\begin{array}{*{20}c} {0\left( {\begin{array}{*{20}c} {the \,i\,th\,noun \,and\,the\,j\,th\,noun\,included\,in\,U } \\ { are\,included\,in\,only\,the\,same\,articles} \\ \end{array} } \right)} \\ {1\left( {Other} \right)} \\ \end{array} } \right.$

If the content of articles are not mixed and are only combined, M _inter will be small. To increase M _inter, students must relate words from different articles to create a mixed summary.

M _intra is defined by Eq. (3).

$$ M_{intra} = \frac{{\mathop \sum \nolimits_{i = 0}^{|S|} D_{i} }}{|S|} $$

(3)

• Here,

• $D_{i} = \left\{ {\begin{array}{*{20}c} {0\left( {\begin{array}{*{20}c} {All\,nouns\,included\,in\,sentence\,i\,of\,mixed\,summary\,U} \\ { are\,included\,in\,only\,the\,same\,articles} \\ \end{array} } \right)} \\ {1\left( {Other} \right)} \\ \end{array} } \right.$

• S: The set of the sentence in mixed summary U

To calculate M _intra, we examine each sentence included in the mixed summary. When the words used in a sentence are taken from several articles, M _intra is large. To increase M _intra, students must relate words from different articles, which may encourage students to identify a new links between articles. Feedback from these three indexes may help students to understand each article better and find more links between articles.

4.3 System Development

We developed a learning support system that can calculate three indexes for the summaries made by students and feed back the numerical value of the indexes to students. Figure 1 shows the user interface of the learning support system.

Fig. 1.

User interface of the learning support system

A student reads articles and inputs the summary of each article into the entry field on the left side of the screen. Next, the student refers to the content of each summary and relates the content of two or more individual articles in the mixed summary entry field on the right side of system screen. The student presses a diagnosis button, the system compares each summary with the mixed summary and calculates the similarity and mixing degree. The calculated indexes are shown in a fixed location on the screen. The student corrects the article summaries and the mixed summary with reference to these indexes. The student can amend inaccurate expressions and relationships in the mixed summary and in each article summary. This will help students notice important expressions and key words that they may have missed.

4.4 Practice

4.4.1 Method

A study was conducted to evaluate the proposed indexes and the system. The participants were eight third year students in our department. The study was conducted from January 8 to January 22, 2015. The students were assigned to a seminar when they entered the third year. They continued to study in the same seminar in the fourth year and proceeded to graduate study. The third year students read articles, made presentations, and participated in discussions to prepare for fourth year. The participants were students who read articles repeatedly from April 2014.

Two tasks were given to the students: reading two articles or reading three articles, and then writing a mixed summary. For the two-article task, students chose two articles from those that they had already read during their usual seminar activity, and made individual article summaries and a mixed summary. For the three-article task, they chose one new article in addition to two articles they had already read, and made individual article summaries and a mixed summary. The results were shared with the students on January 15, 2015 (two-article task) and January 22, 2015 (three-article task).

A questionnaire survey was conducted after the tasks. We expected that the indexes and the system would have the following effects.

• The content of articles would be understood better.

• Participants would be more aware of correct phraseology.

• Participants would be more aware of the relationship among different articles.

• The indexes and systems would be useful for selecting what participants should read next.

Based on these expected effects, the (1) similarity, (2) mixing degree, and (3) learning using the system were evaluated by the students. Questions were prepared about each area, and the participants give answers on a five-point Likert scale (“1. I don’t think so” to “5. I think so”) and through free description.

4.4.2 Results and Discussion

Table 2 shows the number of times each participant received feedback. Participants A, B, and D received more than twice the amount of feedback for each task compared with the other participants. This suggests that the participants modify their summaries after feedback and receive repeated diagnoses. Participants F, G, and H only completed the two-article task.

Table 2. Number of diagnoses

Tables 3 and 4 show the changes in the indexes when participants A and B modified their summaries. The results for the two- and three-article tasks are placed in one table because two cases are ongoing.

Table 3. Feedback for participant A

Table 4. Feedback for participant B

Initially, we found that the mixing degree in both participants was always 100 %. This shows that the students use words from different articles in the same sentence. Therefore, that M _intra is useful for helping students become more deeply aware of the relationships among articles. The results for participant A in Table 3 show no big changes in similarity, but that there is a clear change in M _inter in the 3^rd and 4^th diagnosis. Every time a check is repeated, the similarity tends to increase, and we can also confirm the M _inter for participant B in Table 4 gradually increases.

The improvement in the mixing degree in a sentence was used to confirm the results for participant A. We compared the 3^rd and 4^th diagnoses for participant A. Table 5 shows a comparison of the sentences in a mixed summary by participant A. One of the sentences in the 4^th diagnosis is not seen in the 3^rd diagnosis, “because students esteem activity in time”, which participant A indicates as a reason. This shows that the mixing degree provides useful information for students to consider where they can add an expression or keyword in sentences (Table 5).

Table 5. Mixed summary by participant A

Table 6. Summary of article 1 by participant B

Next, we check the change in the similarity. Table 6 shows a comparison of sentences in the summary of article 1 by participant B. In the 6^th diagnosis, the type of MOOC participant in the study is discussed. However, in the 7^th diagnosis this description is removed, and only a specific instance is mentioned by participant B as “Even if learning goals are achieved, some person who doesn’t accomplish the learning activity is included in the number of finish”. Thus, the type of MOOC participant is not essential information in this summary. Therefore, participant B tried to eliminate expressions that are not important. The similarity is useful for students to identify which expression or words are important.

Table 7. Questionnaire results

Table 7 shows the results of questionnaire survey. They show that many students have a positive impression of each indexes and the system. Participant E gave a negative evaluation for Q1-2 and Q3-4. “Q 1-2: Correct phraseology” was poorly rated because Participant E did not modify summaries after feedback. This result suggests that it is important to include a mechanism to motivate students to improve the summary repeatedly, for example, through peer assessment.

Participant E gave the reason for negative evaluation of “Q 3-4: When the system became available from the first term in the third year, did you want to try it?” as “It is difficult. There was not enough experience in reading articles at the start of third year.” Participant C gave the opinion “Experience in reading articles is important for appreciating the usefulness of the system.” Therefore, it is important to consider the timing of the introduction of the system to students. However, positive opinions were obtained, such as “I wanted to gather many articles to read”, “The system should be used in the long run, and I would like to check the outcome”, “It seems useful for deciding which areas of research to investigate”, “I think more directionality on its graduation theme was decided early.” This shows that the system should be introduced as early as possible after students start to read articles.

5 Conclusion

In this work, we developed an SMS to support seminar activity. Support through CRT was proposed as a function of SMS. The study results showed that the system helped students learn effectively.

In future work, we intend to design a system for accumulating detailed data and analyzing support functions based on the e-Portfolio design in this research. We also intend to improve the CRT function. For example, introducing a peer assessment process may be a good way to motivate students.