Enhancement of student learning performance using personalized diagnosis and remedial learning system

Abstract

Although conventional student assessments are extremely convenient for calculating student scores, they do not conceptualize how students organize their knowledge. Therefore, teachers and students rarely understand how to improve their future learning progress. The limitations of conventional testing methods indicate the importance of accurately assessing and representing student knowledge structures. The personalized diagnosis and remedial learning system (PDRLS) proposed in this study enhances the effectiveness of the Pathfinder network by providing remedial learning paths for individual learners based on their knowledge structure. The sample was 145 students enrolled in introductory JAVA programming language courses at a Central Taiwan technology university. The experimental results demonstrate that learners who received personalized remedial learning guidance via PDRLS achieved improved learning performance, self-efficacy, and PDRLS use intention. The experimental results also indicated that students with lower knowledge level gain more benefits from the PDRLS than those with higher level of knowledge and that field dependence (FD) students obtain a greater benefit from PDRLS than field independence (FI) students do.

Introduction

To learn effectively, students must organize and link their prior knowledge (the knowledge they have already learned or already know) with new knowledge. Students who are unable to link new knowledge with prior knowledge have problems understanding, recalling, and accessing the new knowledge later (Anderson, 1995). As Ausubel (1968) noted, “the most important single factor influencing learning is what the learner already knows.” Thus, identifying and understanding student knowledge status serves as an important work for teacher to design instructional strategies to help students learn efficiently (Carpenter et al., 1998, Cobb et al., 1991). Conventional assessments (e.g., evaluation and recall) are convenient means of calculating student scores and grades but provide little information regarding student knowledge status (or knowledge level) (Lau & Yuen, 2010). Therefore, alternative assessment techniques are needed to remedy the limitations of conventional assessment (Reeves, 2000).

An alternative technique of assessing student knowledge level is concept mapping, which reveals and visualizes student knowledge structures. However, concept maps are often inadequate for assessing and comparing knowledge structure for larger groups of students, and they lack scales for measuring student knowledge quality. One solution for investigating large numbers of students and indicating individual knowledge quality is the Pathfinder network. Pathfinder network applies a scaling algorithm based on graph theory to represent knowledge in a network format, which enables assessment of numerous students as well as evaluation of knowledge quality by comparing similarities in referent structure (Cooke, 1992, Cooke and Schvaneveldt, 1988). This technique obtains three measurements of individual knowledge quality: PRX index, GTD index, and PFC index (C index). Previous studies confirm that these indexes are effective for evaluating student knowledge quality in various learning domains (Chen, 1997, Davis et al., 2003, Goldsmith et al., 1991, Gomez et al., 1996). However, learners with similar knowledge scores may have different understandings or misconceptions and may present different knowledge structures. Pathfinder indices have limited capability to locate specific misconceptions for individual learners (Davis et al., 2003).

Technological advances in information and network technology now enable efficient solutions for this problem. Many researchers have proposed personalized e-learning systems for enhancing individual learning efficiency (Chen et al., 2005, Chen et al., 2006, Chen, 2010, Lau and Yuen, 2010, Papanikolaou and Grigoriadou, 2002, Tang and Mccalla, 2003). Recent studies have investigated the use of adaptive mechanisms in personalized e-learning systems that consider learner preferences, interests and browsing behaviors. These studies, however, overlook the importance of learner ability when providing personalized service (Chen et al., 2005, Chen et al., 2006, Papanikolaou and Grigoriadou, 2002). Therefore, numerous researchers have focused on developing personalized services based on learner ability (Chen et al., 2005, Chen et al., 2006, Lau and Yuen, 2010, Papanikolaou and Grigoriadou, 2002). Some have focused on diagnosing groups (e.g., different gender) rather than individuals (Lau & Yuen, 2010). Others have proposed various scoring mechanisms for representing knowledge level individually (Chen et al., 2005, Chen et al., 2006, Papanikolaou and Grigoriadou, 2002). However, none of these studies have described specific misconceptions and provided remedial service of individual learners.

To add value to personalized services in asynchronous or synchronous online learning and enhance student online learning performance, this study developed a method of personalizing remedial learning based on the knowledge structure of the individual learner. A Web-based intelligent personalized diagnosis and remedial learning system (PDRLS) was used to assess and visualize student knowledge structures and diagnose student misconceptions. Knowledge structure was measured using Pathfinder network (Schvaneveldt, 1990), which enables assessment of large numbers of students. The application of Pathfinder network was then extended to individual knowledge diagnoses and remedial learning paths.

An important step in the individual learning process is the organization of associations and relationships among previously learned knowledge or concepts that are stored in long-term memory into appropriate sequences, or “epistemological orders” (Polya, 1957). The sequence of the epistemological order can be obtained and presented topologically as a “conceptual map” or as a “knowledge structure” (Novak et al., 1983, Plotnick, 1997). The structure of concepts in a knowledge domain is configurable, which enables measurement of knowledge structure and the identification of misconceptions held by novices.

Typically, the first step in revealing knowledge structure is eliciting knowledge, which obtains individual judgments or answers about concept relationships. The judgments or answers are then scaled and represented. Finally, the derived knowledge is compared against a referent structure or “gold standard”, which is often elicited from a domain expert. Many methods have been proposed to reveal knowledge structure. The many proposed methods of revealing knowledge structure include concept mapping (Leauby & Brazina, 1998), word association techniques (Geeslin & Shavelson, 1975), ordered recall (Cooke, Durso, & Schvaneveldt, 1986), card sorting procedures (Frederick, Heiman-Hoffman, & Libby, 1994), paired-comparison (Curtis & Viator, 2000), and the ordered tree technique (Naveh-Benjamin, McKeachie, Lin, & Tucker, 1986). Cluster analysis or multi-dimensional scaling can be used for scaling and representing knowledge structure while holistic scoring, density scoring, and validity scoring (McClure, Sonek, & Suen, 1999) are typical techniques for scoring knowledge structure. Of these methods, concept maps offer an appropriate way of visualizing learner knowledge structure and lead to a better understanding on learner’s knowledge status (Lau and Yuen, 2010, Leauby and Brazina, 1998). However, concept maps are ineffective for studying large groups, particularly in classes larger than twenty students in which the teacher must spend considerable time providing personalized suggestions to individual students.

Pathfinder network technique was proposed by Schvaneveldt (1990) to investigate individual knowledge quality in large groups of students. The pathfinder network reveals local relations among psychologically meaningful concepts compared with other multi-dimensional scaling representations (Cooke, 1992, Cooke and Schvaneveldt, 1988). Thus, this study employed this technique to assess learner knowledge structures.

Pathfinder network applies a scaling algorithm based on graph theory to represent knowledge as a network derived from proximity matrices. The proximity matrices represent the interconnectedness (distance) between concepts and the strengths of the relationships among concepts. Concepts and relations among concepts are represented as nodes and lines. The pathfinder algorithm uses the triangle inequality rule to search the nodes for a minimum length path among concepts and then constructs a proximity matrix. Finally, the matrix is transformed into a network structure. To ensure that the network structure is a better representation of domain knowledge, network is determined by two main parameters r and q. The r parameter determines how the weight of a path is calculated from the weights of path links while the q parameter limits the number of links permitted in the paths. This limit can be incorporated into the network generation procedure to limit the number of links in the paths for which triangle inequality is ensured in the final proximity matrix. Pathfinder analysis provides three measurements for assessing individual knowledge quality by comparing similarities among referent structures provided by experts: PRX, GTD, and PFC.

Pathfinder networks are widely used to represent knowledge structures in diverse domains, including computer programming (Cooke and Schvaneveldt, 1988, Lau and Yuen, 2010), team performance (Lim & Klein, 2006), software requirements (Kudikyala & Vaughn, 2005), flight training (Schvaneveldt, Beringer, & Lamonica, 2001), accounting education (Curtis and Davis, 2003, Rose et al., 2007) and mental health (Ober & Shenaut, 1999). This technique also reveals conceptual changes before and after evolutionary changes in instruction in varying applications (D’Apollonia, Charles, & Boyd, 2004), including statistics (Geske, 2001) and police training (Braverman, 1997). Previous studies indicate that Pathfinder scores can effectively predict self-efficacy (Davis et al., 2003), achievement (Chen, 1997, Goldsmith et al., 1991), and behavior (Gomez et al., 1996). Although Pathfinder scoring mechanisms are adequate for evaluating student knowledge quality, they do not explain student misconceptions by showing inappropriate links in student knowledge structures (Davis et al., 2003). Therefore, the proposed intelligent personalized diagnosis and remedial learning system (PDRLS) extends Pathfinder network to student learning improvement in order to assess student knowledge structure and diagnose student misconceptions, and finally, to provide individual learners with remedial learning paths.