Monitoring students’ actions and using teachers’ expertise in implementing and evaluating the neural network-based fuzzy diagnostic model

https://doi.org/10.1016/j.eswa.2006.02.023Get rights and content

Abstract

In this paper, the implementation of a neural network-based fuzzy modeling approach to assess aspects of students’ learning style in the discovery learning environment “Vectors in Physics and Mathematics” is presented. Fuzzy logic is used to provide a linguistic description of students’ behavior and learning characteristics, as they have been elicited from teachers, and to handle the inherent uncertainty associated with teachers’ subjective assessments. Neural networks are used to add learning and generalization abilities to the fuzzy model by encoding teachers’ experience through supervised neural-network learning. The neural network-based fuzzy diagnostic model is a general diagnostic model which is implemented in an Intelligent Learning Environment by eliciting teachers’ expertise regarding students’ characteristics based on real students’ observation and on data being collected from students’ interaction. The model has been successfully implemented, trained and tested in the learning environment “Vectors in Physics and Mathematics” by using the recommendations of a group of five experienced teachers. The performance of our model in real classroom conditions has been evaluated during an experiment with an experienced Physics teacher and 49 students of secondary school attending Physics lessons.

Introduction

An Intelligent Learning Environment (ILE) is a relatively new kind of Artificial Intelligence (AI) computer-based educational system, which is able to support student-driven learning and knowledge acquisition (Brusilovski, 1994). ILEs are considered to be generalizations of traditional Intelligent Tutoring systems (ITS), which are based on objectivist epistemology and embrace instructional environments which make use of theories of constructivism and situated cognition (Akhras & Self, 2002). Student models are distinguishing features of both Intelligent Tutoring Systems (ITS) (VanLehn, 1988, Wenger, 1987) and Intelligent Learning Environments (ILE) (Akhras and Self, 2002, Brusilovski, 1994). A student model enables the system to adapt its behavior and pedagogical decisions to the individual student who uses it (Brusilovski, 1994, Wenger, 1987).

Ideally, a student model should include all aspects of students’ behavior and knowledge which have repercussions on their performance and learning (Wenger, 1987). In practice, the contents of a student model depend on the application. It normally includes learner goals and plans, capabilities, attitudes and/or knowledge or beliefs, and is used as a tool for adapting ILE behavior to the individual student (Brusilovski, 1994, Holt et al., 1991, Self, 1999, VanLehn, 1988). Inferring a student model is called diagnosis, because it is much like the medical task of inferring a hidden physiological state from observable signs (VanLehn, 1988), i.e. the ILE uncovers a hidden cognitive state (student characteristics) from observable behavior.

The term student behavior can be used to refer to a student’s observable response to a particular stimulus in a given domain which, together with the stimulus, serves as the primary input to the student modeling system (Sison & Shimura, 1998). The input can be an action or the result of that action, and can also include intermediate results (Sison & Shimura, 1998). From this input, the diagnosis unit must infer a student’s unobservable behavior (VanLehn, 1988). Clearly, the less information the unit has the harder its task is (VanLehn, 1988). This makes student modeling a difficult process, given that the evidence about a student’s behavior provided by the student’s inputs to an ILE is usually scanty (Self, 1991), and contains a good deal of uncertainty (Jameson, 1996). A variety of AI techniques have been proposed for this purpose.

Bayesian networks have been proposed in ANDES (Conati et al., 2002, VanLehn and Niu, 2001), in order to relate—in a probabilistic way—a student’s observable behavior to a particular piece of his/her knowledge and in the Prime Climb educational game (Conati, 2002), in order to relate students’ observable behavior to their emotional state. Unsupervized machine learning techniques have also been proposed (Sison, Masayuki, & Shimura, 1998), in order to discover classes of errors, which represent misconceptions and other knowledge errors, from discrepancies in students’ behavior.

Another approach to handle the inherent uncertainty in a student’s behavior and to achieve a human description of knowledge is to use fuzzy logic. Fuzzy logic techniques have been proposed in a variety of user and student modeling approaches (Jameson, 1996). If a user modeling system adopts this approach, its reasoning may be particularly easy for designers and users to understand and/or to modify (Jameson, 1996).

One of the first attempts in using fuzzy student modeling, which was revised some years later (Hawkes & Derry, 1996), has been proposed in TAPS by Hawkes, Derry, and Rundensteiner (1990). In this context, fuzzy logic has been proposed as a flexible and realistic method of easily capturing the way human tutors might evaluate a student and handle tutoring decisions which are not clear-cut. Towards this direction, several other attempts have been proposed in the literature to model student knowledge, mental states and progress as well as student cognitive abilities and personal characteristics. A comprehensive review can be found in Jameson (1996).

Neural networks have also been proposed in student modeling due to their abilities to learn from noisy or incomplete patterns of students’ behavior and generalize over similar examples (Beck and Woolf, 1998, Posey and Hawkes, 1996). This generalized knowledge can then be used to recognize unknown sequences. A problem which arises when trying to apply a neural network to modeling human behavior is knowledge representation. The black-box characteristics of neural networks cannot offer much help, since the weights learned are often difficult for humans to interpret. To alleviate this situation, a neural network approach in which each node and connection has symbolic meaning has been proposed in TAPS (Posey & Hawkes, 1996). The back-propagation algorithm has been used to modify weights which represent importance measures of attributes associated with student performance, in order to refine and expand incomplete expert knowledge. Hybrid rule-based approaches integrating symbolic rules with neurocomputing have been proposed for knowledge representation in Intelligent Tutoring Systems, in order to create improved representations (Hatzilygeroudis & Prentzas, 2004).

Along these lines, this paper presents a neuro-fuzzy synergism for student diagnosis, by monitoring students’ actions in an Intelligent Learning Environment and using teachers’ expertise in implementing, training, testing and evaluating the neural-network based fuzzy diagnostic model in assessing aspects of students’ learning style. The Intelligent Learning Environment consists of the learning environment “Vectors in Physics and Mathematics” (Grigoriadou, Mitropoulos, Samarakou, Solomonidou, & Stavridou, 1999a), and the neural network-based fuzzy diagnostic model (Stathacopoulou, Magoulas, Grigoriadou, & Samarakou, 2005), which is a general diagnostic model which can be used in any learning environment according to designers’ and teachers’ suggestions. The proposed approach allows handling uncertainty of student behavior, by expressing teachers’ qualitative knowledge in a clearly interpretable way with the use of fuzzy logic, while offering the possibility of adaptation to the learning environment and to teachers’ personal constructs in classifying and discriminating among students by employing a neural network implementation of the fuzzy diagnostic model.

The paper is organized as follows. Section 2 presents the Intelligent Learning Environment, giving details on student interaction with the learning environment and providing a brief description of fuzzy knowledge representation and neural network implementation of the neural network-based fuzzy diagnostic model. Section 3 presents how the close monitoring of students’ actions and the use of data being collected from students’ interaction, allow us to generate hypotheses and to classify students regarding their learning characteristics. Section 4 presents the aspects of student learning style diagnosed by our model, the implementation, training and testing of our model using teachers’ expertise, students’ observation and logfiles as well as the evaluation of our model in real classroom conditions. Lastly, conclusions are drawn and directions for future work are presented.

Section snippets

The Intelligent Learning Environment

The Intelligent Learning Environment consists of the learning environment “Vectors in Physics and Mathematics” (Grigoriadou et al., 1999a), and the neural network-based fuzzy diagnostic model (Stathacopoulou et al., 2005).

Monitoring students’ actions

Human tutors obtain diagnostic information from observing what students say and do, and how something is said and done, i.e. tone of voice, inflection, hesitancy, timing of students’ responses, etc. (Derry & Potts, 1998), whereas ILEs are handicapped in this regard, since the communication channel between the student and computer is very restricted (usually a keyboard and a mouse) (Wenger, 1987). Researchers are beginning to develop technology to accurately assess actions in less restricted

Using teachers’ expertise and students’ observation and logfiles in implementing the neural network-based fuzzy diagnostic model

In order to elicit teachers’ knowledge mentioned in Section 2.2 and implement the neural network-based fuzzy diagnostic model in evaluating students’ learning characteristics according to designers’ and teachers’ suggestions, a group of five experts teachers has been used: three of them were experienced in teaching physics in secondary education, one of them was an expert in didactics of physics, and the last one was an expert in the design of educational software. In collaboration with the

Conclusions and future work

This paper presents how teachers’ expertise and students’ logfiles have been used in implementing, training, testing and evaluating the neural network-based fuzzy diagnostic model, which is a general diagnostic model, in diagnosing aspects of students’ learning style in the Intelligent Learning Environment “Vectors in Physics and Mathematics”. Experimental results of implementing, training and testing using simulated students and the recommendations and expertise of a group of five experienced

References (51)

  • J.E. Beck et al.

    Using a learning agent with a student model

  • J. Biggs

    Student approaches to learning and studying

    (1987)
  • Brusilovski, P., (1994). Student model centered architecture for intelligent learning environments. In Proc. of fourth...
  • Conati, C., (2002). Probabilistic assessment of user’s emotions in educational games. Journal of Applied Artificial...
  • C. Conati et al.

    Using Bayesian networks to manage uncertainty in student modeling

    User Modeling & User-Adapted Interaction

    (2002)
  • J. Dae-Sik et al.

    Fuzzy inference system based on fuzzy associative memory

    Journal of Intelligent and Fuzzy Systems

    (1997)
  • B. Dart et al.

    Classroom learning environments and students’ approaches to learning

    Learning Environments Research

    (1999)
  • A. de Vincente et al.

    Informing the detection of the students’ motivational state: An empirical study

  • S.J. Derry et al.

    How tutors model students: A study of personal constructs in adaptive tutoring

    American Educational Research Journal

    (1998)
  • R. Driver et al.

    Children’s ideas in science

    (1985)
  • N. Entwistle

    A model of the teaching-learning process

  • Grigoriadou, M., Mitropoulos, D., Samarakou, M., Solomonidou, C., & Stavridou, E. (1999a). Methodology for the design...
  • Grigoriadou, M., Samarakou, M., Mitropoulos, D., Rigoutsos, A., Stavridou, E., & Solomonidou, C., (1999b). Vectors in...
  • Holt, P., Dubs, S., Jones, M., & Greer, J. (1991). The state of student modeling. In Proceeding of NATO advanced...
  • Jameson, A. (1996). Numerical uncertainty management in user and student modeling: An overview of systems and issues....
  • Cited by (0)

    1

    Tel.: +30 210 5385322; fax: +30 210 5385306.

    View full text