Elsevier

Expert Systems with Applications

Volume 38, Issue 10, 15 September 2011, Pages 12168-12179
Expert Systems with Applications

Creating the aspired intelligent assessment systems for teaching materials

https://doi.org/10.1016/j.eswa.2011.03.050Get rights and content

Abstract

The core objective of the Nine-Year Integrated Curriculum for primary schools is to “enable students to demonstrate their talents instead of just scoring high on exams” around the world. The determining factor in education reform is to declare the competence indicators of necessary educational behavior in primary and junior high school. In the reform process, for all domains, the enriched rate of competence indicators for educational materials and methods is very meaningful. Because educational materials and methods in different domains have their own style, we should evaluate these teaching materials separately. Thus, in this research we propose a novel MCDM (Multiple Criteria Decision Making) framework for evaluating, comparing, and improving the effectiveness of competence indicators in the various publications for teaching materials in primary school based on different viewpoints. The ANP (Analysis Network Process) weights are based on the DEMATEL technique with the MCDM method for resolving the problems of dependence and feedback among criteria. Then, a VIKOR technique with ANP weights is proposed for addressing and reducing the performance gaps for each criterion, thus hopefully improving, re-configuring and selecting the aspired Intelligent Assessment Systems (IAS) for teaching materials. An empirical study of Mandarin Chinese based on this system design of three publishers is illustrated to verify the effectiveness of the proposed method. The results can improve the efficiency and quality of the authored Mandarin Chinese teaching materials and may extend to other Learning Areas.

Highlights

► In this paper the authors propose a new hybrid MCDM model based on the DEMATEL technique for resolving the problems of interdependence and feedback problems among dimensions/criteria in real world; then, a VIKOR method with ANP (analytic network process) weights is proposed for addressing and reducing the performance gaps for each dimension/criteria based on the influential relation map, thus hopefully improving, reconfiguring/selecting and creating the aspired intelligent assessment systems for teaching materials to promote education levels in education reform.

Introduction

Every country has made the cultivation of human talent a priority in the 21st century. As other advanced countries propose education reform in different forms throughout the world, Taiwan also recognizes education as the bedrock of national development, implementing various education reforms such as pre-school education reform, grade 1–9 curriculum reform, the restructuring of secondary education, the enhancement of higher education, and lifelong learning projects (MOE, 2008). Therefore, the purpose of this research is to propose a novel technique and evaluation method that can improve, re-configure and select the aspired Intelligent Assessment Systems (IAS) for teaching materials to promote education levels.

Since 1990, the four British educational reform contexts have been: (a) the right to national centralization reform; (b) the priority of compulsory education; (c) vocational education mainstreaming; and (d) the ability to pursue educational evaluation (DfE, 1991). The report of the Mayer Committee proposed to advise the Australian Education Council (AEC) and the Ministers for Vocational Education, Employment and Training (MOVEET) on employment-related key competencies for post-compulsory education and training (Mayer, 1992). The Education Commission of Hong Kong proposed the following: “Learning is the key to one’s future, and education is the gateway to our society’s tomorrow”. Education enables individuals to develop their potential, construct knowledge and enhance the quality of their personal lives (EC, 2000).

In general, no matter what style of education reform is used, the key competencies are the major concern for national education reform at the beginning of the 21st century.

The core objective of Nine-Year Integrated Curriculum for primary schools in Taiwan is to “enable students to demonstrate their talents instead of just scoring high on exams”. The determining factor in education reform is to declare the competence indicators for necessary educational behavior in primary school (MOE, 2002). In the reform process, for all domains, the enriched rate of competence indicators for educational materials and methods is very meaningful. Because educational materials and methods in different domains have their own style, we should evaluate these teaching materials separately.

Thus, in this research we propose a novel MCDM (Multiple Criteria Decision Making) framework based on the DEMATEL (Decision Making Trial and Evaluation Laboratory) technique for evaluating, comparing, and improving the effectiveness of competence indicators in the various publications for teaching materials in primary school based on different viewpoints. The ANP (Analysis Network Process) weights are based on the DEMATEL technique with the novel MCDM method for resolving the problems of dependence and feedback among criteria. Then, a VIKOR technique with ANP weights is proposed for addressing and reducing the performance gaps for each criterion, thus hopefully improving, re-configuring and selecting the aspired Intelligent Assessment Systems for teaching materials. An empirical study of Mandarin Chinese teaching materials in Grade 1 of primary school based on this system design of three publishers is illustrated to verify the effectiveness of the proposed method. The results can improve the efficiency and quality of the authored Mandarin Chinese teaching materials and may extend to other Learning Areas.

The remainder of this paper is organized as follows. In Section 2, the aspired intelligent assessment systems for teaching materials with MCDM are introduced. In Section 3, a novel MCDM method based on the DEMATEL technique is proposed. In Section 4, an empirical study for the aspired IAS for Mandarin Chinese teaching materials is presented to show the process that our proposed method entails, and discussions are conducted. Finally, in Section 5, concluding remarks are presented.

Section snippets

Intelligent assessment systems for teaching materials with MCDM method

In recent decades, competence-based education has become the mega trend impacting the education reform strategies of the majority number of governments throughout the world. In the following section, the literatures related to core competence (CC), the intertwined effects of an assessment system for teaching materials, will be reviewed as a foundation for the development of the theoretical framework of this paper. We also give an example of Taiwan to explain the basic concepts of Educational

A novel MCDM method based on the DEMATEL technique with ANP

The structure of the MCDM problem will be derived using the DEMATEL method. The priorities of every determinant are based on the structure derived by using the ANP. The VIKOR technique will be leveraged to calculate the compromise ranking of the alternatives. Finally, the assessment system for Mandarin Chinese teaching materials will be derived. In summary, this evaluation framework consists of five main phases: (1) establishing determinants using the questionnaire survey method; (2) building

An empirical study of the aspired assessment systems for mandarin chinese teaching materials

In this section, an example being modified from a real case will be presented to demonstrate the effectiveness of the proposed novel MCDM framework using the DEMATEL technique. One empirical study example will be based on an example modified from three leading publishers.

In this case study, Mandarin Chinese teaching materials (six books) for Grade 1 in primary school in Taiwan are selected; these materials were edited by those publishers.

Concluding remarks

Based on effective concern, begin from Mandarin Chinese. This paper would have mainly advanced work in the field of the assessment system for teaching materials.

First, a novel MCDM framework was proposed to define the determinants of teaching materials (not exclusively for Mandarin Chinese).

Second, the traditional problem of the difficulty of defining the assessment system for teaching materials was resolved based on the novel MCDM approach being proposed. An important reasoning conclusion was

Acknowledgment

Over 20 teachers provided feedback on the questionnaire. Although I do not know who are you, I sincerely appreciate your kindness.

References (39)

  • S. Opricovic et al.

    Extended VIKOR method in comparison with outranking methods

    European Journal of Operational Research

    (2007)
  • G.H. Tzeng et al.

    Evaluating intertwined effects in e-learning programs: A novel hybrid MCDM model based on factor analysis and DEMATEL

    Expert Systems with Applications

    (2007)
  • G.H. Tzeng et al.

    Multi-criteria analysis of alternative-fuel buses for public transportation

    Energy Policy

    (2005)
  • G.H. Tzeng et al.

    Multicriteria selection for a restaurant location in Taipei

    International Journal of Hospitality Management

    (2002)
  • W.W. Wu et al.

    Developing global managers’ competencies using the fuzzy DEMATEL method

    Expert Systems with Applications

    (2007)
  • Chen, C.H., & Tzeng, G.H. (2009a). Combined DEMATEL technique with a novel MCDM method for creating the aspired...
  • Chen, C. H., & Tzeng, G.H. (2009b). Creating the aspired intelligent fuzzy assessment systems for mandarin Chinese...
  • S.J. Chen et al.

    Fuzzy multiple attribute decision making: Methods and applications

    (1992)
  • Y.J. Chiu et al.

    Marketing strategy based on customer behavior for the LCD-TV

    International Journal of Management and Decision Making

    (2006)
  • Cited by (48)

    • Analytic network process: An overview of applications

      2020, Applied Mathematics and Computation
    • Analytic network process: Academic insights and perspectives analysis

      2019, Journal of Cleaner Production
      Citation Excerpt :

      From 2011 to 2015, the number of articles authored by Tzeng, G. H. remained the highest. The articles authored by Tzeng, G. H. cover a wide range of topics, such as education (Chen and Tzeng, 2011; Hu et al., 2018), investment strategies (Ho et al., 2011), and marketing strategies (Wang and Tzeng, 2012), and the topics of the articles of the last three years have focused on mobile commerce (Hu et al., 2015; Lu et al., 2015), the digital economy (Hsu et al., 2018), and sustainable development (Liu et al., 2018a). Among 1485 articles about the ANP published during 1997–2018, 48 (3.23%) articles were cited more than 100 times, 107 (7.21%) articles were cited between 50 and 99 times, and 189 (12.73%) of articles were cited between 25 and 49 times.

    • A state of the art literature review of VIKOR and its fuzzy extensions on applications

      2016, Applied Soft Computing Journal
      Citation Excerpt :

      Fu et al. [285] combined fuzzy AHP and VIKOR for training institutions to allocate their resources appropriately by focusing on the critical success factors. Chen and Tzeng [286] proposed a novel MCDM consisting of DEMATEL, ANP and VIKOR for evaluating, comparing, and improving the effectiveness of competence indicators in the various publications for teaching materials in primary school based on different viewpoints. Ranjan et al. [287] developed a MCDM framework by integrating DEMATEL, entropy and VIKOR for performance evaluation and ranking of 16 engineering departments in an Indian university.

    View all citing articles on Scopus
    View full text