ELECTRE-III-H: An outranking-based decision aiding method for hierarchically structured criteria

https://doi.org/10.1016/j.eswa.2015.02.016Get rights and content

Highlights

  • A new ELECTRE-III method is proposed handling hierarchical structure of criteria.

  • ELECTRE-III-H generates a partial pre-order of alternatives in each node of the hierarchy.

  • Concordance and discordance indices are adapted to aggregation of partial pre-orders.

  • The pseudo-criteria concept is extended on intermediate criteria of the hierarchy.

  • A real-case study about the evaluation of the quality of websites is presented.

Abstract

This paper proposes a method for ranking a set of alternatives evaluated using multiple and conflicting criteria that are organised in a hierarchical structure. The hierarchy permits the decision maker to identify different intermediate sub-problems of interest. In that way, the analysis of the criteria is done according to the subsets defined in the hierarchy, and following the precedence relations in a bottom-up approach. To deal with this type of hierarchical structures, an extension of the ELECTRE-III method, called ELECTRE-III-H, is presented. As all methods of ELECTRE family, this one also relies on building a binary outranking relation on the set of alternatives on the basis of concordance and discordance tests. The exploitation of this outranking relation generates a partial pre-order, establishing an indifference, preference or incomparability relation for each pair of alternatives. The idea of a bottom-up application of the classical ELECTRE-III method to sub-problems involving subsets of criteria at the intermediate levels of the hierarchy is infeasible because the evaluations of alternatives by criteria aggregating some sub-criteria have the form of partial pre-orders, and not complete pre-orders. Thus, we propose a new procedure for building outranking relations from a set of partial pre-orders, as well as a mechanism for propagating these pre-orders upwards in the hierarchy. With this method, the decision maker is able to analyse the problem in a decomposed way and gain information from the outputs obtained at intermediate levels. In addition, ELECTRE-III-H gives the decision maker the possibility to define a local preference model at each node of the hierarchy, according to his objectives and sub-problem characteristics. We show an application of this method to rank websites of tourist destination brands evaluated using a hierarchy with 4 levels.

Introduction

In ranking problems, the decision maker (DM) wants to find an order structure on a set of alternatives taking into account his/her preference of one alternative over the others. This task is not straightforward when multiple conflicting criteria must be considered.

The order structure of the alternatives depends on how well they perform on particular criteria and how important each criterion is to the DM. Thus, the performances of the alternatives on the whole set of criteria must be aggregated, taking into account the preferences of the DM expressed by the relative importance of each criterion (Rudas, Pap, & Fodor, 2013). To aggregate the performances and formulate a recommendation, many decision aiding methods have been proposed, each with its own informational requirements and mathematical properties (Roy and Slowinski, 2013, Torra and Narukawa, 2007). This work focuses on solving the multiple criteria ranking problem with outranking methods (Figueira, Greco, & Ehrgott, 2005). In particular, we study the case of criteria that are organised in a hierarchy with different levels of generality.

The utility-based and outranking approaches prevail nowadays in the Multiple Criteria Decision Aiding (MCDA), a discipline deriving from the field of Operational Research. Outranking methods have been very successful because they are easy to understand by DMs and they are based on realistic assumptions. The aim of the outranking method is to build a binary relation S, where aSb means “a is at least as good as b”. It was proposed by Roy (1996) to establish a realistic representation of four basic situations of preference: indifference, weak preference, strict preference, and incomparability. The assertion aSb is considered to be true if there are sufficient arguments to affirm that a is not worse than b, and if there is no essential reason to refuse this assertion. These concepts are formalised in the definition of concordance and discordance indices with respect to aSb.

A well-known family of outranking methods is called ELECTRE (Figueira, Greco, Roy, & Slowinski, 2013). Since the proposal of the basic ELECTRE-I method, several subsequent versions have been developed for specific decision problems, e.g., ELECTRE-Is for the selection of the best alternatives, ELECTRE-II and ELECTRE-III/IV for constructing a ranking, and ELECTRE-TRI for sorting problems. ELECTRE methods have been widely acknowledged as effective and efficient decision aiding tools, with successful applications in different domains (Abedi et al., 2012, Arondel and Girardin, 2000, Botti and Peypoch, 2013, Colson, 2000, Damaskos and Kalfakakou, 2005, Papadopoulos and Karagiannidis, 2008, Shanian et al., 2008, Xu and Ouenniche, 2012).

ELECTRE methods have strengths and weaknesses (Figueira et al., 2013). The strengths include the following:

  • ELECTRE methods are able to take into account the qualitative nature of some criteria, allowing the DM to consider the original data directly, without the need to make transformations into artificial numerical scales.

  • ELECTRE methods can deal with heterogeneous criteria scales, preserving the original scores of the alternatives on each criterion, without the need for normalisation techniques or the estimation of a value function. This heterogeneity of scales is usually an inconvenience for many decision support systems, which often require a common measurement scale for all criteria.

  • ELECTRE acknowledges the non-compensatory character in the aggregation, unlike other utility-based approaches such as AHP and MACBETH. In the outranking approach, if, on a certain criterion, an alternative is strongly opposed to the assertion aSb, this fact is enough to reject the assertion aSb. This characteristic is called “right to veto”.

The main weaknesses of ELECTRE methods are as follows:

  • When the aim is to calculate an overall score for each alternative, ELECTRE methods are not suitable and other scoring methods should be applied.

  • Classical ELECTRE methods assume that all the criteria are at the same level of generality and do not consider the possibility of working with subsets of criteria in a hierarchical structure.

This paper addresses the second weakness of the classical ELECTRE method. In some real-world decision problems, criteria are naturally defined in a hierarchical structure with different levels of generality that model the implicit taxonomical relations between the criteria. This hierarchical approach is particularly suitable for complex problems with a large number of criteria. In such cases it may become cognitively difficult for the DM to consider all of the criteria together (Mustajoki, 2012).

In many applications, the hierarchical structure has the form of a tree, where the root corresponds to the general goal of the DM, the nodes of the tree descending from the goal are sub-criteria, the nodes descending from these sub-criteria are the lower-level sub-criteria, and so on. Finally, the leaves correspond to the elementary criteria, in which the alternatives are directly evaluated. This process decomposes a complex goal into smaller problems with subsets of criteria, enabling the DM to analyse the alternatives with respect to single subsets of criteria at different levels of generality.

The main contribution of this paper is a ranking method for a hierarchical set of criteria that extends the classical ELECTRE-III method (Figueira et al., 2013). ELECTRE-III follows two basic steps: (1) construction of a binary outranking relation based on partial concordance and discordance indices obtained from the consideration of a given set of criteria; these indices determine the aggregation conditions based on social choice models (majority rule and respect to minorities), and (2) exploitation of the outranking relation via distillation to obtain a ranking of alternatives in the form of a partial pre-order.

The method proposed is called ELECTRE-III-H and is designed to generate and propagate the partial pre-orders calculated from the bottom level up to the root of the hierarchy tree. We first propose an iterative procedure that maintains the two steps of the classical ELECTRE method (construction and exploitation of the outranking relation) in all intermediate nodes of the hierarchy up to the root. First, the classical ELECTRE-III is applied at the bottom of the tree, aggregating the most specific elementary criteria to their direct parent and obtaining the first results in the form of partial pre-orders. These partial pre-orders are interpreted as inputs for the intermediate criteria from the upper level. The partial pre-orders are aggregated with the construction of a new pairwise credibility matrix. Next, the classical exploitation process (known as distillation) is applied to generate a partial pre-order at the parent node. With this approach, the DM obtains a result (i.e., a partial pre-order) at each of the intermediate levels of the tree, in addition to the overall partial pre-order at the root level.

The second contribution is the definition of new partial concordance and discordance indices that take into account threshold values on partial pre-orders induced by criteria aggregated at intermediate levels of the hierarchy. The DM can also specify the relative importance of each criterion in the context of the same subset at each level of the hierarchy.

The following subsection presents a range of application domains and concrete examples of decision problems involving a hierarchical structure of criteria, in which the DM needs to study the results at different levels of the tree.

The rest of the paper is structured as follows: Section 2 reviews the classical ELECTRE-III model; Section 3 defines the concepts and notations with regard to the hierarchy of criteria; Section 4 describes the extension of the ELECTRE-III method, called ELECTRE-III-H; Section 5 presents a real-world application in which the DM considers a hierarchical structure of criteria to construct a preference ranking of tourist destination websites; finally, Section 6 presents the conclusions of the work.

Decisions based on a hierarchical structure of criteria appear in several fields. For example, environmental resource management discipline commonly involves conflicting interests such as economic, environmental impact and social criteria (Bobylev, 2011, Nordstrom et al., 2010, Sánchez-Lozano et al., 2013, Valls et al., 2010). For example, in Nordstrom et al. (2010), a case study of a planning process for an urban forest in Sweden is addressed. The paper evaluates three alternative strategic forest plans for areas around the urban forest in Lycksele, Sweden. The interests of four social groups are considered (timber producers, environmentalists, recreationists and reindeer herders). For each group, different sub-criteria with differing preferences are taken into account (e.g., timber producers want to maximise the fertilised area, while reindeer herders wish to minimise it).

Complex decision models appear also in medicine (Ahsan and Bartlema, 2004, Mendis and Gedeon, 2012, Reddy et al., 2014). In Reddy et al. (2014), it is presented a study for producing national guidance relating to the promotion of good health and the prevention and treatment of disease, at the Centre for Public Health (CPH) at the United Kingdom’s National Institute for Health and Care Excellence (NICE). The objective is to choose the most appropriate topics for this guidance taking into account a 3-level hierarchy of criteria with 3 main sub-criteria: size of the problem, making the difference, and current variation in practice.

Another area that is growing in popularity is the construction of rankings based on Quality Assessment, such as institution rankings (Aydin et al., 2012, Buyukozkan et al., 2011, Hsu and Pan, 2009, Torres-Salinas et al., 2011). Complex sets of diverse criteria are used to build a ranking of alternatives taking into account different topics. For example, in Buyukozkan et al. (2011), a model to evaluate perceived service quality in the healthcare sector and to evaluate the performance of pioneering Turkish hospitals on criteria such as responsiveness, professionalism and empathy is presented, with a 3-level hierarchy of criteria. In Aydin et al. (2012), the European Foundation for Quality Management (EFQM) Excellence Award evaluates organisations on the basis of three main parameters: -leadership, strategy and processes- that are decomposed into a 3-level hierarchy of criteria.

In Shen, Hermans, Brijs, and Wets (2012), a road safety performance evaluation for a group of European countries is presented. Several road safety performance criteria including speed, alcohol consumption and protective systems are structured hierarchically, allowing the analysis of each country’s performance for each one of this criterion based on index scores.

Business management is based on strategic decisions that include complex criteria and therefore can be modelled in a hierarchical structure (Arbenz et al., 2012, Chang et al., 2015, Kilic et al., 2015, Muerza et al., 2014, Wang et al., 2004, Yang et al., 2009). For example, in Wang et al. (2004), a manufacturing chain decision problem is analysed. The overall goal is to achieve optimal supplier efficiency with regard to a hierarchical structure of criteria, from basic indicators to four general measures of efficiency: delivery reliability, flexibility and responsiveness, cost, and assets. In Kilic et al. (2015) the aim is to select the best enterprise resource planning system using a hierarchy of criteria with 3 main branches: business, cost and technical issues.

In the MCDA literature, very few methods consider the decomposition of decision problems using a hierarchy of criteria. The best known method for managing hierarchical structures is the Analytic Hierarchical Process (AHP) (Saaty, 1987), which belongs to the utility-based approach.

AHP permits the DM to focus on specific sub-criteria to find the weights of each criterion depending on its position on the hierarchy by means of pairwise comparison of criteria having the same parent, which yields the relative trade-off weights. The pairwise comparison of alternatives and criteria is based on the judgment ratio scale from 1 to 9, in which 1 represents “Equally preferred” and 9 represents “Extremely preferred”. Once the comparison matrix has been given by the DM, weights or priorities are derived finding the normalized eigenvector of the matrix. This requires the matrix to be consistent (or near consistent) to obtain meaningful priorities. Then, a numerical rating is obtained for each of the decision alternatives by means of an additive aggregation operator. AHP was applied to some of the case studied mentioned above (Ahsan and Bartlema, 2004, Bobylev, 2011, Reddy et al., 2014, Hsu and Pan, 2009, Muerza et al., 2014, Nordstrom et al., 2010). In Buyukozkan et al. (2011), the Analytic Network Process (ANP) method, which is a generalisation of AHP for networks instead of hierarchies, is applied. The difference between AHP and ANP, is such that ANP does not consider the alternatives as independent actions.

Despite the large literature and applications of AHP, the method has also received some critics. The consistency condition is difficult to achieve, several consistency indices have been proposed, as well as methods to obtain a transitive matrix (Bana e Costa & Vansnick, 2008). The additive nature of the aggregation has also been posed into question because it generates rank reversals (Ishizaka & Labib, 2011), but also because it is a compensative trade-off approach, which is not appropriate in some applications.

Another weakness is the imprecision and uncertainty of the linguistic scale used for the construction of the pairwise comparison matrices. To overcome this weakness, the Fuzzy-AHP method has been proposed, which applies a range of value to incorporate possible DM’s uncertainty instead of merely crisp ratio values. In Aydin et al. (2012), Fuzzy-AHP is used to achieve a performance assessment of firms for EFQM Excellence Award using fuzzy scales to make pairwise comparisons. Several Fuzzy-AHP applications are presented in Mardani, Jusoh, and Zavadskas (2015) from 1994 to 2014. For ANP, a fuzzy approach has also been introduced (Chang et al., 2015).

In other complex problems, AHP and ANP are combined with other methods to treat hierarchical structures of criteria. For example, in Sánchez-Lozano et al., 2013, Kilic et al., 2015, AHP and ANP respectively are applied only to establish the weights of the criteria in the hierarchy.

There are some other utility-based approaches where aggregation operators are used to generate ratings of alternatives at different levels of generality. An interesting case is the method called Logic Scoring of Preference (LSP), where the operators are parametrized and can range from full conjunction, partial conjunction, partial disjunction and full disjunction. In addition, mandatory and optional criteria can be defined and treated accordingly in the different levels of the hierarchy. Some applications of this method for decision aiding are Dujmović and Tré, 2011, Hatch et al., 2014, Pijuan et al., 2010. The main limitation is the complexity of the problem modeling using such high level operators on the basis of its logical properties. Moreover, all the values need to be in the same numerical scale, not allowing heterogeneity as in an outranking-based approach.

Hierarchies of objectives are also considered in DEA (Data Envelopment Analysis) (Shen et al., 2012) and PGP (Preemptive Goal Programming) (Wang et al., 2004), but these methods concern a continuous space rather than a discrete set, as we study in this paper.

In recent years, another methodology, called Multiple Criteria Hierarchy Process (MCHP), has been proposed to deal with hierarchical structures of criteria (Corrente, Greco, & Slowinski, 2012). It can be applied to any MCDA method, including utility-based and outranking methods. For the outranking methods, this process is explained in Corrente, Greco, and Slowinski (2013b). It builds binary outranking relations at each node of the hierarchy. The selected ELECTRE method is applied first on the lowest level of the hierarchy to build a binary outranking preference relation for each subset of elementary criteria. Then, at upper levels, MCHP continues to construct binary outranking relations which are propagated up to the root. The preference information used to construct the outranking relations can be provided by the DM either directly (in form of outranking model parameters, like criteria weights and comparison thresholds) or indirectly (in form of pairwise comparisons of some alternatives). In the latter case, MCHP is combined with the Robust Ordinal Regression (ROR) (Corrente et al., 2013b). The ROR takes into account all sets of outranking model parameters compatible with the preference information provided by the DM to give a solution in terms of necessary and possible outranking relations, by applying all the compatible preference models on the considered alternatives. The authors present an illustrative example regarding the evaluation of students who are competing for a scholarship based on Mathematics and Chemistry that decompose to more specific subjects. The approach based on indirect preference information relies on having a suitable set of decision examples, which may sometimes be hard to find when there is no historical data or the user is inexperienced. This is a potential shortcoming in some applications.

Based on the advantages of the outranking approach, this paper proposes a novel procedure based on ELECTRE that allows decision analysis at different levels of the hierarchy tree. Given the limitations of an indirect approach, we use direct elicitation of preference information, in which the parameters of the method are given directly by the DM. The main difference between our proposal and the direct method described in Corrente et al. (2013b) for outranking methods is that we intend to apply the ELECTRE procedure (steps 1 and 2) at all levels of the hierarchy. With this approach we have two advantages: first, the DM obtains a result (i.e., a partial pre-order) at each of the intermediate levels of the tree and not only at the root level, thus the DM can analyse the ranking of the alternatives for each sub-problem, gaining more detailed knowledge of the complex decision problem; second, the DM can determine the level of concordance and the power of discordance at each level of the hierarchy. The model is therefore more complete and close to the DM needs because the aggregation conditions are different for each node depending on its particular meaning.

Section snippets

The ELECTRE-III method

In this section, we review the basic concepts and steps of the ELECTRE-III method:

  • A = {a,b,c,} is the finite set of alternatives,

  • n is the number of alternatives in A,

  • G = {g1,g2,,gm} is the finite set of criteria (it is assumed that m3),

  • gj(a) represents the performance of alternative a on criterion gjG. We assume, without loss of generality, that all the criteria are of the gain type, i.e., the greater the value, the better. A performance matrix M is built for A×G, where gj(a) is the performance

Hierarchical structure of criteria

In this section we define the concepts and notation with regard to a hierarchical structure of criteria. We will distinguish between three types of criteria depending on their level of generality in the taxonomy:

  • R is a set composed by a unique element that is the most general criterion. This corresponds to the root node, placed at the top of the tree. This criterion represents the main goal of the DM.

  • E is the set of the most specific criteria, called elementary criteria. They are placed at the

ELECTRE-III-H method

In this section we propose the ELECTRE-III-H method, which calculates partial pre-orders at all levels of the hierarchy of criteria. The construction procedure is analogous to that of ELECTRE-III, presented in Section 2, in the sense that it also entails the calculation of the outranking relation (i.e., concordance and discordance tests) and the exploitation of this relation by distillation. However, this procedure also takes into account that evaluations of alternatives on intermediate

Case study

In this case study we wish to construct a priority ranking of websites designed to promote tourist destination brands. Websites have become crucial tools for communicating destination brands and for selling a variety of tourism services and related products (Fernández-Cavia & Huertas-Roig, 2009). However, as some authors highlight, the tourism industry urgently needs to agree on sector-wide use of specific website evaluation techniques that are well established and can be used in the long term (

Conclusions

In complex real-world multiple criteria ranking problems, a hierarchical structure of criteria facilitates more detailed analysis of preference relations in the set of alternatives at different levels of generality.

In this paper, we have proposed an outranking method that extends the classical ELECTRE-III method for the case of a hierarchical structure of criteria, called ELECTRE-III-H. This method permits upward propagation of results, following the organisation of the criteria in the tree.

Acknowledgements

This project has been funded by the Spanish research project SHADE (TIN-2012-34369: Semantic and Hierarchical Attributes in Decision Making). The first author is supported by a FI predoctoral Grant from Generalitat de Catalunya. The research of the last two authors has been supported by the Polish National Science Centre. The authors want to thank the collaboration of the experts in the case study, from the project CODETUR (CSO 2011-22691) funded by the Spanish Government. We especially thank

References (52)

  • S. Corrente et al.

    Multiple criteria hierarchy process in robust ordinal regression

    Decision Support Systems

    (2012)
  • S. Corrente et al.

    Multiple criteria hierarchy process with ELECTRE and PROMETHEE

    Omega

    (2013)
  • J. Figueira et al.

    Determining the weights of criteria in the ELECTRE type methods with a revised Simos’ procedure

    European Journal of Operational Research

    (2002)
  • T.H. Hsu et al.

    Application of Monte Carlo AHP in ranking dental quality attributes

    Expert Systems with Applications

    (2009)
  • A. Ishizaka et al.

    Review of the main developments in the analytic hierarchy process

    Expert Systems with Applications

    (2011)
  • H.S. Kilic et al.

    Selecting the best ERP system for SMEs using a combination of ANP and PROMETHEE methods

    Expert Systems with Applications

    (2015)
  • R. Law et al.

    Progress in tourism management: A review of website evaluation in tourism research

    Tourism Management

    (2010)
  • A. Mardani et al.

    Fuzzy multiple criteria decision-making techniques and applications two decades review from 1994 to 2014

    Expert Systems with Applications

    (2015)
  • B.S.U. Mendis et al.

    Complex structured decision making model: A hierarchical frame work for complex structured data

    Information Sciences

    (2012)
  • J. Mustajoki

    Effects of imprecise weighting in hierarchical preference programming

    European Journal of Operational Research

    (2012)
  • E.M. Nordstrom et al.

    Integrating multiple criteria decision analysis in participatory forest planning: Experience from a case study in northern Sweden

    Forest Policy and Economics

    (2010)
  • A. Papadopoulos et al.

    Application of the multi-criteria analysis method ELECTRE III for the optimisation of decentralised energy systems

    Omega

    (2008)
  • B. Reddy et al.

    Prioritising public health guidance topics in the national institute for health and care excellence using the analytic hierarchy process

    Public Health

    (2014)
  • M. Rogers et al.

    Choosing realistic values of indifference, preference and veto thresholds for use with environmental criteria within ELECTRE

    European Journal of Operational Research

    (1998)
  • B. Roy et al.

    Questions guiding the choice of a multicriteria decision aiding method

    EURO Journal on Decision Processes

    (2013)
  • I.J. Rudas et al.

    Information aggregation in intelligent systems: An application oriented approach

    Knowledge-Based Systems

    (2013)
  • Cited by (59)

    • Analysis of the innovation capacity of Mexican regions with the multiple criteria hierarchy process

      2022, Socio-Economic Planning Sciences
      Citation Excerpt :

      In this method, authors assert that the hierarchy of criteria decomposes and simplifies the preference elicitation concerning pairwise comparisons of criteria with respect to relative importance. The ELECTRE-III-H method is proposed by Del Vasto-Terrientes et al. [25]. It is an extension of the ELECTRE-III method to deal with a hierarchy of criteria.

    • Inferring a hierarchical majority-rule sorting model

      2022, Computers and Operations Research
    View all citing articles on Scopus
    View full text