Decision Aiding
Pseudo-criteria versus linear utility function in stochastic multi-criteria acceptability analysis

https://doi.org/10.1016/S0377-2217(01)00276-4Get rights and content

Abstract

Stochastic multi-criteria acceptability analysis (SMAA) is a multi-criteria decision support method for multiple decision-makers (DMs) in discrete problems. SMAA does not require explicit or implicit preference information from the DMs. Instead, the method is based on exploring the weight space in order to describe the valuations that would make each alternative the preferred one. Partial preference information can be represented in the weight space analysis through weight distributions. In this paper we compare two variants of the SMAA method using randomly generated test problems with 2–12 criteria and 4–12 alternatives. In the original SMAA, a utility or value function models the DMs' preference structure, and the inaccuracy or uncertainty of the criteria is represented by probability distributions. In SMAA-3, ELECTRE III-type pseudo-criteria are used instead. Both methods compute for each alternative an acceptability index measuring the variety of different valuations that supports this alternative, and a central weight vector representing the typical valuations resulting in this decision. We seek answers to three questions: (1) how similar are the results provided by the decision models, (2) what kind of systematic differences exists between the models, and (3) how could one select indifference and preference thresholds of the pseudo-criteria model to match a utility model with given probability distributions?

Introduction

We consider methods for supporting real-life discrete multi-criteria decision making (MCDM) with multiple decision-makers (DMs). A large number of different methods have been proposed. The main approaches can be classified based on the type of decision model they apply:

  • 1.

    Value- or utility function-based methods such as multi-attribute utility theory (MAUT) (Keeney and Raiffa, 1976), SMART (Von Winderfelt and Edwards, 1986), and the analytic hierarchy process (AHP) (Saaty, 1980).

  • 2.

    Outranking methods such as ELECTRE II (Roy and Bertier, 1971), ELECTRE III (Roy, 1978) ELECTRE IV (Roy and Hugonnard, 1982), and PROMETHEE methods (Brans and Vincke, 1985). For a recent summary of ELECTRE methods, see Rogers et al. (1999).


Different decision aid methods often provide different results with the same data. In order to choose a suitable method for a real-life problem, it is important to understand the differences and similarities between different methods. For example, the interpretation of criteria weights is fundamentally different in the utility and outranking models. In the linear utility model, weight ratios represent trade-off ratios between criteria (Keeney and Raiffa, 1976). In the outranking model, weights can be interpreted as votes for different criteria (Vincke, 1992). Unfortunately it is not possible to form any simple relationship between the weights of the utility model and the outranking model. However, since the purpose of different MCDM models is to aid the human DM to make good decisions, some kind of implicit, fuzzy relationship between the models must exist. In particular, one could expect different decision models often to discern clearly superior and inferior alternatives from the remaining alternatives identically, but produce more or less different results for a gray mass of mediocre alternatives.

The authors have previously compared SMART, PROMETHEE I and II, and ELECTRE III methods using real-life data (Salminen et al., 1998). Such an approach is necessarily restricted, because only a relatively small number of cases can be considered. Furthermore, real-life DMs are often not willing to participate in elaborate tests of different decision models.

In this paper we compare the linear utility function model with the pseudo-criteria outranking model using a large number of problems with different number of criteria and alternatives. Since it is not possible to conclude much about the preference models in real applications, we solve a large number of randomly generated test problems with both methods and analyze the differences in the results. The comparisons are performed using two variants of stochastic multi-criteria acceptability analysis (SMAA), which is a family of preference information free decision support methods.

Preference information free methods that generate weights internally are particularly suitable for comparing the decision models, because these methods do not depend on the interpretation of the weights. SMAA and SMAA-3 methods were selected for the comparisons, because these methods are based on an additive utility function and an outranking model, correspondingly. While SMAA models' inaccuracy or uncertainty of the criteria through probability distributions, SMAA-3 applies the `French School' pseudo-criteria and models inaccuracy/uncertainty using indifference and preference thresholds (so-called double threshold model).

Various preference information free methods have been developed for discrete MCDM problems. Charnetski (1973) and Charnetski and Soland (1978) introduced the comparative hyper-volume criterion, based on computing for each alternative the volume of the multi-dimensional weight space that makes the alternative most preferred. This method can handle preference information in the form of linear constraints for the weights, but is restricted to deterministic criteria values and an additive utility or value function. Rietveld and Ouwersloot (1992) presented a similar method for problems with ordinal criteria and ordinal preference information. Bana e Costa (1986) introduced the overall compromise criterion method for identifying alternatives generating the least conflict between several DMs. This method can handle partial preference information in the form of arbitrary weight distributions. Lahdelma et al. (1998) developed the SMAA method, which is able to handle weight distributions, stochastic criteria values, and additive utility functions. The SMAA-2 method by Lahdelma and Salminen (1997) and Lahdelma and Salminen (2001) is able to handle arbitrarily shaped utility functions and extends the weight space analysis to all ranks. The method provides acceptability indices describing the variety of different preferences supporting an alternative for a particular rank, central weights describing the preferences of a typical DM supporting an alternative, and confidence factors describing the reliability of the analysis. The SMAA-D method by Lahdelma et al. (1999) is similar to SMAA-2, but uses the data envelopment analysis efficiency measure as a kind of utility function. The SMAA-O method extends SMAA-2 to handle ordinal criteria (Miettinen et al., 1999). The SMAA-3 method by Hokkanen et al. (1998) applies, instead of a utility function, the ELECTRE III outranking model with pseudo-criteria.

The SMAA methods were developed in the context of aiding public environmental multi-criteria decision-making problems with large groups of DMs. Past SMAA applications include developing the Helsinki general cargo harbor (Hokkanen et al., 1999a; Salminen et al., 1996), defining the implementation order of a general plan with SMAA-3 (Hokkanen et al., 1998), selecting a region for a waste treatment facility (Hokkanen et al., 1999b), supporting a technology competition for cleaning polluted soil (Hokkanen et al., 2000), and choosing a reparation method for a landfill (Lahdelma et al., 2000b). In such decision-making problems the DMs are often unable or unwilling to provide weights for decision criteria during the decision process (Lahdelma et al., 2000a).

We seek answers to three questions:

  • 1.

    How similar are the results provided by the models?

  • 2.

    Are there systematic differences between the models?

  • 3.

    How could one select the thresholds in the pseudo-criteria model to match a utility model with given probability distributions?


In the following, we first describe the two variants of the SMAA method. In Section 3 we describe the experimental design and how the test problems were generated. In Section 4 we provide the results of the tests. This is followed by a conclusion.

Section snippets

The multi-criteria decision problem

We consider a multiple DM, discrete, finite, multiple criteria decision problem. The decision problem is defined by a set of m disjoint alternatives X={x1,x2,…,xm}, from which one is to be chosen. Alternatively, the task could be to choose a set of good alternatives or to provide a partial or complete ranking of the alternatives. Each alternative is evaluated in terms of n noncommensurate criteria, i.e., the alternatives are represented by vectors xiRn, whose elements are denoted by xij.

Experimental design

The two preference models were compared by using randomly generated test problems. Only disjoint nondominated alternatives were included in the test problems, because dominated alternatives can in general be rejected from further analysis regardless of the MCDM method used. The test problems were generated randomly using a subroutine Nondom(m,n,s) implemented in C++. The arguments are m=number of alternatives, n=number of criteria, and s=seed for the random number generator. The random seed is

Results

It is very rare that two or more alternatives share the first rank in the utility model. In contrast, in the pseudo-criteria model with large indifference thresholds, shared first ranks are frequent particularly in problems with few criteria. Thus, the pseudo-criteria model produces on average larger acceptabilities than the utility model. According to the test runs for βp=βq=50% and m=8 the acceptability index is on average three times larger in the pseudo-criteria model than in the utility

Conclusion

We have compared two techniques for modeling the DMs' preference structure and uncertainty of criteria using SMAA. The original SMAA method is based on utility theory and probability distributions for uncertain criteria while SMAA-3 uses an ELECTRE III-type pseudo-criteria (double threshold model) for modeling both the preference structure and inaccuracy/uncertainty of the criteria (i.e., the `French School' technique). Both methods were developed for situations where it is difficult or

Acknowledgements

This research was supported by the National Technology Agency of Finland, the Academy of Finland, and the Wihuri Foundation.

References (28)

  • J. Hokkanen et al.

    Determining the implementation order of a general plan by using a multicriteria method

    Journal of Multi-Criteria Decision Analysis

    (1998)
  • Hokkanen, J., Lahdelma, R., Salminen, P., 1999b. Selecting a region for a waste treatment facility using stochastic...
  • R.L. Keeney et al.

    Decisions with Multiple Objectives: Preferences and Value Tradeoffs

    (1976)
  • Lahdelma, R., Salminen, P., 1997. Identifying compromise alternatives in group decision-making by using stochastic...
  • Cited by (0)

    View full text