Visualising multi-criteria weight elicitation by multiple stakeholders in complex decision systems

Abstract

An efficient and transparent weight elicitation technique is proposed for inclusion into the adaptive, systemic, control and multi-criteria-based methodology, in short ASCM, the purpose of which is piloting in real time complex systems by combining system dynamics (SD) and multi-criteria decision analysis (MCDA). Piloting policies are established and revised on a regular basis and/or constant real-time observation by means of SD simulations; at each revision step groups of stakeholders choose by means of MCDA tools the best policy to be implemented for the ensuing time periods when adaptations are necessary to account for the actual system evolution. An essential but difficult issue at each policy revision step is the weight elicitation process of multiple criteria by the multiple stakeholder groups (SH). The proposed procedure with a strong mathematical background does not require excessive cognitive effort for SH with different priorities and decisional powers. It consists in a two-step approach defining firstly importance classes on ordinal Likert scales, and secondly profiles on those scales for the criteria. It appears to be simple though rigorous; it easily allows fast sensitivity analyses when confronting different opinions. A didactic example and a fishery-management case study illustrate these properties by means of visualisation tools facilitating consensus-seeking among SH.

Appendix: Proofs of the property that the centroid weights of ranked weights are the same as Rank-Sum surrogate weights (Sect. 3.1 of the main text)

We start with the observation that not-normalised weights are defined up to a positive multiplicative factor. In the elicitation process, there is no need to produce normalised weights, so that the Rank Order Centroid (ROC) simplex (2) in Sect. 3.1 should not include a normalising constraint, while the not-normalised weights can be bound by any positive number, using 1 without loss of generality we then consider the weight simplex:

$$W \in \left\{ {0 \le W_{1} \le W_{2} \le \cdots \le W_{K} \le 1} \right\}$$

(A1.1)

1.1 First proof

All point values in the simplex (A1.1) give equally valid sets of not-normalised weights, because the sole available information from stakeholders is ranking (A1.1). In the K-D space the simplex is a triangle K₂ in 2-D, a tetrahedron K₃ in 3-D, and generally a K-simplex. It is well-known that triangle-based simplices (A1.1) are such that their centroid’s coordinates (=centre-of-gravity in the physical sense) are the mean values of the coordinates of their extreme points (Ahn 2017).

The (K + 1) extreme points of any K-simplex are clearly $\left( {0, \ldots ,0} \right);\left( {0,0, \ldots ,1} \right);\left( {0, \ldots ,1,1} \right); \ldots ;\left( {1, \ldots ,1} \right)$, so that the centroid coordinates, i.e., the centroid weights are given by:

$$W = \frac{{\left( {1,2, \ldots ,K} \right)}}{K + 1}$$

(A1.2)

These values are proportional to the not-normalised rank-sum (RS) weights (1) in Sect. 3.1.□

1.2 Second proof

As all point values in the K-simplex (A1.1) bounded by 1 can equally be chosen as not-normalised weight sets, the centroid weights are obtained as the set of mean values of K random ordered-values in [0,1]. This is easily done by generating many times K random numbers uniformly distributed in the [0,1] interval, ranking them in increasing order, and obtaining their mean values. The K ranked random numbers obey order-statistics probability laws (Arnold et al. 1992; pp. 13–14); it is known that their mean values—being the not-normalised weights—are given by (A1.2) as obtained numerically in approximation with simulations.□