Effect of sub-indicator weighting schemes on the spatial dependence of multidimensional phenomena

Abstract

The weighting of sub-indicators is widely debated in the composite indicator literature. However, these weighting schemes’ effects on the composite indicator’s spatial dependence property are still little known. This research reveals a direct relationship between the weighting scheme of sub-indicators and the spatial autocorrelation of the composite indicator. The Global Moran's Index (I) of composite indicators built using Data-driven (Moran’s I = 0.636) and Hybrid (Moran’s I = 0.597) weighting schemes is, on average, eleven percent higher than in the Equal-weights (Moran's I = 0.549) and Expert opinion (Moran's I = 0.560) weighting schemes. The average score of the composite indicator is higher when they are built by weighting schemes that better describe the spatial dependence. The spatial dependence of sub-indicators and composite indicators are not related. All fifteen sub-indicators show lower spatial autocorrelation than the composite indicators built by Expert opinion, Hybrid, and Data-driven weighting schemes. The spatial weighting matrix influences the spatial autocorrelation but does not change the robustness and quality parameters of the composite indicator. The research develops a Data-driven weighting scheme that allows individually or simultaneously considering the opinion of experts and parameters of quality and robustness of the composite indicator. It also offers the means to reduce judgment errors and evaluation biases in Expert opinion sub-indicator weighting schemes.

Appendix: weighting of sub-indicators by expert opinion

1.1 Selection of experts

Three criteria were used to select the experts. First, have a doctorate in human and social sciences, such as sociology and geography. Second, have a publication in scientific journals on social phenomena such as inequality, social exclusion, poverty, and social vulnerability. Third, know the social aspects of the study area.

The selection of experts from this set of criteria has advantages and disadvantages. On the one hand, this set of criteria limits the number of experts qualified to carry out the assessments. On the other hand, this set of criteria favors the quality and homogeneity of the evaluations.

Four experts were selected based on the virtual curriculum system created and maintained by the Brazilian National Council for Scientific and Technological Development. Different evaluation formats and the degree of consensus were adopted to avoid these experts' judgment errors and evaluation biases.

1.2 Assessment of alternatives

The assignment of weights by Expert opinion is commonly associated with the problem of judgment errors (Greco et al. 2019). One way to reduce judgment errors is to offer experts the possibility to choose the format for assessing alternatives (sub-indicators) that best suits them. Alternative assessment formats are not without limitations, but the flexibility of choosing the assessment format gives the expert psychological comfort that reduces judgment errors (Ekel et al. 2020).

• Ordering of Alternatives ($OA$) an array $O = \left\{ {o\left( {x_{1} } \right),o\left( {x_{2} } \right), \ldots , o\left( {x_{k} } \right), \ldots , o\left( {x_{n} } \right)} \right\}$, where $o\left( {x_{k} } \right)$ is a permutation function that defines the position of the alternative $x_{k}$ among the integer values {1,2, …, k,…, n}.

• Multiplicative Preference Relations ($MR$) reflect the preference intensity ratio between the alternatives $x_{k}$ and $x_{l}$, being understood as $x_{k }$ is $m\left( {x_{k} ,x_{l} } \right)$ times as good as $x_{l}$.

• Utility Values ($UV$) the preferences in $X$ are given as a set of n $UV$: $U = \{ u\left( {x_{1} } \right),u\left( {x_{2} } \right), \ldots ,u\left( {x_{k} } \right), \ldots ,u\left( {x_{n} } \right)$}, where $u\left( {x_{k} } \right) \in \left[ {0,1} \right]$ represents the $UV$ assigned to the alternative $x_{k}$.

• Fuzzy Estimates ($FE$) a fuzzy number that can be specified directly or through a linguistic variable in which the elements of ${\text{X}}$ are directly assessed by experts using a set of estimates $L = \left\{ {l\left( {x_{1} } \right),l\left( {x_{2} } \right), \ldots ,l\left( {x_{k} } \right), \ldots ,l\left( {x_{n} } \right)} \right\}$, where $l\left( {x_{k} } \right)$ is the $FE$ associated with the alternative $x_{k}$.

• Non-Reciprocal Fuzzy Preference Relations ($FR$) indicate the degree to which the alternative $x_{k}$ is, at least, as good as the alternative $x_{l}$, employing its membership function $0 \le {\upmu }\left( {x_{k} ,x_{l} } \right) \le 1.$

The experts assessed the weights of the sub-indicators through the Utility Values (two experts), Ordering of Alternatives, and Multiplicative Preference Relations evaluation formats. The assessments were standardized in the Non-Reciprocal Fuzzy Preference Relations format using the preference format transformation functions:

$${\text{FR}}\left( {x_{k} , x_{l} } \right) = \left\{ {\begin{array}{*{20}l} {\frac{1}{2} + \frac{{{\text{OA}}_{l} - {\text{OA}}_{k} }}{{2\left( {n - 1} \right)}}} \hfill & { {\text{if}}\quad {\text{ OA}}_{k} > {\text{OA}}_{l} } \hfill \\ 1 \hfill & { {\text{ if}}\quad A_{k} \le {\text{OA}}_{l} } \hfill \\ \end{array} } \right.$$

(13)

for the transformation ${\text{OA}}$ → ${\text{FR}}$;

$${\text{FR}}\left( {x_{k} , x_{l} } \right) = \left\{ {\begin{array}{*{20}l} {1 + \frac{1}{2}\log_{m} \frac{{{\text{MR}} \left( {x_{k} , x_{l} } \right)}}{{{\text{MR}} \left( {x_{l} , x_{k} } \right)}}} \hfill & { {\text{if}}\quad \log_{m} M\left( {x_{k} , x_{l} } \right) < 0 } \hfill \\ 1 \hfill & { {\text{if}}\quad \log_{m} M\left( {x_{k} , x_{l} } \right) \ge 0} \hfill \\ \end{array} } \right.$$

(14)

for the transformation ${\text{MR}}$ → ${\text{FR}}$;

$${\text{FR}}\left( {x_{k} , x_{l} } \right) = \left\{ {\begin{array}{*{20}l} {\frac{{{\text{UV}}\left( {x_{k} } \right)}}{{{\text{UV}}\left( {x_{l} } \right)}}} \hfill & { {\text{if}}\quad {\text{UV}}\left( {x_{k} } \right) < {\text{UV}}\left( {x_{l} } \right) } \hfill \\ 1 \hfill & { {\text{if}}\quad {\text{ UV}}\left( {x_{k} } \right) \ge {\text{UV}}\left( {x_{l} } \right)} \hfill \\ \end{array} } \right.$$

(15)

for the transformation ${\text{UV}}$ → ${\text{FR}}$.

Applying the transformation functions is to obtain the evaluations in a single format. Then, it is possible to calculate the weights of the sub-indicators according to the group of experts, applying the following expression:

$$F_{p}^{c} \left( {x_{k} } \right) = \mathop \sum \limits_{y = 1}^{v} w_{y} F_{p}^{y} \left( {x_{k} } \right)$$

(16)

where $x_{k} \in X,\, 0 \le w_{y} \le 1,$ for $y = 1,2, \ldots ,v,$ taking into account the condition $\sum\nolimits_{y = 1}^{v} {w_{y} = 1}$.

Table 

Table 10 Weights by experts, highest and lowest weights, and weights according to the group

10 shows the weights assigned by each expert, the highest and lowest weights per sub-indicator, and the weights according to the group of experts.

1.3 Individual and group degree of consensus

Another common problem of weighting by Expert opinion is associated with evaluation bias. This problem is especially relevant when the number of experts is small because biased assessments more strongly influence the results (Musa et al. 2019).

The degree of individual consensus measures how much an expert opinion diverges from the collective opinion, signaling possible evaluation biases. The individual degree of consensus is obtained applying the expression:

$$C^{I} = \mathop \sum \limits_{l \in L}^{m} \left( {1 - \frac{{\left| {O^{G} \left( {z_{k} } \right) - O^{{E_{i} }} \left( {z_{k} } \right)} \right|}}{n - 1}} \right)$$

(17)

Evaluation biases can be identified by considering the degree of consensus between the expert's assessments and the expert group's assessment. The acceptance threshold of the degree of consensus is a subjective measure defined by the decision-maker (Ekel et al. 2020). When the acceptance threshold of the degree of consensus is not reached, the decision-maker may request the reassessment of the alternatives or disregard the expert's assessments (Pedrycz et al. 2011). After identifying and eliminating evaluation biases, it is possible to calculate the degree of consensus of the group by the following expression:

$$C^{G} = \frac{{\mathop \sum \nolimits_{k = 1}^{n} C^{I} }}{n}$$

(18)

The research adopts the threshold of 0.70 as the individual and group degree of consensus. Table 11 shows that the four experts reached the adopted degree of consensus threshold.

Table 11 Expert and group degree of consensus