Multiple criteria decision making with reliability of assessment

Abstract

The weight and reliability of an individual assessment are two important concepts considered in the evidential reasoning (ER) approach. Through analyzing the existing studies on the combination of individual assessments with both their weights and reliabilities considered in the ER context, their deficiencies are identified in accordance with two principles. One principle is developed in the situation where a specific individual assessment is fully unreliable and the other is developed in the situation where all individual assessments are fully reliable. To address the deficiencies, this paper proposes a new method. In the method, a combination process that takes into account both the weights and reliabilities of individual assessments simultaneously is developed to generate the overall assessment. It is theoretically proven that the combination process satisfies the two principles. Three ways are designed to help a decision maker to flexibly provide individual assessments and determine their reliabilities. A strategic project evaluation problem for an enterprise located in Changzhou, Jiangsu, China is analyzed using the proposed method as a case study to demonstrate its validity and applicability. These are highlighted by its comparison with two existing methods.

Appendix Proof of Theorem 1

Without the loss of generality, suppose that r_i,l = 0 and r_j,l > 0 for j ≠ i. Under this assumption, it can be derived from Eqs. (22)–(24) that

$$ \hat{\beta }_{n,b(i)} (a_{l} ) = \vec{\beta }_{n,b(i - 1)} (a_{l} ), $$

(27)

$$ \hat{\beta }_{\Omega ,b(i)} (a_{l} ) = \vec{\beta }_{\Omega ,b(i - 1)} (a_{l} ), $$

(28)

and

$$ \hat{\beta }_{P(\Omega ),b(i)} (a_{l} ) = \vec{\beta }_{P(\Omega ),b(i - 1)} (a_{l} ). $$

(29)

This indicates that the individual assessment B(e_i(a_l)) has no influence on the combination result of the first i-1 assessments and further on the overall assessment.

When there are multiple individual assessments with zero-valued reliabilities, it can be similarly known that these assessments contribute nothing to the overall assessment. As a result, the principle presented in Proposition 1 is satisfied by the combination presented in Definition 3.

To focus on the principle presented in Proposition 2, the process of iteratively combining individual assessments developed by Yang and Xu (2013) is presented.

Definition A.1

(Yang & Xu, 2013) Given the individual assessments B(e_i(a_l)) (i = 1, …, L) and their weights w_i, the combination result of the first i assessments is defined as.

$$ \left\{ {\left( {H_{n} ,\beta_{n,b(i)} \left( {a_{l} } \right)} \right),n = { 1}, \ldots ,N; \, \left( {\Omega ,\beta_{\Omega ,b(i)} \left( {a_{l} } \right)} \right)} \right\}, $$

(30)

where

$$ \beta_{n,b\left( i \right)} \left( {a_{l} } \right) \, = \frac{{\hat{\beta }_{n,b(i)} (a_{l} )}}{{\sum\nolimits_{n = 1}^{N} {\hat{\beta }_{n,b(i)} (a_{l} )} + \hat{\beta }_{\Omega ,b(i)} (a_{l} )}}, $$

(31)

$$ \beta_{\Omega ,b\left( i \right)} \left( {a_{l} } \right) \, = \frac{{\hat{\beta }_{\Omega ,b(i)} (a_{l} )}}{{\sum\nolimits_{n = 1}^{N} {\hat{\beta }_{n,b(i)} (a_{l} )} + \hat{\beta }_{\Omega ,b(i)} (a_{l} )}}, $$

(32)

$$ \vec{\beta }_{n,b(i)} (a_{l} ) = \frac{{\hat{\beta }_{n,b(i)} (a_{l} )}}{{\sum\nolimits_{n = 1}^{N} {\hat{\beta }_{n,b(i)} (a_{l} )} + \hat{\beta }_{\Omega ,b(i)} (a_{l} ) + \hat{\beta }_{P(\Omega ),b(i)} (a_{l} )}}, $$

(33)

$$ \vec{\beta }_{\Omega ,b(i)} (a_{l} ) = \frac{{\hat{\beta }_{\Omega ,b(i)} (a_{l} )}}{{\sum\nolimits_{n = 1}^{N} {\hat{\beta }_{n,b(i)} (a_{l} )} + \hat{\beta }_{\Omega ,b(i)} (a_{l} ) + \hat{\beta }_{P(\Omega ),b(i)} (a_{l} )}}, $$

(34)

$$ \vec{\beta }_{P(\Omega ),b(i)} (a_{l} ) = \frac{{\hat{\beta }_{P(\Omega ),b(i)} (a_{l} )}}{{\sum\nolimits_{n = 1}^{N} {\hat{\beta }_{n,b(i)} (a_{l} )} + \hat{\beta }_{\Omega ,b(i)} (a_{l} ) + \hat{\beta }_{P(\Omega ),b(i)} (a_{l} )}}, $$

(35)

$$ \begin{aligned} \hat{\beta }_{n,b(i)} (a_{l} ) = & \left[ {\left( {1 - w_{i} } \right)\vec{\beta }_{n,b(i - 1)} (a_{l} ) + \vec{\beta }_{P(\Omega ),b(i - 1)} (a_{l} )w_{i} \cdot \beta_{n,i} \left( {a_{l} } \right)} \right]\\ & + \vec{\beta }_{n,b(i - 1)} (a_{l} ) \cdot w_{i} \beta_{n,i} (a_{l} ) \\ & + \vec{\beta }_{n,b(i - 1)} (a_{l} ) \cdot w_{i} \beta_{\Omega ,i} (a_{l} ) + \vec{\beta }_{\Omega ,b(i - 1)} (a_{l} ) \cdot w_{i} \beta_{n,i} (a_{l} ) \\ \end{aligned}, $$

(36)

$$ \begin{aligned} \hat{\beta }_{\Omega ,b(i)} (a_{l} ) =& \left[ {\left( {1 - w_{i} } \right)\vec{\beta }_{\Omega ,b(i - 1)} (a_{l} ) + \vec{\beta }_{P(\Omega ),b(i - 1)} (a_{l} )w_{i} \cdot \beta_{\Omega ,i} \left( {a_{l} } \right)} \right]\\ & + \vec{\beta }_{\Omega ,b(i - 1)} (a_{l} ) \cdot w_{i} \beta_{\Omega ,i} (a_{l} ) \end{aligned}, $$

(37)

and

$$ \hat{\beta }_{P(\Omega ),b(i)} (a_{l} ) = \left( {1 - w_{i} } \right)\vec{\beta }_{P(\Omega ),b(i - 1)} (a_{l} ). $$

(38)

Suppose that r_i,l = 1 for i = 1, …, L. Then, it can be found that Eqs. (22)-(24) reduce to Eqs. (36)–(38). In this situation, the overall assessment generated using Definition 3 is the same as the one generated using Definition A.1.

In other situations, where the above assumption is not satisfied, one can find that the combination result derived from Definition 3 is different from the result derived from Definition A.1 through comparing Eqs. (22)–(24) with Eqs. (36)–(38).

As a whole, this theorem is verified. □