On the conjoint estimation of inconsistency and intransitivity of pairwise comparisons

Abstract

Consistency and transitivity are two desirable properties of valued preferences which, however, are seldom satisfied in real-world applications. Different indices have been proposed to measure inconsistency and intransitivity separately, and recently scholars tried to merge these two concepts and use them in concert to estimate the irrationality of preferences. In this paper we formally investigate the existence (or non-existence) of functions capable of capturing both phenomena at the same time.

Introduction

Very often, in the practice of operations research and multi-criteria decision making, mathematical models require the elicitation of weights of alternatives and criteria. In this framework, psychological reasons related to our cognitive limits were among the factors triggering the introduction of the method of pairwise comparisons. Such a method allows the derivation of weights of alternatives from a set of pairwise comparisons between them. Different representations of valued preferences have been proposed in the literature, but the most widely known is probably the multiplicative model, which is also (but not exclusively) employed in the Analytic Hierarchy Process  [17]. It is to the multiplicative model that the findings of this paper directly apply, bearing in mind that the same conclusions can be extended to other representations of preferences, by means of appropriate group isomorphisms  [4]. Formally, in its multiplicative form, the method of pairwise comparisons assumes that, given a set of alternatives $\{1,\dots ,n\}$, a decision maker can express pairwise judgments $a_{ij}>0\forall i,j\in \{1,\dots ,n\}$ on them, where the value of $a_{ij}$ is an estimation of the ratio $w_i/w_j$ where $w_i$ and $w_j$ are the weights of $i$ and $j$, respectively. A pairwise comparison matrix $A={\left(a_{ij}\right)}_{n\times n}$ is nothing else but a convenient mathematical structure where the preferences of a decision maker in the form of pairwise comparisons are collected. Since reciprocity $a_{ij}=1/a_{ji}$ is usually assumed, a pairwise comparison matrix has the following canonical and simplified forms, respectively:

$$A={\left(a_{ij}\right)}_{n\times n}=\left(\begin{array}{cccc}\hfill a_{11}\hfill & \hfill a_{12}\hfill & \hfill \cdots \hfill & \hfill a_{1n}\hfill \\ \hfill a_{21}\hfill & \hfill a_{22}\hfill & \hfill \cdots \hfill & \hfill a_{2n}\hfill \\ \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill \ddots \hfill & \hfill ⋮\hfill \\ \hfill a_{n1}\hfill & \hfill a_{n2}\hfill & \hfill \cdots \hfill & \hfill a_{nn}\hfill \end{array}\right)=\left(\begin{array}{cccc}\hfill 1\hfill & \hfill a_{12}\hfill & \hfill \cdots \hfill & \hfill a_{1n}\hfill \\ \hfill \frac{1}{a_{12}}\hfill & \hfill 1\hfill & \hfill \cdots \hfill & \hfill a_{2n}\hfill \\ \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill \ddots \hfill & \hfill ⋮\hfill \\ \hfill \frac{1}{a_{1n}}\hfill & \hfill \frac{1}{a_{2n}}\hfill & \hfill \cdots \hfill & \hfill 1\hfill \end{array}\right)\text{.}$$

In the following, we will call $\mathcal{A}$ the set of all pairwise comparison matrices,

$$\mathcal{A}=\{A={\left(a_{ij}\right)}_{n\times n}|a_{ij}>0,a_{ij}{a}_{ji}=1\forall i,j,n\ge 2\}\text{.}$$

One reasonable expectation is that, for instance, a decision maker stating that $w_i$ is 2 times $w_j$ ($a_{ij}=2$) and $w_j$ is 3 times $w_k$ ($a_{jk}=3$), should also state that $w_i$ is 6 times $w_k$ ($a_{ik}=6$). This intuition translates into the following condition, according to which a pairwise comparison matrix is consistent if and only if

$$a_{ik}=a_{ij}{a}_{jk}\forall i,j,k\text{.}$$

Such a condition of consistency is often called cardinal consistency. In the following, we call ${\mathcal{A}}^{\ast }\subset \mathcal{A}$ the set of all consistent pairwise comparison matrices, i.e. 

$${\mathcal{A}}^{\ast }=\{A={\left(a_{ij}\right)}_{n\times n}|A\in \mathcal{A},a_{ik}=a_{ij}{a}_{jk}\forall i,j,k\}\text{.}$$

Among others, Saaty claimed that a matrix should be ‘near consistent’ to represent the true preferences of a decision maker. Thus, various functions $I:\mathcal{A}\to \mathbb{R}$, usually called inconsistency indices, have been proposed in the literature to measure the deviation of a matrix from the consistent condition (1). One can refer to a review and numerical study of these indices  [2]. Recently, a set of properties of inconsistency indices was introduced and studied  [1], [3]. Since these properties are going to be used to derive the main results of this paper, it is convenient to recall them here.

• Index $I$ attains its minimum value $\nu \in \mathbb{R}$ if and only if $A$ is consistent, i.e. $I\left(A\right)=\nu \iff A\in {\mathcal{A}}^{\ast }\forall A\in \mathcal{A}$.

• Index $I$ is invariant under permutation of alternatives, i.e. $I\left(A\right)=I\left(PA{P}^T\right)\forall A\in \mathcal{A}$ and for all permutation matrices $P$.

• As the preferences are intensified the inconsistency cannot decrease. Define $A\left(b\right)={\left(a_{ij}^b\right)}_{n\times n}$. Formally, $I\left(A\left(b\right)\right)\ge I\left(A\right)\forall A\in \mathcal{A}$ and $b>1$.

• Consider a matrix $A\in {\mathcal{A}}^{\ast }$ and the matrix $A_{pq}\left(\delta \right)$ which is the same as $A$ except for entries $a_{pq}$ and $a_{qp}$ which are replaced by $a_{pq}^{\delta }$ and $a_{qp}^{\delta }$, respectively. Then, $I\left(A_{pq}\left(\delta \right)\right)$ is a quasi-convex function of $\delta \in [0,\infty [$ with minimum in $\delta =1$.

• Index $I$ is a continuous function of the entries of $A$ for all $A\in \mathcal{A}$.

• Index $I$ is invariant under inversion of preferences, i.e. $I\left(A\right)=I\left(A^T\right)\forall A\in \mathcal{A}$.

Although the necessity of these properties to characterize inconsistency indices is a moot point, knowing that a function satisfies P1–P6 certainly provides evidence that the function behaves reasonably when used to quantify the inconsistency.

Motivated by the excessive restrictiveness of the condition of consistency, a weaker condition has been often used to assess the rationality of pairwise comparisons. Its motivation is deeply grounded in rational choice theory where “transitivity of preferences is a fundamental principle shared by most major contemporary rational, prescriptive, and descriptive models of decision making”  [15]. The principle of transitivity simply states that a decision maker preferring $i$ to $j$ ($a_{ij}\ge 1$) and $j$ to $k$ ($a_{jk}\ge 1$), should also prefer $i$ to $k$ ($a_{ik}\ge 1$). Thus, transposing this principle into our framework we obtain that a pairwise comparison matrix is transitive if and only if

$$a_{ij}\ge 1\text{and}{a}_{jk}\ge 1\Rightarrow a_{ik}\ge 1\forall i,j,k\text{.}$$

In the literature, the concept of transitivity has equivalently gone under the name of ordinal consistency, or weakly stochastic transitivity in the context of reciprocal relations  [6]. Similarly to the case of inconsistency, indices have been proposed to assess the extent of the violation of this condition in pairwise comparison matrices. Prominent examples are the studies by Kendall and Babington Smith  [11], Jensen and Hicks  [10] and Iida  [9]. It appears that a common idea behind all these evaluations of the degree of intransitivity is that the more violations of condition (2) there are in the preferences, the more intransitive these should be considered. Hence, we can formulate the following property, seemingly important to characterize the extent of the violation of transitivity, which requires a type of monotonicity of the function $I$ with respect to the number of cycles in the preferences.

• Let $C\left(A\right)$ be the number of violations of condition (2) in $A\in \mathcal{A}$, and consider two pairwise comparison matrices $A,B\in \mathcal{A}$ of the same order. Then a function $I$ satisfies P7 if $C\left(A\right)\ge C\left(B\right)$ implies $I\left(A\right)\ge I\left(B\right)$.

Recently, the interest of researchers has been drawn by the connections between consistency and transitivity. Kwiesielewicz and Van Uden  [13] underlined that consistency implies transitivity, but not vice versa. Li and Ma  [14] proposed a bi-objective optimization problem to minimize two objective functions related to inconsistency and intransitivity. By means of computational experiments, Siraj et al.  [18] showed that the well-known index $\mathrm{CR}$ has problems to quantify the intransitivity of preferences. Later on, Siraj et al.  [19] proposed both an inconsistency index (congruence) and an intransitivity index (dissonance) and they used the latter to provide additional information on the former. Very recently, Cooper and Yavuz  [5] acknowledged the existence of studies on (i) cardinal inconsistency, (ii) ordinal inconsistency, (iii) a combination of both. All in all, the literature is rich of heuristic approaches to embed both consistency and transitivity in the same mathematical model to analyze and improve the rationality of the decision maker  [8], [21], [20].