Mining maximum consensus sequences from group ranking data

Abstract

In the last decade, the problem of getting a consensus group ranking from all users’ ranking data has received increased attention due to its widespread applications. Previous research solved this problem by consolidating the opinions of all users, thereby obtaining an ordering list of all items that represent the achieved consensus. The weakness of this approach, however, is that it always produces a ranking list of all items, regardless of how many conflicts exist among users. This work rejects the forced agreement of all items. Instead, we define a new concept, maximum consensus sequences, which are the longest ranking lists of items that agree with the majority and disagree only with the minority. Based on this concept, algorithm MCS is developed to determine the maximum consensus sequences from users’ ranking data, and also to identify conflict items that need further negotiation. Extensive experiments are carried out using synthetic data sets, and the results indicate that the proposed method is computationally efficient. Finally, we discuss how the identified consensus sequences and conflict items information can be used in practice.

Introduction

In recent decades, the group ranking problem has become an important and interesting study in decision making [8], [9], [10], machine learning [14], web search strategies [3], [7], and sports and behavioral issues. The essence of this problem is how to consolidate and aggregate decision makers’ rankings to obtain a group ranking that is representative of “better coherent” ordering for the decision makers’ rankings.

According to the completeness of preference information provided by decision makers, the group ranking problem can be roughly classified into two major approaches, the total ranking approach and the partial ranking approach. The former needs individuals to appraise all alternatives, while the latter only a subset of alternatives. Moreover, the compromised results of these approaches can be divided into two types: a full order and a partial order. Basically, requesting a full order result is a common goal in both the total ranking approach and the partial ranking approach. However, quite a few works in the partial ranking approach generate partial order results [9], [10], [16], [18], because when the provided preference information was too fragmented, the final result was hard to consolidate [9]. Each approach has its own advantages and has been successfully used in many applications. In this paper, we follow the line of the total ranking approach and propose a new group ranking algorithm which can generate partial order compromised results.

Roughly speaking, the goal of most total ranking methods is to determine a full ordering list of items that expresses the consensus achieved among a group of decision makers. Therefore, the advantage of these researches is that no matter how much users’ preferences conflict, an ordering list of all items to represent the consensus is always produced. Unfortunately, this advantage is also a disadvantage, because when there is no consensus or only slight consensus on items’ rankings, the previous approach still generates a total ordering list using their ranking algorithms. In such a situation, what we obtain is really not a consensus list, but merely the output of algorithms. Relying on this list to make any decision would be risky.

In the following, we use an example to explain the situation. Assume that we have four users U₁, U₂, U₃, and U₄ and five proposals A, B, C, D, and E. Table 1 shows the users’ ranking lists of the proposals, where “X > Y” means X is preferable to Y, and “X ⩾ Y” means X is at least as good as Y.

As seen in Table 1, all users agreed with the relationship {A > D > E}, but there is a conflict of opinion on proposals {A, C}, {B, C}, and {C, D}. In this situation, different algorithms will produce different results for these conflict items. For example, one algorithm may generate list {A > B > D > C > E}, but another may generate {C ⩾ A > B > D > E}. It is unwise to use either list to make decisions, because users only have consensus on {A > D > E} while the other parts are actually determined by the algorithm’s design.

This example indicates that we may have consensus on some items and conflicts on other items. In such situations, consensus decision-making theory stresses that we need a process to achieve the most agreeable decision among participants [25]. To support this decision process, we must have an algorithm that can find the maximum consensus lists from users’ ranking data and also identify conflict items that need further negotiation.

Traditionally, there are three formats to express users’ preferences about items in the total ranking approach. These formats include weights/scores of items, set of pairwise comparisons on the items and ranking lists of items. In this paper, we assume that an individual’s preferences can be represented as a total ranking list. An ordering sequence of items is called a consensus if a majority of users agree on this ordering and only a minority of users disagrees. Accordingly, we propose an algorithm to discover maximum consensus sequences from users’ ranking lists, where the maximum consensus sequences represent the maximum possible consensuses that can be achieved among most of users. By applying our algorithm to the above example, we can generate two types of results: the maximum consensus sequences, such as {A > D > E} in the above example, and the conflict items where users have no consensus.

Our approach has the following advantages in supporting group decision making:

• The majority preference can be met by the maximum consensus sequence proposed in this study. The degree of majority is adjustable.

• Users may have different opinions on items. During the decision process, users need to negotiate and resolve the objections of the minority in order to achieve the most agreements. Our algorithm can identify the conflict items that need further negotiation.

• Most previous research consolidated all users’ preferences into a total ranking list. This may suppress or ignore the opinions of the minority and could make the achieved decision difficult to implement. In our approach, we find only the maximum agreeable consensus from users, rather than forcefully producing a total ranking list for all users.

In the next section, we briefly review previous research about group ranking. Then, the problem definition and the proposed concept, consensus sequence, are described in detail. In the methodological section, an algorithm is established to find the maximum consensus sequences. In addition, a series of experiments is conducted to analyze the proposed method. After the experimental section, we will discuss how to apply the consensuses sequence in practice. Finally, we summarize some valuable and constructive conclusions.