An ordinal ranking criterion for the subjective evaluation of alternatives and exchange reliability

Highlights

• Decision makers (DMs) choose among alternatives from different information sources.

• Each source must describe the finitely many characteristics defining his alternative.

• DMs and sources differ in their respective perceptions of the available alternatives.

• Differences between their preference orders determine expected evaluation frictions.

• A reliability ranking determines the DM’s likelihood of choosing a given alternative.

Abstract

We consider the problem of a decision maker (DM) who must choose among a set of alternatives offered by different information senders (ISs). Each alternative is characterized by finitely many characteristics. We assume that the DM and the ISs have their own perception of the available alternatives. These perceptions are reflected by the evaluations provided for the characteristics of the alternatives and the order of importance assigned to the characteristics. Due to these subjective components, the DM may not envision the exact alternative that an IS describes, even when a complete description of the alternative is provided. These subjective biases are common in the literature analyzing the effect of framing on the behavior of the DMs. This paper provides a normative setting illustrating how the DMs should consider these differences in perception when interacting with other DMs. We design an evaluation criterion that allows the DM to generate a reliability ranking on the set of ISs and, hence, to quantify the likelihood of choosing any alternative. This ranking is based on the existing differences between the preference order of the DM and those of the ISs. Our results constitute a novel approach to choice and search under uncertainty that enhances the findings of the expected utility literature. We provide several examples to demonstrate the applicability of the method proposed and exhibit the efficacy of the ranking criterion designed.

Introduction

It is widely known in the social sciences that decision makers (DMs) are highly sensitive to the way and the order in which information is presented to them. This fact is referred to as the framing effect and imposes a subjective bias in the choices of DMs. The literature on the framing effect encompasses several disciplines, ranging from psychology to economics. Kahneman and Tversky [28] provide an extensive description of the literature in both fields. In particular, as emphasized by Gächter et al. [20], this effect prevails even among those academics who study it, mainly experimental economists.

Thus, the order in which the characteristics of a choice object are described to the DMs is fundamental in determining their behavior. Similarly, the way in which an information sender (IS) describes a given choice object reflects the subjective importance assigned to each one of the characteristics of the object. To our knowledge, a formalization of the normative framework determining the behavior of the DMs when interacting with others while acknowledging this fact has not been provided in the literature.

The aim of this paper is to define an ordinal criterion that allows the DMs to generate a ranking based on the differences existing between their preference order and those of the ISs. Indeed, the main characteristics of a product according to a DM are generally given a more detailed and careful description than those considered as secondary. For example, when writing a paper, we academics highlight those results and implications that we consider more relevant and develop and explain them to a greater extent than those we consider as secondary. When selecting a research paper or book to read, we concentrate on the main features described by the authors and compare them to those we prioritize in our own research. In the same way, preference similarity and common reactions to news, illustrating the common/coordinated interests existing between investors, have been shown to determine their investment behavior [1], [26].

In the current paper, the order in which information is displayed by the ISs will directly affect the choices made by the DMs. Operational researchers and computer scientists have already started considering the elicitation of product characteristics and valuations from the text that describes it. For example, [21] illustrate how characteristic and value pairs can be reliably extracted from textual product descriptions.

Several academic disciplines deal with the search, choice and exchange problems of DMs. These problems are analyzed from substantially different perspectives depending on the discipline. We will concentrate on the main findings provided by the consumer choice, psychology, marketing and economic literatures from an applied perspective. A common starting point to all these disciplines is the fact that the DMs must decide whether or not to purchase a product given their evaluation of its characteristics. These evaluations are assumed to be based on the DMs’ subjective perception of the product and determine the likelihood of the DMs purchasing the product.

In consumer choice, the acquisition of a product and the basis of models such as the technology adoption one rely on the relevant characteristics of the product and their importance considered from the DM’s point of view. See, among others, Brown et al. [8], Blackwell et al. [7], Davis [11], Davis et al. [12], Dillon and Morris [17], Hess et al. [24], Shackel [41], and Venkatesh et al. [48].

Psychologists studying the behavior of consumers have found that characteristics are ranked based on their order of importance and their influence on the attitude of the DMs towards the product. See, for instance, Jaccard et al. [27] and Kimmel [29]. In particular, Fishbein [18] shows that the attitude towards a product is generally defined as the weighted sum of the DM’s beliefs regarding its salient characteristics.

Several marketing papers [5], [6], [7], [22], [33], [39] have illustrated empirically how the attitude of the DMs towards a product can be described as a function of the relative importance of each characteristic and the DMs’ belief regarding the performance of the product on each characteristic.

The economic literature has given considerable importance to the improvement of characteristics when dealing with the introduction of new products in the market [23]. Furthermore, it introduced the idea of defining products as vectors of characteristics [31]. Nevertheless, it has not pursued this specific line of research when analyzing market exchange environments.

Finally, information scientists have also dealt with the behavior of DMs when facing an uncertain environment with choices determined by multiple characteristics. Their contributions are developed at a more formal level and generally follow a fuzzy environment approach to account for the frictions caused by the uncertainty. See, among others, Aliev et al. [3], [4], Di Caprio et al. [15] and Tavana et al. [44].

We consider the problem of a DM who must rank several alternatives described by other DMs, who are referred to as information senders (ISs). Each alternative consists of a fixed number of characteristics. We assume that each IS is endowed with a preference order on the characteristics of the alternative he is offering to the DM and that this fact affects their description of the alternative. The preference order of each IS will generally differ from that of the DM and those of the other ISs.

Thus, a DM whose preference order does not exactly coincide with that of the IS, a fact that is directly reflected in the order in which the characteristics are described, should expect a larger friction in the evaluation of the alternative than if the orders were to exactly coincide. In this latter case, the IS will provide a more detailed and careful description of the characteristics that the DM considers more important. Detailed descriptions allow the DM to form a more reliable expectation of the product to be obtained from an exchange.

Following the consumer behavior and marketing approaches to choice under uncertainty [34], [38], we will assume that the DM chooses to interact with the IS that he considers most reliable while providing the highest expected utility.

Tavana et al. [44] propose a theoretical model where each DM is endowed with a product. The DMs are aware of the fact that they do not perceive the characteristics of the products in the same way and define their beliefs based on these differences in perception. The DMs’ beliefs are used to construct their subjective expected utility derived from each alternative. The authors illustrate how the DMs may agree to exchange products and end up with a less preferred product that the one they initially owned. This is the case even if strategic reporting is excluded from the model. However, Tavana et al. [44] make no attempt to quantify the distance among the perceptions of the DMs. Also, the authors do not consider the problem of ranking several alternatives.

We introduce a novel and easy to implement ranking criterion on which the DMs can base their choice from a set of alternatives described by different ISs. We will base the reliability between the DMs and the ISs on the ordinal properties of the preferences defining the utility functions of the DMs [14]. This will be done in a fully cooperative environment where all valuation frictions are reflected in the order of the characteristics chosen to describe the alternatives being considered. That is the information transmitted by the ISs is assumed to be truthful and reliable in the following sense.

• We do not consider strategic reporting between the ISs and the DMs. The capacity of the ISs to misrepresent the information content transmitted to the DMs has been widely studied in the economic literature [2], [13], [32].

• We do not consider linguistic imprecisions in the description of the alternatives by the ISs as is the case in several fuzzy variants of the decision making literature; see, among others, Aliev et al. [3], [4].

We introduce two novel indexes to quantify the differences between the order assigned by the DM on the characteristics of an alternative and that assigned by the ISs. Namely,

• A coordination index, in order to measure the shortest distance between the position of the most valued characteristic in the DM’s order and that of the same characteristic in the order of each IS.

• A synchronization index, in order to measure the pairwise distances between the order position assigned by the DM to each characteristic and the order positions assigned to the same characteristics by each IS.

Both these indexes will be combined to define a composite index that allows the DMs to weight and rank the expected utility values of the alternatives proposed by the ISs. We will provide several examples to demonstrate the applicability of the method proposed and exhibit the efficacy of the ranking criterion designed.

The main research areas where the results obtained can be directly applied are the following.

• The analysis of (bilateral or multilateral) economic exchange settings where an ordered description of a set of product characteristics is provided by the ISs to the DMs [44]. This type of information structure applies also to online search environments, where consumers are presented with a set of product characteristics displayed within several links when browsing through different websites [10].

• A second area of application follows from online search environments and concentrates on the agent recommendations provided to the DMs based on their purchasing history [35], [40], [43]. These recommendations involve the description of a series of product characteristics by unknown third parties to the DM. Absent credibility and trust considerations [19], [30], the DM must choose which one among the recommendations available to follow.

• Finally, a similar approach must be considered when deciding what project to undertake or how to proceed with one already being developed based on the different descriptions provided by third parties [37], [47]. This scenario can take place within a given production chain [42], [36] with DMs subject to time pressure constraints [25], [9].

The paper proceeds as follows. In Section 2, we define the basic concepts and notations. In Section 3, we model the point of view of the ISs and the DM. We introduce relative coordination and synchronization indexes in Sections 4 Introducing the relative coordination index, 5 The relative synchronization index, respectively, and compare them numerically in Section 6. Both these indexes are combined together in Section 7 to define the ordinal reliability ranking criterion proposed in the paper. Section 8 summarizes the main findings and suggests potential extensions.