Modeling and predicting emerging inference-based decisions in complex and ambiguous legal settings

Abstract

Many decisions people make are based on multitudes of inferences. People have been shown to generate sense quite effortlessly––and compulsively––even in highly opaque situations. Recently, it has been suggested that the making of an inference-based decision may be accompanied by an increase in the coherence of assessments of the individual arguments related to the alternatives at hand. This suggests a constraint satisfaction reasoning process. In two complex and ambiguous law-related emerging decisions, assessments of inferences increasingly spread apart, even if no additional information was provided. Two approaches for studying emerging coherence are developed. First, the structures that emerge as participants progress from stage to stage in the judgment process are captured as principal components through factor analysis. Second, discriminant analysis is employed to test the predictive strength of the emerging cognitive structures vis-à-vis each sequential decision.

Introduction

Many decisions people make are based on multitudes of inferences. People have been shown to generate sense quite effortlessly––and compulsively––even in highly opaque situations. However, we do not know particularly much about how people generate rationally justifiable inferences and make decisions in these complex, ambiguous, and conflict-laden situations. It is particularly important to gain insight into how people integrate conflicting information. As a timely example, Brian Jenkins of the RAND Corporation––a long-time student of terrorism––claims that journalists are trained to make sense of events and create a narrative: “Journalists tell coherent stories. …Reality is far less coherent. In the interest of a coherent story, they may tell the wrong story” (Barringer, 2001, p. C9). The same can be said about many other areas of everyday and professional reasoning. Business, for example, provides an abundance of reasoning and decision making environments that can be characterized as opaque, risky, and ill-structured: trading, forecasting versus day-after market analysis, and the generation, dissemination, and utilization of buy–hold–sell recommendations. Other particularly interesting areas include political and ideological reasoning and the formation of attitudes and beliefs. For example, current US conservatism often combines favoring free access to guns, restricted access to abortion, with support for the death penalty, whereas liberalism tends to combine the opposite views; however, it is unclear what the three points of dispute have to do with one another (cf. Holyoak and Thagard, 1989, Holyoak and Thagard, 1997; Holyoak and Simon, 1999).

Some attempts to understand and model sense-making processes have focused on broad generalizations (Weick, 1995), others on logics suitable for on-line reasoning (Lundberg, 2000), and most frequently on the central role of decision rules on varying levels of abstraction (cf. Svenson, 1992). Many prominent rules make perfect sense after the fact––after sense has been made––but tend to be somewhat unwieldy and/or overly generic a priori. Likewise, judgment heuristics are likely to play a central role in everyday and professional reasoning, but do not, by themselves, specify how conflicting information can be integrated (Holyoak and Simon, 1999). Recently, Holyoak and Simon (1999) have suggested that the making of an inference-based decision may be accompanied by an increase in the coherence of assessments of the individual arguments related to the alternatives at hand. In these emerging decisions, assessments of inferences increasingly spread apart, with those supporting the favored alternative growing stronger and those supporting the less favored alternative growing weaker. This suggests a bi-directional (constraint satisfaction) reasoning process where the decisions are based on the inferences made from the provided information, and the emerging decisions, in turn, work backward to alter the strength of the inferences in order to yield coherent support. Importantly, the constraint satisfaction framework goes against a deep-rooted assumption about reasoning: that the flow of inference is inherently uni-directional.

More generally, making sense can be viewed as the activity of fitting something confusing into coherent mental representations that include concepts, beliefs, goals, and actions. Coherence theory (Thagard, 2001) approaches problems in terms of the satisfaction of multiple interacting soft constraints with highly interconnected elements. The elements are propositional or other types of representations that are connected via weighted links of coherence (positive or excitatory constraints) and incoherence (negative, interfering, or inhibitory constraints). Connection weights represent the sign and strength of the relations and are bi-directional to permit cognitions to mutually influence each other. External inputs to units represent influences from the environment, while internal constraints [e.g., the biases in Shultz and Lepper’s (2001) model] involve only relations among the elements. Parallel constraint satisfaction is then achieved by algorithms for updating the activations of interlinked units. To illustrate, Holyoak and Thagard’s (1997) multiconstraint analogy theory postulates three basic constraints: similarity, structure, and purpose. People’s use of analogy is guided by these constraints whose constant interplay encourages coherence; the resolution of local contradictions between constraints and the movement toward a satisfying (internally coherent) compromise. In these spreading activation networks, coherence is a state where similarly implicated inferences are similarly activated.

One possible model structure is sketched in Fig. 1. The structure has four key components: excitatory and inhibitory inputs (signals), an integration procedure (Σ), a transfer function (TF), and an output that in turn may influence the inputs. Clearly, the above depiction resembles a neurode, the central building block of neural networks. An excitatory input provides supporting (positive) evidence for a belief, e.g., the guilt of a suspect, whereas an inhibitory input provides contrary (negative) evidence. These inputs are weighted and integrated. The above sketch––borrowing again from neural networks––suggests a simple additive procedure. If the sum is positive, we lean towards considering the suspect guilty, whereas a negative sum supports the opposite opinion. The TF tempers the neurode’s output. For example, a sigmoidal function would damper weak (ambiguous) evidence, whereas a step-function would require a critical amount of evidence for the neurode to provide any directional output. The model then suggests that the output in turn would influence the various inputs. This process may continue until sufficient differentiation (cf. Svenson, 1992) has been generated between the competing beliefs.

Svenson, 1992, Svenson, 1996, Svenson, 1999 differentiation and consolidation theory of decision making models human decision making as an active, creative process in which one alternative gradually is sufficiently differentiated from other available alternatives. The theory then predicts consolidation processes that work to affirm the chosen alternative’s superiority. The greater the degree of differentiation favoring a chosen alternative, the smaller the risk of post-decision regret or decision reversal. Both the pre- and the post-decision phases involve negotiable decision rules, facts, and attractiveness representation (created when a decision maker’s value system is mapped onto a decision problem) and structuring. However, the decision maker may not be conscious of these underlying processes, and may view her/himself as more consistent and less revisionist than s/he actually is. Constraint satisfaction models may prove to be useful operationalizations of both differentiation and consolidation processes [cf. Shultz et al., 1992a, Shultz et al., 1992b Shultz and Lepper’s (1996) model of cognitive dissonance phenomena and Shultz and Lepper’s (2001) belief perseverance model].

Holyoak and Simon (1999) demonstrate that coherence can be achieved in a case characterized by significant ambiguity, that the pressure to achieve coherence guides the decision making (i.e., the pre-decision) process itself (not just the post-decision phase), that coherence-based shifts in beliefs and attitudes trigger correlated shifts in memory, and that the impact of spreading coherence can extend through intermediate inferences to produce remote changes in beliefs. For further background on constraint satisfaction type models see McClelland and Rumelhart (1981), Lundberg (1988), Thagard (1989), Thagard and Millgram (1995), Kunda and Thagard (1996), and Read and Miller (1998).

It is also important to connect the present experimentation with constraint–satisfaction type models to the role of associations and imagery in decision making. Slovic et al. (1989), for example, have applied the method of continued associations (Szalay and Deese, 1978) to demonstrate the relationship between environmental imagery and choice behavior. On a related note, Holyoak and Simon (1999) illuminate the important connection between constraint satisfaction and analogical reasoning. For example, there is reason to believe that inter-analogy competition is especially important in argumentation. In debates about the merits of NATO intervention in Kosovo, the two sides did not argue about details of a mapping (how to label the leaders of the Yugoslav government) but, rather about what overall analogy was most appropriate. A person whose reasoning was guided by a “Rwanda” or a “Holocaust” analogy, is likely to be much less troubled by a NATO intervention, than is a person whose reasoning is guided by a “meddling in internal affairs” (à la Afghanistan, Vietnam, Chile, and Nicaragua) analog. For further insights into the use of analogy in human thinking see Holyoak and Thagard (1997).

In this paper, the constraint satisfaction predictions are tested and illustrated in complex and ambiguous judgment environments. Participants in a first experiment were provided information about an auditing scenario, including a balanced set of key conditions (three supporting a favorable audit report and three supporting an unfavorable report), were asked to state the strength of each condition’s support for either report, and were then asked to make a set of judgments (as jurors) about whether an auditing firm was negligent or not. Participants in a second experiment were provided information about a missing-person/murder case based on a comprehensive summary in the New York Times (Flynn and Bagli, 2001), from where 12 key issues were isolated. The participants were asked to rate the 12 pieces of evidence as to whether they provide support for charging a suspect or not, and subsequently whether they provide support for finding the charged person guilty or not. Each experiment contains three stages––a pre-judgment, a leaning, and a verdict stage––and condition/evidence strength measurements were made in each stage. In addition, factor analysis is utilized to capture the structures that emerge as the participants progress through the judgment process, and discriminant analysis to test the predictive strength of the emerging cognitive structures.

It is hypothesized that inference assessments increasingly will spread apart (even when no further information is provided), with those supporting a not negligent/not guilty verdict moving in one direction and those supporting a negligent/guilty verdict moving in the opposite direction. Also, if a constraint satisfaction process is at work, one can expect a gradual strengthening of the correlations between the decision variables and the verdict, as well as a gradual reduction in the number of negative correlations. Also, the confidence that participants report with respect to their judgments is expected to increase as the decision emerges.

Further, it is hypothesized that factor analysis will reveal a structure that becomes consecutively simpler with the number of significant factors decreasing as the participants get closer to a decision (have generated more sense). Finally, it is hypothesized that a significant number of judgments can be predicted utilizing discriminant analysis, and that this predictability increases as the participants progress from stage to stage in the judgment process.