Cognitive Defeasible Reasoning: the Extent to Which Forms of Defeasible Reasoning Correspond with Human Reasoning

Abstract

Classical logic forms the basis of knowledge representation and reasoning in AI. In the real world, however, classical logic alone is insufficient to describe the reasoning behaviour of human beings. It lacks the flexibility so characteristically required of reasoning under uncertainty, reasoning under incomplete information and reasoning with new information, as humans must. In response, non-classical extensions to propositional logic have been formulated, to provide non-monotonicity. It has been shown in previous studies that human reasoning exhibits non-monotonicity. This work is the product of merging three independent studies, each one focusing on a different formalism for non-monotonic reasoning: KLM defeasible reasoning, AGM belief revision and KM belief update. We investigate, for each of the postulates propounded to characterise these logic forms, the extent to which they have correspondence with human reasoners. We do this via three respective experiments and present each of the postulates in concrete and abstract form. We discuss related work, our experiment design, testing and evaluation, and report on the results from our experiments. We find evidence to believe that 1 out of 5 KLM defeasible reasoning postulates, 3 out of 8 AGM belief revision postulates and 4 out of 8 KM belief update postulates conform in both the concrete and abstract case. For each experiment, we performed an additional investigation. In the experiments of KLM defeasible reasoning and AGM belief revision, we analyse the explanations given by participants to determine whether the postulates have a normative or descriptive relationship with human reasoning. We find evidence that suggests, overall, KLM defeasible reasoning has a normative relationship with human reasoning while AGM belief revision has a descriptive relationship with human reasoning. In the experiment of KM belief update, we discuss counter-examples to the KM postulates.

Supported by Centre for Artificial Intelligence Research (CAIR).

A Appendix: Supplementary Information

1.1 A.1 External Resources

We have created a GitHub repository which contains additional resources for this project. In this repository, we include our survey questions, our raw data and the codebooks used for our data analysis. As mentioned in the abstract, this work is the product of merging three independent papers: one each for KLM [13] defeasible reasoning, AGM belief revision [1] and KM belief update [11]. These independent papers are also included in the GitHub repository. The GitHub repository can be accessed by clicking here. In addition, a summary of our project work is also showcased on our project website which can be viewed by clicking here.

1.2 A.2 Defeasible Reasoning

KLM Postulates. Table 1 presents the KLM postulates. For ease of comparison, we present the postulates translated in a manner similar to [27]. We write $C_n(S)$ to represent the smallest set closed under classical consequence containing all sentences in S, and $D_C(S)$ to represent the resulting set if defeasible consequence is used instead. $D_C(S)$ is assumed defined only for finite S. $C_n(\alpha )$ is an abbreviation for $C_n(\{\alpha \})$, and $D_C(\alpha )$ is an abbreviation for $D_C(\{\alpha \})$.

Table 1. KLM postulates

Reflexivity states that if a formula is satisfied, it follows that the formula can be a consequence of itself. Left Logical Equivalence states that logically equivalent formulas have the same consequences. Right Weakening expresses the fact that one should accept as plausible consequences all that is logically implied by what one thinks are plausible consequences. And expresses the fact that the conjunction of two plausible consequences is a plausible consequence. Or says that any formula that is, separately, a plausible consequence of two different formulas, should also be a plausible consequence of their disjunction. Cautious Monotonicity expresses the fact that learning a new fact, the truth of which could have been plausibly concluded, should not invalidate previous conclusions.

Additional Postulates. Table 2 presents additional defeasible reasoning postulates. Cut expresses the fact that one may, in his way towards a plausible conclusion, first add an hypothesis to the facts he knows to be true and prove the plausibility of his conclusion from this enlarged set of facts and then deduce (plausibly) this added hypothesis from the facts. Rational Monotonicity expresses the fact that only additional information, the negation of which was expected, should force us to withdraw plausible conclusions previously drawn. Transitivity expresses that if the second fact is a plausible consequence of the first and the third fact is a plausible consequence of the second, then the third fact is also a plausible consequence of the first fact. Contraposition allows the converse of the original proposition to be inferred, by the negation of terms and changing their order.

Table 2. Additional postulates

1.3 A.3 Belief Revision

AGM Postulates. Table 3 presents the AGM postulates. $K*\alpha$ is the sentence representing the knowledge base after revising the knowledge base K with $\alpha$. We assume that K is a set that is closed under classical deductive consequence.

Table 3. AGM postulates

Closure implies logical omniscience on the part of the ideal agent or reasoner, including after revision of their belief set. Success expresses that the new information should always be part of the new belief set. Inclusion and Vacuity are motivated by the principle of minimum change. Together, they express that in the case of information $\alpha$, consistent with belief set or knowledge base K, belief revision involves performing expansion on K by $\alpha$ i.e. none of the original beliefs need to be withdrawn. Consistency expresses that the agent should prioritise consistency, where the only acceptable case of not doing so is if the new information, $\alpha$, is inherently inconsistent - in which case, success overrules consistency. Extensionality effectively expresses that the content i.e. the belief represented, and not the syntax, affects the revision process, in that logically equivalent sentences or beliefs will cause logically equivalent changes to the belief set. Super-expansion and sub-expansion is motivated by the principle of minimal change. Together, they express that for two propositions $\alpha$ and $\phi$, if in revising belief set K by $\alpha$ one obtains belief set K’ consistent with $\phi$, then to obtain the effect of revising K with $\alpha \wedge \phi$, simply perform expansion on K’ with $\phi$. In short, $K * (\alpha \wedge \phi ) = (K * \alpha ) + \phi$.

Table 4. KM postulates

1.4 A.4 Belief Update

KM Postulates. Table 4 presents the KM postulates. For ease of comparison, the postulates have been rephrased as in the AGM paradigm [22]. We use $\diamond$ to represent the update operator. U1 states that updating with the new fact must ensure that the new fact is a consequence of the update. U2 states that updating on a fact that could in principle be already known has no effect. U3 states the reasonable requirement that we cannot lapse into impossibility unless we either start with it, or are directly confronted by it. U4 requires that syntax is irrelevant to the results of an update. U5 says that first updating on $\alpha$ then simply adding the new information $\gamma$ is at least as strong (i.e. entails) as updating on the conjunction of $\alpha$ and $\gamma$. U6 states that if updating on $\alpha _1$ entails $\alpha _2$ and if updating on $\alpha _2$ entails $\alpha _1$, then the effect of updating on either is equivalent. U7 applies only to complete knowledge bases, that is knowledge bases with a single model. If some situation arises from updating a complete K on $\alpha$ and it also results from updating that K from $\phi$ then it must also arise from updating that K on $\alpha \vee \phi$. U8 is the disjunction rule. U*9 is not necessary in the propositional formulation of the postulates and is listed for completeness. It was not tested in the survey.

Fig. 1.

Hit rate (%) for defeasible reasoning postulates

Fig. 2.

Hit rate (%) for belief revision postulates

1.5 A.5 Results

In Fig. 1, we show the Hit Rate (%) for each defeasible reasoning postulate. In Fig. 2, we show the Hit Rate (%) for each belief revision postulate. In Fig. 3, we show the Hit Rate (%) for each belief update postulate.

Fig. 3.

Hit rate (%) for belief update postulates