Probing LLMs for Logical Reasoning

Abstract

Recently, the question of what types of computation and cognition large language models (LLMs) are capable of has received increasing attention. With models clearly capable of convincingly faking true reasoning behavior, the question of whether they are also capable of real reasoning—and how the difference should be defined—becomes increasingly vexed. Here we introduce a new tool, Logic Tensor Probes (LTP), that may help to shed light on the problem. Logic Tensor Networks (LTN) serve as a neural symbolic framework designed for differentiable fuzzy logics. Using a pretrained LLM with frozen weights, an LTP uses the LTN framework as a diagnostic tool. This allows for the detection and localization of logical deductions within LLMs, enabling the use of first-order logic as a versatile modeling language for investigating the internal mechanisms of LLMs. The LTP can make deductions from basic assertions, and track if the model makes the same deductions from the natural language equivalent, and if so, where in the model this happens. We validate our approach through proof-of-concept experiments on hand-crafted knowledge bases derived from WordNet and on smaller samples from FrameNet.

F. Manigrasso and S. Schouten—Equal contribution.

Appendices

A Hyperparameters

For all experiments, the aggregation function parameter in the knowledge base defined in Eq. 27 is set to $p_\forall =2$. We implemented the probes in PyTorch, using the LTNtorch library [8] and training was done using the Adam optimizer.

1.1 A.1 WordNet

To train the WordNet probes, we adopted a learning rate of 1e-5 and trained our architecture for 100 epochs. The knowledge base during training included batches of 128 randomly chosen sentences, both positive and negative. Experiments were done on a single Nvidia 2080 Ti GPU.

1.2 A.2 FrameNet

To train the FrameNet probes, we adopted a learning rate of 1e-3 and trained our architecture for 15 epochs and a batch size of 128. The experiments were performed on a single Nvidia 3090 GPU.

B WordNet LTN

1.1 B.1 Variables, Predicates and Axioms

In this subsection, we present the foundations of variables and predicates with the definition of the knowledge base $\mathcal {K}$.

Grounding Variables and their corresponding domains are grounded as follows:

$$\begin{aligned} &\mathcal {G}(n) = \mathbb {N}^{N}, \,\, \mathcal {G}(v) = \mathbb {N}^{V}, \,\, \mathcal {G}(g) = \mathbb {N}^{G}, \,\, \mathcal {G}(q) = \mathbb {N}^{Q}, \,\, \nonumber \\ &\mathcal {G}(h) = \mathbb {N}^{H}, \,\, \mathcal {G}(l) = \mathbb {N}^{L}, \,\, \mathcal {G}(s) = \mathbb {R}^{B \times D}, \,\, \nonumber \\ &\mathcal {G}(sub),\, \mathcal {G}(act),\, \mathcal {G}(obj)= \mathbb {R}^{T \times m} \,\, \nonumber \\ \end{aligned}$$

(11)

where g, q, n, a, h represent the class labels belonging to a set of classes G, macroclasses Q, names N, actions A, and habitats H sets. Additionally, l represents the label used to distinguish different sections of the sentence, such as subject and object. The variables B and D are used in the definition of s to represent the dimensions of the feature space, where B is the batch size and D is the dimensionality of the features.

On the other hand sub, act and obj retrieved with the attention model of an entire sentence s, are grounded into a feature space. obj can be translated into macroclass when the sentence is in the form isOfName isOfAction isOfMacroclass instead of isOfName isOfAction isofObject. Within the FOL language, different predicates are defined: isOfName(sub, n), isOfAction(act, a), isOfClass(obj, c) with the aim of categorizing the sentence components, livesInHabitat(obj, h) and isOfMacroclass(obj, q) allow classifying objects respect to habitat and macroclass and finally isSubject(sub, l) with isObject(obj, l) to predict if a sentence is logically true by investigating whether the subject and its complement are inserted in a correct order.

The predicate groundings $\mathcal {G}(\texttt {isOfName})$, $\mathcal {G}(\texttt {isOfAction})$, $\mathcal {G}(\texttt {isOfClass})$ are formed by the similarity between the input features and the corresponding trainable class vectors with an MLP $p_1$ with softmax activation function, as the classes are mutually exclusive.

$$\begin{aligned} \mathcal {G}(\texttt {isOfName}): & \text {sub},n \rightarrow {n}^T p_1(\mathcal {G}(sub),n) \end{aligned}$$

(12)

$$\begin{aligned} \mathcal {G}(\texttt {isOfAction}): & \text {act},v \rightarrow {v}^T p_1(\mathcal {G}(act),v) \end{aligned}$$

(13)

$$\begin{aligned} \mathcal {G}(\texttt {isOfClass}): & \text {obj},g \rightarrow {g}^T p_1(\mathcal {G}(obj),g) \end{aligned}$$

(14)

Likewise, the $\mathcal {G}(\texttt {isOfMacroclass}), \mathcal {G}(\texttt {livesInHabitat})$ predicates are grounded with two simple MLP layers with a softmax activation function $p_2, p_3$:

$$\begin{aligned} \mathcal {G}(\texttt {isOfMacroclass}): & \text {obj},q \rightarrow {v}^T p_2(\mathcal {G}(obj),q) \end{aligned}$$

(15)

$$\begin{aligned} \mathcal {G}(\texttt {isOfMacroclass}): & \text {obj},q \rightarrow {v}^T p_2(\mathcal {G}(obj),q) \end{aligned}$$

(16)

$$\begin{aligned} \mathcal {G}(\texttt {livesInHabitat}): & \text {obj},h \rightarrow {g}^T p_3(\mathcal {G}(obj),h) \end{aligned}$$

(17)

Finally, for the grounding of isSubject and isObject two parametric similarity functions based on two simple MLP layers with softmax activation function $p_4, p_5$:

$$\begin{aligned} \mathcal {G}(\texttt {isSubject}): & \text {sub},l \rightarrow {l}^Tp_4(\mathcal {G}(sub),l) \end{aligned}$$

(18)

$$\begin{aligned} \mathcal {G}(\texttt {isObject}): & \text {obj},l \rightarrow {l}^Tp_5(\mathcal {G}(obj),l) \end{aligned}$$

(19)

where l allows us to distinguish the different sections (subject, action, object) of the sentence.

1.2 B.2 Learning from Labeled Examples

The following is each axiom representing the labeled examples used during training:

$$\begin{aligned} \phi _{1} &= \forall \text {Diag}(sub,n) (\texttt {isOfName}(sub,n)) \end{aligned}$$

(20)

$$\begin{aligned} \phi _{2} &= \forall \text {Diag}(act,v) (\texttt {isOfAction}(act,v)) \end{aligned}$$

(21)

$$\begin{aligned} \phi _{3} &= \forall \text {Diag}(obj,g) (\texttt {isOfClass}(obj,g)) \end{aligned}$$

(22)

$$\begin{aligned} \phi _{4} &= \forall \text {Diag}(obj,q) (\texttt {isOfMacroclass}(obj,q)) \end{aligned}$$

(23)

$$\begin{aligned} \phi _{5} &= \forall \text {Diag}(obj,h)(\texttt {livesInHabitat}(obj,h)) \end{aligned}$$

(24)

$$\begin{aligned} \phi _{6} &= \forall \text {Diag}(sub,l) (\texttt {isSubject}(sub,l)) \end{aligned}$$

(25)

$$\begin{aligned} \phi _{7} &= \forall \text {Diag}(obj,l)(\texttt {isObject}(obj,l)) \end{aligned}$$

(26)

1.3 B.3 Grounding Logical Connectives and Aggregators

The knowledge base $\mathcal {K}$ includes a collection of axioms, with a view to reconstructing logical connectives and aggregators into Real Logic. Gradient descent is adopted to train the probe to maximize the satisfiability of the problem. In this configuration, we make use of the standard negation $\lnot$ defined as $N_S(a) = 1 - a$, and the Reichenbach implication $\rightarrow$ defined as $I_R(a, b) = 1 - a + ab$, where a and b are both truth values within the range of [0, 1]. To approximate the universal quantifier $\forall$, we use the generalized mean with respect to the error, denoted as $A_{pME}$, as described in [2, 34]. Given a set of n truth values $a_1, \ldots , a_n \in [0, 1]$:

$$\begin{aligned} \forall : A_{p M E} & \left( a_1, \ldots , a_n\right) = \nonumber \\ & 1 - \left( \frac{1}{n} \sum _{i=1}^n \left( 1-a_i\right) ^{p_{\forall }}\right) ^{\frac{1}{p_{\forall }}} p_{\forall } \geqslant 1 \end{aligned}$$

(27)

$A_{p M E}$ is a measure of how much, on average, truth values $a_i$ deviate from the true value of 1. Further details on the role of $p_{\forall }$ can be found in [2].

1.4 B.4 Knowledge Base for Inference Time

The knowledge base used at testing time is shown below:

$$\begin{aligned} \phi _{8} &= \forall \text {Diag}(sub,n,act,v,obj,g,l) \nonumber \\ & \quad (\texttt {isOfName}(sub,n) \wedge \texttt {isSubject}(sub,l)) \wedge \texttt {isOfAction}(act,v) \nonumber \\ &\wedge (\texttt {isOfClass}(obj,g) \wedge \texttt {isObject}(obj,l)) & \end{aligned}$$

(28)

$$\begin{aligned} \phi _{9} &= \forall \text {Diag}(sub_{1},n,act_{1},v,obj_{1},g,l,sub_{2},act_{2},obj_{2}) \nonumber \\ &\quad (\texttt {isOfName}(sub_{1},n) \wedge \texttt {isSubject}(sub_{1},l)) \wedge \texttt {isOfAction}(act_{1},v) \nonumber \\ &\wedge (\texttt {isOfClass}(obj_{1},g) \wedge \texttt {isObject}(obj_{1},l)) \nonumber \\ &\wedge (\texttt {isOfName}(sub_{2},n) \wedge \texttt {isSubject}(sub_{2},l)) \wedge \nonumber \\ &(\texttt {isOfMacroclass}(obj_{2},q) \wedge \texttt {isObject}(obj_{2},l)) & \end{aligned}$$

(29)

$$\begin{aligned} \phi _{10} &= \forall \text {Diag}(sub_{1},n,act_{1},v,obj_{1},g,l,sub_{2},obj_{2}) \nonumber \\ &\quad (\texttt {isOfName}(sub_{1},n) \wedge \texttt {isSubject}(sub_{1},l)) \wedge \texttt {isOfAction}(act_{1},v) \nonumber \\ &\wedge (\texttt {isOfClass}(obj_{1},g) \wedge \texttt {isObject}(obj_{1},l)) \nonumber \\ &\wedge ((\texttt {isOfName}(sub_{2},n) \wedge \texttt {isSubject}(sub_{2},l))) \nonumber \\ &\wedge (\texttt {livesInHabitat}(obj_{2},h)) \wedge \texttt {isObject}(obj_{2},l)) & \end{aligned}$$

(30)

The subscript introduced in the annotation allows us to identify whether the extracted representations come from the previous sentence or the following one. Axiom $\phi _{8}$ represent the degree of truthiness of a sentence like “Gordon is a German Shepherd”, axiom $\phi _{9}$ express “Gordon is a German Shepard and Gordon is Carnivorous”, axiom $\phi _{10}$ highlights “Gordon is a German Shepard and Gordon lives in a domestic environment”.

C FrameNet Data Details

1.1 C.1 Frame-Frame Implications

The following is the complete list of all implications included with the FrameNet experiments. For more information on the meaning of each of the frames, please see https://framenet.icsi.berkeley.edu/frameIndex.