Pairwise saturations in inductive logic programming

Abstract

One of the main issues when using inductive logic programming (ILP) in practice remain the long running times that are needed by ILP systems to induce the hypothesis. We explore the possibility of reducing the induction running times of systems that use asymmetric relative minimal generalisation (ARMG) by analysing the bottom clauses of examples that serve as inputs into the generalisation operator. Using the fact that the ARMG covers all of the examples and that it is a subset of the variabilization of one of the examples, we identify literals that cannot appear in the ARMG and remove them prior to computing the generalisation. We apply this procedure to the ProGolem ILP system and test its performance on several real world data sets. The experimental results show an average speedup of $36\,\%$ compared to the base ProGolem system and $12\,\%$ compared to ProGolem extended with caching, both without a decrease in the accuracy of the produced hypotheses. We also observe that the gain from using the proposed method varies greatly, depending on the structure of the data set.

Appendix: Proof of Theorem 1

Proof

Literal $L \in \bot _1$ has no compatible literal in $\bot _2$ if for each $L' \in \bot _2$ at least one of the conditions for compatibility in Definition 8 fails to hold.

Ad 8: If $var(\bot _1)$ contains a literal $L = P(A_1, \dots A_n)$ for which no literal $L' = P'(B_1, \dots B_m) \in \bot _2$ exists such that $P = P'$ and $n = m$, L cannot appear in $H = ARMG(e_2\vert {}e_1)$. Remember that the ARMG must cover $e_2$, implying that for some substitution $\Theta _2$, $H\Theta _2 \subseteq \bot _2$. Since substitutions do not affect the predicate symbols and their arity, there can be no such $\Theta _2$ that $L\Theta _2 \in \bot _2$, and consequently $(H \cup L) \Theta _2 \not \subseteq \bot _2$—meaning that no H containing L can be part of an $ARMG(e_2\vert {}e_1)$.

Ad 8: Let $var(\bot _1)$ contain a literal $L = P(\dots c_1, \dots c_{n\#})$ with mode declaration $P(\dots \#_1, \dots \#_{n\#})$. Without loss of generality we assume that $\bot _2$ contains a single literal of the form $L' = P(\dots {} c'_1, \dots c'_{n\#})$. Since substitutions do not affect constants in the hypothesis, there can be no $\Theta _2$ such that $H\Theta _2{} \subseteq \bot _2$ if $L \in H$ and $c_i \ne c'_i$ for some i.

Ad 8: In order for the intersection of instantiation sets IS[a] and $IS[a']$ to be empty, each pair of instantiations $\langle {}P/i/l\rangle \in IS[a]$ and $\langle {}P'/i'/l'\rangle \in IS[a']$ must disagree in at least one value (predicate symbol (case 1), argument position (case 2) or layer (case 3)). Without loss of generality we assume that IS[a] and $IS[a']$ each contain a single entry $\langle {}P/i/l\rangle$ and $\langle {}P'/i'/l'\rangle$ respectively.

Let $L = T(a_{1}\dots {}a_{n+}, b_{1}\dots {}b_{n-}) \in \bot _1$ where $a_i$ stands for input, $b_i$ for output arguments, be the literal we are currently considering for inclusion in the ARMG. Let $L' = T(a'_{1}\dots {}a'_{n+}, b'_{1}\dots {}b'_{n-})$ be a literal present in $\bot _2$.

• Case 1: $\mathbf {P \ne P'}$ Let the instantiation set of some $a_i$ be $IS[a_i] = \{\langle {}P/i/l\rangle \}$, and of $a'_i$ be $IS[a'_i] = \{\langle {}P'/i/l\rangle \}$, where $P\ne {}P'$. By construction of the instantiation sets each entry denotes either an input argument of the head of the clause or an output argument of a literal in its body. The bottom clauses of the two examples must then be of the following form:

  $\bot _1 = \dots P(\dots a_i \dots ), T(a_1, \dots a_i \dots a_{n+}, b_1, \dots b_{n-}), \dots$

  $\bot _2 = \dots P'(\dots a'_i \dots ), T(a'_1, \dots a'_i \dots a'_{n+}, b'_1, \dots b'_{n-}), \dots$

  $var(\bot _1) = \dots P(\dots A_i \dots ), T(A_1, \dots A_i \dots A_{n+}, B_1, \dots B_{n-}), \dots$

  If $L_v = T(A_1, \dots A_i \dots A_{n+}, B_1, \dots B_{n-})$ appears in $H = ARMG(e_2\vert {}e_1)$, then so must $P(\dots A_i \dots )$, since it is the only way to instantiate $A_i$ – otherwise H would not be head-connected. It follows that $A_i/a'_i$ must be in $\Theta$ to instantiate $L_v$ to $L'$. The substitution $A_i/a'_i$ also affects $P(\dots A_i \dots )\Theta = P(\dots a'_i \dots )$ – but, if a literal of the form $P(\dots a'_i \dots )$ would be present in $\bot _2$, $\{\langle {}P/i/l'\rangle \}$ would be present in $IS[a']$. Since this $IS[a']$ has a single element $\langle {}P'/i/l\rangle$ we have reached a contradiction. The predicate symbols thus need to be equal.

• Case 2: $\mathbf {i \ne i'}$

  Let the instantiation set of some $a_i$ be $IS[a_i] = \{\langle {}P/i/l\rangle \}$, and of $a'_{i'}$ be $IS[a'_{i'}] = \{\langle {}P/i'/l\rangle \}$, where $i \ne i'$, and $a_i$ and $a'_{i'}$ appear at the same input position in literals L and $L'$ respectively. Without loss of generality we assume $i < i'$.

  The bottom clauses of the two examples are of the following form:

  $\bot _1 = \dots P(\dots a_i, \dots a_{i'} \dots ), T(a_1, \dots a_i \dots a_{n+}, b_1, \dots b_{n-}), \dots$

  $\bot _2 = \dots P'(\dots a'_i \dots a'_{i'} \dots ), T(a'_1, \dots a'_{i'} \dots a'_{n+}, b'_1, \dots b'_{n-}), \dots$

  $var(\bot _1) = \dots P(\dots A_i \dots A_{i'} \dots ), T(A_1, \dots A_i \dots A_{n+}, B_1, \dots B_{n-}), \dots$

  We follow the same line of reasoning as in the previous case. If the ARMG contains literal $L_v = T(A_1, \dots A_i \dots A_{n+}, B_1, \dots B_{n-})$, then there must exist some $\Theta _2$ such that $L_v\Theta _2 \in \bot _2$ – consequently $A_i/a'_{i'} \in \Theta _2$. Applying $\Theta _2$ to the ARMG would also cause $P(\dots A_i \dots A_i' \dots )\Theta _2 = P(\dots a'_{i'} \dots A_{i'} \dots {})$. However, if a literal of this form would exist in $\bot _2$, then the instantiation set of $a'_{i'}$ would include the entry $\{\langle {}P/i/l\rangle \}$. We have reached a contradiction since either $i = i'$ or the instantiation set of $a'_{i'}$ contains more than one element. Therefore the argument positions must match.

• Case 3: $\mathbf {l \ne l'}$

  Let $\mathcal {L}_1(\bot _i)$ to $\mathcal {L}_{nl}(\bot _i)$ denote the sets of literals occurring at layers 1 through nl of $\bot _i$. Without loss of generality we assume that the instantiation sets of all appearing values contain a single entry³.

  Let our L and $L'$ be defined as above, with $L_v$ being the variabilization of L. For some input argument position i of literals L and $L'$, let the instantiation sets of $a_i$ and $a'_i$ be $IS[a_i] = \{\langle {}P_1/k/l_1\rangle \}$ and $IS[a'_i] = \{\langle {}P_1/k/l'_1\rangle \}$ such that $l_1 \ne l'_1$.

  There must exist some literal $L_{1}$ in $\mathcal {L}_l(\bot _1)$ with predicate symbol $P_1$ that has $a_i$ as its k-th output argument. Similarly, there is a literal $L'_{1}$ in $\mathcal {L}_{l'}(\bot _2)$ with predicate symbol $P_1$ that has $a'_i$ as its k-th output argument.

  Let us examine what the substitution $\Theta _2: H \Theta _2 \subseteq \bot _2$ looks like if $L_v$ is included in the ARMG H. As before, $\Theta _2$ must include $A_i/a'_i$.

  In order for the ARMG to be head-connected, the variabilization of $L_{1} = P_1(a_{11}, \dots {} a_{1n^+}, \dots a_i\dots )$ needs to be part of it (since it is the only way to instantiate $a_i$). When applying $\Theta _2$ to $L_{1}$ its k-th output argument becomes $a'_i$. Since the only literal in $\bot _2$ in which $a'_i$ appears as the k-th argument is $L'_{1} = P_1(a'_{11}, \dots {} a'_{1n^+}, \dots a'_i \dots )$, $\Theta _2$ must include substitutions for all of its input arguments $A_{11}/a'_{11}\dots {}A_{1n^+}/a'_{1n^+}$.

  Since $l_1 \ne l'_1$, there are two scenarios: either $l_1 < l'_1$ or $l_1 > l'_1$.

• Subcase 1: $l_1 > l'_1$ Because $L_{1}$ appears on layer $l_1$ at least one of its input variables $a_{1i}$ has to be instantiated at layer $l_2 = l_1 - 1$, causing $IS[a_{1i}] = \{\langle {}P_{2}/k_{2}/l_2\rangle \}$. The corresponding value in $L'_{1}$ is $a'_{1i}$ with its instantiation set $IS[a'_{1i}] = \{\langle {}P'_{2}/k'_{2}/l'_2\rangle \}$ with $l_2' < l_2$. Again we can find the literals $L_{2} \in \mathcal {L}_{l_2}(\bot _1)$, $L_2 = P_2(a_{21}, \dots {} a_{2n^+}, \dots a_{1i} \dots )$ and $L'_{2} \in \mathcal {L}_{l'_2}$, $L'_2 = P'_2(a'_{21}, \dots {} a'_{2n^+}, \dots a'_{1i} \dots )$. We repeat this process, at each iteration j choosing the argument position k in such a way that the layer of the literal that instantiates $a_{jk}$, $L_{j+1}$, decreases by 1 with each step. The layer of the corresponding literal $L'_{j+1}$ in $\bot _2$ decreases by at least 1 in each step. After $s \le l_1'$ steps the corresponding literal $L'_{s}$ is in $\mathcal {L}_0(\bot _2)$ and is equal to the head of the clause. However, $L_s$ is in $\mathcal {L}_{l-s}(\bot _1)$ where $l-s > 0$ – meaning that at least one of its input arguments is instantiated outside of the head of the clause. At this point for some i, $IS[a'_{si}] = \{\langle {}P'_s/i'_s/0\rangle \}$ and $IS[a_{lsi}] = \{\langle {}P_s/i_s/(l-s-1)\rangle \}$, but now no further values are available in $\bot _2$ to which the variables at the inputs of the literal which instantiates $A_{si}$ will bind. This leads to $\Theta _2$ assigning the same value to multiple variables in H, which contradicts our assumption that the instantiation sets all contain a single entry (since each distinct variable in H is instantiated in a distinct way, assigning the same value to multiple variables would cause the instantiation set of that value to contain multiple entries).

• Subcase 2: $l_1' > l_1$ The argument runs along the same lines, only that this time at each step we choose the argument position i that reduces the layer of the literal in $\bot _2$ by 1. After $s \le l_1$ steps, some argument of $L'_s$, $a'_{si}$ is instantiated in layer $s-1$, but $a_{si}$ is instantiated in literal $L_s$ in $\mathcal {L}_0(\bot _1)$ – $L_s$ is the head of the clause (i.e. the value we follow is an input to the head of the clause). Because of the way we chose the variables to follow, at this point the layer at which $L'_s$ appears is equal to $l'_1 - s$. At this point $\Theta _2$ already contains substitutions for all the variables appearing in H, meaning that all the substitutions required to instantiate $L'_s$ must already be present in $\Theta _2$ (if that were not the case, variables that do not yet appear in $\Theta _2$ would need to be present in H) – contradicting the assumption that every value has a single entry in its instantiation set, because in this case $a'_{si}$ must appear both as an input argument of the head and as an output argument of some literal $L'_{s-1} \in \mathcal {L}_{s-1}$, which would cause its instantiation set to contain two entries.

Since we have reached a contradiction in both subcases, we conclude that $l_1$ and $l_1'$ must be equal. $\square$