Logical Inference as Cost Minimization in Vector Spaces

Abstract

We propose a differentiable framework for logic program inference as a step toward realizing flexible and scalable logical inference. The basic idea is to replace symbolic search appearing in logical inference by the minimization of a cost function ${\mathbf{J}}$ in a continuous space. ${\mathbf{J}}$ is made up of matrix (tensor), Frobenius norm and non-linear functions just like neural networks and specifically designed for each task (relation abduction, answer set computation, etc) in such a way that ${\mathbf{J}} ( {\mathbf{X}} ) \ge 0$ and ${\mathbf{J}} ( {\mathbf{X}} ) = 0$ holds if-and-only-if ${\mathbf{X}}$ is a 0–1 tensor representing a solution for the task. We compute the minimizer X of ${\mathbf{J}}$ giving ${\mathbf{J}} ( {\mathbf{X}} ) = 0$ by gradient descent or Newton’s method. Using artificial data and real data, we empirically show the potential of our approach by a variety of tasks including abduction, random SAT, rule refinement and probabilistic modeling based on answer set (supported model) sampling.

Appendix

Here we describe how the Jacobian ${\mathbf{J}} _\mathbf{a}^{abd}$ in Sect. 1 is derived. Recall that our cost function (1) is

$$\begin{aligned} {\mathbf{J}} ^{abd}( {\mathbf{X}} )= & {} \frac{1}{2} \{ \Vert \text{ min}_1( {\mathbf{R}} _1 {\mathbf{X}} ) - {\mathbf{R}} _3 \Vert _F^2 + \ell \cdot \Vert {\mathbf{X}} \odot (\mathbbm {1}- {\mathbf{X}} ) \Vert _F^2 \}. \end{aligned}$$

First we introduce a dot product for two matrices ${\mathbf{X}}$ and ${\mathbf{Y}}$ by $( {\mathbf{X}} \bullet {\mathbf{Y}} ) = \sum _{ij} {\mathbf{X}} _{ij} {\mathbf{Y}} _{ij}$. Then $\Vert X \Vert _F^2 =( {\mathbf{X}} \bullet {\mathbf{X}} )$ holds. Also $(( {\mathbf{X}} {\mathbf{Z}} ) \bullet {\mathbf{Y}} ) = ( {\mathbf{Z}} \bullet ( {\mathbf{X}} ^T {\mathbf{Y}} ))$ and $(( {\mathbf{X}} \odot {\mathbf{Z}} ) \bullet {\mathbf{Y}} ) = ( {\mathbf{Z}} \bullet ( {\mathbf{X}} \odot {\mathbf{Y}} ))$ hold. Let ${\mathbf{X}} _{pq}$ be the (p, q) element of a matrix ${\mathbf{X}}$ and ${\mathbf{I}} _{pq}$ a zero matrix except the (p, q) element which is one. We also put ${\mathbf{C}} = {\mathbf{R}} _1 {\mathbf{X}}$, ${\mathbf{B}} = \text{ min}_1( {\mathbf{C}} )- {\mathbf{R}} _3$ and use the fact that ${\mathbf{C}} _{\le 1}\odot {\mathbf{B}} = {\mathbf{C}} _{\le 1}\odot ( {\mathbf{C}} - {\mathbf{R}} _3)$ for simplification. Now we have

$$\begin{aligned}&{ \partial {\mathbf{J}} ^{abd}/\partial {\mathbf{X}} _{pq} } \\&\,\, = (( {\mathbf{C}} _{\le 1}\odot ( {\mathbf{R}} _1 {\mathbf{I}} _{pq})) \bullet {\mathbf{B}} ) + \\&\qquad \,\, \ell \cdot ( ( {\mathbf{I}} _{pq} \bullet ( {\mathbf{X}} \odot (\mathbbm {1}- {\mathbf{X}} )\odot (\mathbbm {1}- {\mathbf{X}} ))) - ( {\mathbf{I}} _{pq} \bullet ( {\mathbf{X}} \odot {\mathbf{X}} \odot (\mathbbm {1}- {\mathbf{X}} )))) \\&\,\, = (( {\mathbf{R}} _1 {\mathbf{I}} _{pq}) \bullet ( {\mathbf{C}} _{\le 1}\odot {\mathbf{B}} )) + ( {\mathbf{I}} _{pq} \bullet \ell \cdot ( {\mathbf{X}} \odot (\mathbbm {1}- {\mathbf{X}} )\odot (\mathbbm {1}-2 {\mathbf{X}} ))) \\&\,\, = ( {\mathbf{I}} _{pq} \bullet ( {\mathbf{R}} _1^T( {\mathbf{C}} _{\le 1}\odot {\mathbf{B}} ))) + ( {\mathbf{I}} _{pq} \bullet \ell \cdot ( {\mathbf{X}} \odot (\mathbbm {1}- {\mathbf{X}} )\odot (\mathbbm {1}-2 {\mathbf{X}} ))) \\&\,\, = ( {\mathbf{I}} _{pq} \bullet ( {\mathbf{R}} _1^T( {\mathbf{C}} _{\le 1}\odot {\mathbf{B}} ) + \ell \cdot ( {\mathbf{X}} \odot (\mathbbm {1}- {\mathbf{X}} )\odot (\mathbbm {1}-2 {\mathbf{X}} )))) \\&\,\, = ( {\mathbf{I}} _{pq} \bullet ( {\mathbf{R}} _1^T( {\mathbf{C}} _{\le 1}\odot ( {\mathbf{C}} - {\mathbf{R}} _3)) + \ell \cdot ( {\mathbf{X}} \odot (\mathbbm {1}- {\mathbf{X}} )\odot (\mathbbm {1}-2 {\mathbf{X}} )))) \nonumber \end{aligned}$$

Since this holds for any (p, q), we reach the Jacobian ${\mathbf{J}} _\mathbf{a}^{abd}$ (5):

$$\begin{aligned} {\mathbf{J}} _\mathbf{a}^{abd}= & {} \partial {\mathbf{J}} ^{abd}/\partial {\mathbf{X}} \\= & {} {\mathbf{R}} _1^T(( {\mathbf{R}} _1 {\mathbf{X}} )_{\le 1}\odot ( {\mathbf{R}} _1 {\mathbf{X}} - {\mathbf{R}} _3)) + \ell \cdot ( {\mathbf{X}} \odot (\mathbbm {1}- {\mathbf{X}} ) \odot (\mathbbm {1}-2 {\mathbf{X}} )) \nonumber \end{aligned}$$