KRL_Match: knowledge graph objects matching for knowledge representation learning

Abstract

The way of obtaining the embeddings of the knowledge graph objects through modeling with binary classification method from the level of triple structure is coarser in granularity for the existing knowledge representation learning models based on the probability, and the space-time efficiency of negative sampling is lower for the most of the knowledge representation learning models at present. To solve these problems, this paper proposes a knowledge representation learning model KRL_Match, which carries out the knowledge graph objects matching centered on a certain kind of knowledge graph objects (head entity, tail entity, relation), and executes multi-classification learning to determine the true matching and dynamic implicit negative sampling. Specifically, first, we make two classes of the knowledge graph objects of target and source in the same kind of knowledge graph objects matched mutually by their matrix multiplication operation in a knowledge graph batch sample space, which is constructed by random sampling from the universe set of the knowledge graph instance, and the knowledge graph objects matching sample spaces will be implicitly generated meanwhile; then, we measure the matching degree of each matching of the knowledge graph objects by softmax regression multi-classification method in each implicit sample space; finally, we fit the real probability with the matching degree by optimizing the cross-entropy loss based on the local closed world assumption. We conduct the knowledge graph objects matching for the knowledge representation learning inspired by the attention mechanism and firstly create the dynamic implicit negative sampling method in the knowledge representation learning. Experiments show that the KRL_Match model has achieved better performances compared with the baselines: Hits@10 (filter) has increased by 12.2% and 6.1% on benchmarks FB15K and FB15K237 respectively for the entity prediction task, and accuracy has increased by 12.6% on benchmark FB13 for the triple classification task. In addition, space-time efficiency test indicates that the negative sampling of KRL_Match is less 7395.59s and half in time and the storage space separately than TransE’s on benchmark FB15K (BS = 12000).

Appendices

Appendices

1.1 A. Proof

Proof:

Sufficiency:

$\because max\left\{ p\left( {\left\{ o_{k1} \rightarrow o_{k2}^{\prime } \right\} }^{ overall} \right) \right\} = p\left( {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=*} \right)$,

$\therefore \forall \left\{ {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=?} \right\} \in {\left\{ o_{k1} \rightarrow o_{k2}^{\prime } \right\} }^{ overall}, p\left( {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=*} \right) > p\left( {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=?} \right)$;

$\because \forall S_{ batch}, \forall {\left\{ o_{k1} \rightarrow o_{k2}^{\prime } \right\} }^{ batch}, {\left\{ o_{k1} \rightarrow o_{k2}^{\prime } \right\} }^{ batch} \subset {\left\{ o_{k1} \rightarrow o_{k2}^{\prime } \right\} }^{ overall}$,

$\therefore \forall \left\{ {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=?} \right\} \in {\left\{ o_{k1} \rightarrow o_{k2}^{\prime } \right\} }^{ batch}, p\left( {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=*} \right) > p\left( {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=?} \right)$.

Sufficiency is easily proved.

Necessity:

$\because \forall S_{ batch}, max\left\{ p\left( {\left\{ o_{k1} \rightarrow o_{k2}^{\prime } \right\} }^{ batch} \right) \right\} = p\left( {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=*} \right)$,

$\therefore \forall \left\{ {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=?} \right\} \in {\left\{ o_{k1} \rightarrow o_{k2}^{\prime } \right\} }^{batch,0}, p\left( {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=*} \right) > p\left( {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=?} \right)$,

$\forall \left\{ {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=?} \right\} \in {\left\{ o_{k1} \rightarrow o_{k2}^{\prime } \right\} }^{batch,1}, p\left( {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=*} \right) > p\left( {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=?} \right)$,

$\cdots$,

$\forall \left\{ {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=?} \right\} \in {\left\{ o_{k1} \rightarrow o_{k2}^{\prime } \right\} }^{batch,BS-1}, p\left( {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=*} \right) > p\left( {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=?} \right)$,

They can be concluded that:

$\forall \left\{ {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=?} \right\} \in {\left\{ o_{k1} \rightarrow o_{k2}^{\prime } \right\} }^{batch,0} \cup {\left\{ o_{k1} \rightarrow o_{k2}^{\prime } \right\} }^{batch,1} \cup \cdots \cup {\left\{ o_{k1} \rightarrow o_{k2}^{\prime } \right\} }^{batch,BS-1} = {\cup }_{k4}^{BS-1}{\left\{ o_{k1} \rightarrow o_{k2}^{\prime } \right\} }_{k4}^{ batch}$,

namely,

$\forall \left\{ {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=?} \right\} \in {\left\{ o_{k1} \rightarrow o_{k2}^{\prime } \right\} }^{ overall}$, then there are: $p\left( {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=*} \right) > p\left( {o_{k1} \rightarrow }{ o_{k2}^{\prime } \vert }_{k2=?} \right)$,

therefore,

$max\left\{ p\left( {\left\{ o_{k1} \rightarrow o_{k2}^{\prime } \right\} }^{ overall} \right) \right\} = p\left( {o_{k1} \rightarrow o_{k2}^{\prime } \vert }_{k2=*} \right)$.

The necessity is proved.

1.2 B. Comparison of probability knowledge representation learning models

See Table 17.

Table 17 Comparison of KRL models based on probability

Notes:

• 1. element-wise vector product.

• 2. replacing h or t in triple only.

• 3. f($\cdot$): nonlinear function, e.g., RLU(rectified linear units); ${\bar{h}},{\bar{r}}$:2D form of h, r;[,]: concatenating; $*$:convolution operator $\omega$: filter; vec($\cdot$): reshaping as a vector;W:linear transformation matrix; replaing t with T(embeddings of multiple entities) when score involving multiple triples.

• 4. $h_p,t_p$: projection vectors of head and tail entity.

• 5. $\lambda$: scaling coefficient.

• 6. $\textbf{p,m}$: embedding of a patient(p), a medicine(m) or a disease(d).

• 7. M: a set of medicines.

• 8. N: size of a knowledge graph instance.