Double-phase locality-sensitive hashing of neighborhood development for multi-relational data

Abstract

Multi-relational (MR) data refer to the objects that involve multi-associated tables or a relational database, and they are widely used in diverse applications. In spite of rich achievements in concrete classification or regression tasks, a neighborhood development algorithm customized to MR data is still missing. The reason lies in the fact that MR data are high-dimensional and highly structured. To address these two difficulties, this paper presents a double-phase locality-sensitive hashing (DPLSH) algorithm to develop neighborhood for MR data. DPLSH consists of offline and online hashing schemas to draw and summarize local closeness information from each involved table. Based on the closeness information, three criteria of neighborhood development are proposed. DPLSH is encoded with parameterization heuristics to make the algorithm data adaptive and less costly. Extensive experiments indicate that for MR data, the quality of neighborhoods produced by DPLSH is better than its peers; besides, in common data environment, DPLSH also exhibits the competitive behaviors with the state of the art.

Appendices

Appendix A

For dataset $\left\{ {x_1,\ldots ,x_N } \right\} ,\,x_i \in \mathfrak {R}^n$, OCSVM constructs an objective function that takes the value of ‘+1’ in a small region capturing most of the data points and ‘$-1$’ elsewhere. Denote $\phi$ as the map from the input space to the feature space, and $G$ as the penalty coefficient. Parameters $w$ and $b$ of the hyperplane separating the data from the origin are determined by solving the following quadratic programming problem (Schölkopf et al. 2001):

$$\begin{aligned}&\min \frac{1}{2}\vert \vert w\vert \vert ^2+b+G\sum \limits _{i=1}^N \xi _i\\&\text{ s.t }.\,\,\,\,\,\,w\cdot \phi (x_i )+b\ge -\xi _i ,\,\,\,\xi _i \ge 0\,,\,\,i=1,2\ldots , N \nonumber \end{aligned}$$

(11)

Denote $\gamma _{i }\ge$ 0 as the Lagrange multipliers. Rewrite (11) into the Lagrange form and use the Kernel trick $k(x_{i}, x_{j})=\langle \phi (x_{i}), \phi (x_{j})\rangle$, and there is:

$$\begin{aligned}&\min \quad \sum \limits _{i=1}^N \sum \limits _{j=1}^N \gamma _i \gamma _j k(x_i ,x_j )-\sum \limits _{i=1}^N \gamma _i k(x_i ,x_i )\nonumber \\&\text{ s.t. }\,\,\,\,\sum \limits _{i=1}^N \gamma _i =1,\,\,\,0\le \gamma _i \le G \end{aligned}$$

(12)

Points with $\gamma _{i }\ne$ 0 are support vectors (SVs). Denote $x_{s}$ as one SV; then according to $w \cdot \phi (x)+b = 0$, $b$ is computed as:

$$\begin{aligned} b=-w\cdot \phi (x_s )&=-\sum \limits _{i=1}^N \gamma _i \langle \phi (x_i ),\phi (x_s )\rangle \nonumber \\ {}&=-\sum \limits _{i=1}^N \gamma _i k(x_i ,x_s ) \end{aligned}$$

(13)

The quadratic optimization of OCSVM consumes huge cost. We give another version of OCSVM that employs the square of the slack variable, named as SSOCSVM, to transfer the quadratic optimization problem to solve a system of linear equations. SSOCSVM is formulated as:

$$\begin{aligned}&\min \quad \frac{1}{2}\vert \vert w\vert \vert ^2+b+\frac{G}{2}\sum \limits _{i=1}^N \xi _i^2\nonumber \\&\text{ s.t. }\,\,\,\,w\cdot \phi (x_i )+b=-\xi _i ,\,\,\,i=1, 2,\ldots ,N \end{aligned}$$

(14)

Therein the employment of $\xi _{i}^{2}$ allows the removing of the constraint $\xi _{i} \ge$ 0. Write the Lagrange function of (14):

$$\begin{aligned} L={1 \over 2}\vert \vert w\vert \vert ^2+b+{G \over 2}\sum \limits _{i=1}^N \xi _i^2 -\sum \limits _{i=1}^N \beta _i (w\cdot \phi (x_i )+b+\xi _i ) \end{aligned}$$

(15)

Therein $\beta _{i}$ are the Lagrange multipliers. The conditions for optimality are:

$$\begin{aligned} {{\partial L} \over {\partial w}}=w-\sum \limits _{i=1}^N \beta _i \phi (x_i )=0,\,\,\,\text{ then }\,\,w=\sum \limits _{i=1}^N \beta _i \phi (x_i ) \end{aligned}$$

(16)

$$\begin{aligned} {{\partial L} \over {\partial \xi _i }}=G\xi _i -\beta _i =0,\,\,\,\text{ then }\,\,\xi _i ={{\beta _i } \over G} \end{aligned}$$

(17)

$$\begin{aligned} {{\partial L} \over {\partial b}}=-1+\sum \limits _{i=1}^N \beta _i =0,\,\,\text{ then } \sum \limits _{i=1}^N \beta _i =1 \end{aligned}$$

(18)

$$\begin{aligned} {{\partial L} \over {\partial \beta _i }}=w\phi (x_i )-b+\xi _i =0,\,\,\text{ then }\,\,b=w\phi (x_i )+\xi _i \end{aligned}$$

(19)

which can be reformulated as the solution to the following set of linear equations:

$$\begin{aligned} \left( \begin{array}{llll} I &{}\quad {\overrightarrow{0} } &{}\quad 0 &{}\quad {-\phi ^T} \\ {\overrightarrow{0} } &{}\quad {GI} &{}\quad 0 &{}\quad {-I} \\ {\overrightarrow{0} } &{}\quad {\overrightarrow{0} } &{}\quad 0 &{}\quad I \\ \phi &{}\quad I &{}\quad {-1} &{}\quad {\overrightarrow{0} } \\ \end{array} \right) \left( \begin{array}{l} w \\ \xi \\ b \\ \beta \\ \end{array} \right) =\left( \begin{array}{l} 0 \\ 0 \\ 1 \\ 0 \\ \end{array} \right) \end{aligned}$$

(20)

with $\phi = ( {\phi ( {x_1 }),\ldots ,\phi ( {x_N })})^T,\xi =( {\xi _1 ,\ldots ,\xi _N })^T,\beta =( {\beta _1 ,\ldots ,\beta _N })^T$ . $I$ is the identity matrix of size $N\times N$. $\overrightarrow{0}$ is a zero matrix of size $N\times N$. $E$ is the identity matrix. The solution of SSOCSVM is obtained by solving the following linear equation:

$$\begin{aligned} \left( \begin{array}{ll} 0 &{}\quad I \\ {-I^T} &{}\quad {\phi \phi ^T+G^{-1}E} \\ \end{array} \right) \left( \begin{array}{l} b \\ \beta \\ \end{array} \right) =\left( \begin{array}{l} 1 \\ {\overrightarrow{0} } \\ \end{array} \right) \end{aligned}$$

(21)

Denote ($K)_{ij }$= ($\phi \phi ^{T})_{ij}=k(x_{i}$, $x_{j})$, thus (21) is identical to (1) mentioned in Sect. 4.3. Note that SSOCSVM runs on the instances of main table, so ${\phi }= ({\phi }(x_{(0)1}),\ldots , {\phi }(x_{(0)N}))^{T}$ and $K_{ij}=k$($\phi$($x_{(0)i})$, $\phi$($x_{(0)j}))$.

We select those data whose support values are large as SVs. In the single-table data environment, $N_\mathrm{SV}$ (the number of SVs) can be predefined problem-dependently. For MR data, here we set $N_\mathrm{SV}$ as $C$, the number of the selected dimensions, so as to facilitate the consequent online hashing process.

Appendix B

Proof of Statement 1. We first investigate OCSVM and then come to SSOCSVM.

For OCSVM, the convex quadratic optimization leads to the sparseness of the solution of (11). It means that only a few points with nonzero support values serve as SVs. Points with nonzero support values are SVs, so we reach the conclusion that SVs make contribution to the hyperplane formulation and reveal the discriminative information of dataset.

For SSOCSVM, the sparseness of the solution is missing since it solves a system of linear equations. That makes most data equipped with nonzero support values. But according to (16) and (19), those data with large support values determine the direction $w$ and the offset $b$ of the hyperplane. And those data points with small support values make lesser contribution to model formulation. So the Statement 1 holds.

Proof of Statement 2. According to Statement 1, the points corresponding to the top section of the sorted support value list serve as SVs. And they are located around the boundaries to tell the discrimination information of dataset. Figure 7 shows an example of 2-dimension points. Denote point A as a SV. Obviously its ordinate is the upper bound of all data’s ordinates. For another SV point B, its abscissa is quite close to the upper bound of all data’s abscissas. That proofs the Statement 2.

Fig. 7

The illustration of Statement 2

Proof of Statement 3. Denote the top $C$ SV points as: ${SV}_{(1)}, \ldots {SV}_{(C)}$. Without loss of generality, we assume that the ${SV}_{(j),d}$ (coordinate of ${SV}_{(j) }$ in the $d$th dimension) serves as the upper bound (or lower bound) of coordinates of other data in that dimension. When we use the resulted $v_{d}$ to do the partition in the $d$th dimension, we actually plot a circle that takes $q$ as the center and $v_{d}$ as the radius. That leads to the scenario shown in Fig. 8. According to the specification (2), $v_{d} = (q_{d}+{SV}_{(j),d})/2$, which indicates such a circle is located within the boundaries. Thus, the cells produced by the inequality: $x_{i,j}<v_{j}$ will include the same-cluster members of $q$. Consequently, the resulted neighborhood is well expected to cover the same-cluster members of the query. That proofs the Statement 3.

Fig. 8

The illustration of Statement 3