Diversified Top-k Querying in Knowledge Graphs

Abstract

The existing literatures of the query processing on knowledge graphs focus on an exhaustive enumeration of all matches, which is time-consuming. Users are often interested in diversified top-k matches, rather than the entire match set. Motivated by these, this paper formalizes the diversified top-k querying (DTQ) problem in the context of RDF/SPARQL and proposes a diversification function to balance importance and diversity. We first prove that the decision problem of DTQ is NP-complete, and give a baseline algorithm with an approximation ratio of 2. Secondly, an index-based algorithm with the early termination property is proposed. The index is adept in parallel diversified top-k selection in multicore architectures. Using real-world and synthetic data, we experimentally verify that our algorithms are efficient and effective in computing meaningful diversified top-k matches.

Appendix

We will prove the approximation ratio of Algorithm 1 is 2. $\mathbb {M}(G_D,Q_D)$ is named $\mathbb {M}$ for short. For disjoint subsets $S,T \subseteq \mathbb {M}$, let $d(S)=\sum _{u,v\in S}\delta _D(u,v)$, $d(S,T)=\sum _{u\in S, v\in T}\delta _D(u,v)$ and $f(S)=\sum _{u\in S}\delta _I(u)$.

Now we define marginal gain. For any given subset $S\subseteq \mathbb {M}$ and an element $u\in \mathbb {M}\backslash S$, let F(S) be the value of the objective function, $d_u(S)=\sum _{v\in S}\delta _D(u,v)$ be the marginal gain on the diversity, $f_u(S)=f(S+u)-f(S)=f(u)=\delta _I(u)$ be the marginal gain on the importance, and $F_u(S) = (1-k)(1-\lambda )f_u(S) + 2\lambda d_u(S)$ be the total marginal gain on the objective function. Let $f_u^{'}(S) = \frac{1}{2}(k-1)(1-\lambda )f_u(S)$ and $F_u^{'}(S) = f_u^{'}(S) + 2\lambda d_u(S)$.

We utilize the a theorom in [13]: Given a metric distance function $d(\cdot ,\cdot )$, and two disjoint sets X and Y, the following inequality holds: $(|X|-1)d(X,Y)\ge |Y|d(X)$.

Let O be the optimal solution and G be the greedy one at the end of the Algorithm 1. Let $G_i$ be the greedy solution at the end of step i, $i<k$; let $A=O\cap G_i$, $B=G_i\backslash A$ and $C = O\backslash A$. By Lemma 1, we have the following three inequalities: (1) $(|C|-1)d(B,C)\ge |B|d(C)$; (2) $(|C|-1)d(A,C)\ge |A|d(C)$; (3) $(|A|-1)d(A,C)\ge |C|d(A)$. Besides, we have (4) $d(A,C)+d(A)+d(C) =d(O)$.

When $k = 1$, match u with the largest $\delta _I(u)$ must be in both G and O, so $F(G)=\frac{1}{2}(k-1)(1-\lambda )f_u(S)+2\lambda d_u(S)=\frac{1}{2}(k-1)(1-\lambda )\delta _I(u)+0=\frac{1}{2}F(O)$ apparently.

When $k>1$, suppose $|C|=1$ and $i=k-1$. Let v be the element in C, and let u be the element taken by the greedy algorithm in the next step, and then $F_u^{'}(G_i)\ge F_v^{'}(G_i)$. Therefore, $\frac{(k-1)(1-\lambda )}{2}f_u(G_i) + 2\lambda d_u(G_i) \ge \frac{(k-1)(1-\lambda )}{2}f_v(G_i) +2\lambda d_v(G_i)$, which implies $F_u(G_i)=(k-1)(1-\lambda )f_u(G_i)+2\lambda d_u(G_i) \ge \frac{(k-1)(1-\lambda )}{2}f_u(G_i) +2\lambda d_u(G_i) \ge \frac{(k-1)(1-\lambda )}{2}f_v(G_i) + 2\lambda d_v(G_i) \ge \frac{1}{2}F_v(G_i)$, hence $F(G)\ge \frac{1}{2}F(O)$.

Now we can suppose that $k>1$ and $|C|>1$. We apply the following non-negative multipliers to Inequality. (1) (2) (3) and Eq. (4) and add them: (1)$\,\times \,\frac{1}{|C|-1}$ + (2)$\,\times \,\frac{|C|-|B|}{k(|C|-1)}$ + (3)$\,\times \,\frac{i}{k(k-1)}$ + (4)$\,\times \,\frac{i|C|}{k(k-1)}$; then we have $d(A,C)\,+\,d(B,C)-\frac{i|C|(k-|C|)}{k(k-1)(|C|-1)}d(C)\ge \frac{i|C|}{k(k-1)}d(O)$.

Since $k>|C|$, we have $d(C,G_i)=d(C,A+B)=d(C,A)+d(C,B)\ge \frac{i|C|}{k(k-1)}d(O)$. Suppose P is a set, we define function $f^{'}(P)=\sum _{x\in P}f_x^{'}(P)$. Then, $\sum _{v\in C}f_v^{'}(G_i) = f^{'}(C\cup G_i)-f^{'}(G_i) = f^{'}(O)-f^{'}(G)$. Therefore,

$$\begin{aligned} \sum _{v\in C}F_v^{'}(G_i)&=\sum _{v\in C}[f_v^{'}(G_i) + 2\lambda d(\{v\},G_i)] \\&=\sum _{v\in C}f_v^{'}(G_i)+2\lambda d(C,G_i) \ge (f'(O)-f'(G))+2\lambda \times \frac{i|C|}{k(k-1)}d(O) \end{aligned}$$

Let $u_{i+1}$ be the element taken at step $(i+1)$, and then we have $F_{u_{i+1}}^{'}(G_i)\ge \frac{1}{k}(f'(O)-f'(G))+\frac{2\lambda i}{k(k-1)}d(O)$. Summing over all i from 0 to $k-1$, we have $F^{'}(G)=\sum _{i=0}^{i=k-1}F_{u_{i+1}}^{'}(G_i) \ge (f'(O)-f'(G))+\lambda d(O)$. Hence, $F^{'}(G)=f'(G)+2\lambda d(G)\ge f'(O)-f'(G)+\lambda d(O)$, and $F(G)=(k-1)(1-\lambda )f(G)+2\lambda d(G)=2f'(G)+2\lambda d(G)\ge f'(O)+\lambda d(O)=\frac{1}{2}[(k-1)(1-\lambda )f(O)+2\lambda d(O)]=\frac{1}{2}F(O)$.

So the approximation ratio of Algorithm 1 is 2. This completes the proof.    $\square$