DeepRank: improving unsupervised node ranking via link discovery

Abstract

This paper proposes an unsupervised node-ranking model that considers not only the attributes of nodes in a graph but also the incompleteness of the graph structure. We formulate the unsupervised ranking task into an optimization task and propose a deep neural network (DNN) structure to solve it. The rich representation capability of the DNN structure together with a novel design of the objectives allow the proposed model to significantly outperform the state-of-the-art ranking solutions.

Appendices

Appendices

1.1 Appendix A: Notations

See Table 4.

Table 4 Commonly used notations

1.2 Appendix B: DeepRank pseudo code

1.3 Appendix C: Derivation of an upper bound of PageRank objective function

$$\begin{aligned}&\sum _{j \in V} \left( \sum _{i \in P_{j}} \frac{\pi _{i}}{n_{i}} - \pi _{j} \right) ^{2} + \lambda \left( \sum _{j \in V} \pi _{j} - s \right) ^{2} \nonumber \\&\quad = \sum _{j \in V} \left( \sum _{i \in P_{j}} \frac{\pi _{i}}{n_{i}} - \pi _{j} \right) ^{2} + \lambda \left( \sum _{j \in V} \pi _{j} \right) ^{2} - 2 \lambda s \sum _{j \in V} \pi _{j} + \lambda s^{2} \nonumber \\&\quad \le \sum _{j \in V} \left( \sum _{i \in P_{j}} \frac{\pi _{i}}{n_{i}} - \pi _{j} \right) ^{2} + 2 \lambda s \sum _{j \in V} \left( \sum _{i \in P_{j}} \frac{\pi _{i}}{n_{i}} - \pi _{j} \right) + \lambda \left( \sum _{j \in V} \pi _{j} \right) ^{2} + \lambda s^{2} . \end{aligned}$$

(16)

On the right-hand side of the inequality, we add a term $0 \le \Delta = 2 \lambda s \sum _{j \in V} \sum _{i \in P_{j}} \frac{\pi _{i}}{n_{i}}$. The inequality holds due to non-negative $\lambda , s, \pi _{i} \forall i$ and $n_{i} \forall i$. Completing the square to the upper bound (16), we have:

$$\begin{aligned} \sum _{j \in V} \left( \sum _{i \in P_{j}} \frac{\pi _{i}}{n_{i}} - \pi _{j} + \lambda s \right) ^{2} + \lambda \left( \sum _{j \in V} \pi _{j} \right) ^{2} + \lambda s^{2} - \sum _{j \in V} \lambda ^{2} s^{2}. \end{aligned}$$

(17)

Definitely $\Delta$ is the difference between the upper bound and the original objective function. For ease of presentation, we use matrix representations to derive $\Delta$. Let vector $\varvec{\pi } \in [0, \infty ) ^{N}$ be the ranking score vector for all N nodes, while $\varvec{1}$ represents a N-dimensional constant vector of all 1’s. Matrix $\varvec{Q} \in [0, \infty ) ^{N \times N}$ denotes a transition matrix where each entry $q_{ij} = n_{i}^{-1}$ for row i and column j. By the definition of $\varvec{Q}$, the entry sum of each row of $\varvec{Q}$ is exactly 1, that is, $\varvec{Q} \varvec{1} = \varvec{1}$. Having the matrix representations, we derive $\Delta$ as follows,

$$\begin{aligned} \Delta = 2 \lambda s \sum _{j \in V} \sum _{i \in P_{j}} \frac{\pi _{i}}{n_{i}} = 2 \lambda s \varvec{1}^{\top } \left( \varvec{Q}^{\top } \varvec{\pi } \right) = 2 \lambda s \left( \varvec{1}^{\top } \varvec{Q}^{\top } \right) \varvec{\pi } = 2 \lambda s \varvec{1}^{\top } \varvec{\pi } \!=\! 2 \lambda s \sum _{j \in V} \pi _{j}. \end{aligned}$$

(18)

1.4 Appendix D: Proof for the reduction of node ranking

Suppose that for all link $(i, j) \in E$, the inequality $\frac{\pi _{j}}{m_{j}} \ge \frac{\pi _{i}}{n_{i}}$ holds. Then for any arbitrary node j, $\frac{\pi _{j}}{m_{j}} \ge \frac{\pi _{i}}{n_{i}}$ for all direct predecessors i of j. That is, $\frac{\pi _{j}}{m_{j}}$ must be no less than the average of $\frac{\pi _{i}}{n_{i}}$ of all direct predecessors $i \in P_{j}$. Given $m_{j} = | P_{j} |$, we have:

$$\begin{aligned} \frac{\pi _{j}}{m_{j}} \ge \frac{\pi _{i}}{n_{i}} \text { } \forall \text { } i \in P_{j} \implies \frac{\pi _{j}}{m_{j}} \ge \underbrace{\frac{1}{m_{j}} \sum _{i \in P_{j}} \frac{\pi _{i}}{n_{i}}}_{\text {Average}} \implies \pi _{j} \ge \sum _{i \in P_{j}} \frac{\pi _{i}}{n_{i}} . \end{aligned}$$

1.5 Appendix E: Introduction of competitors

1.5.1 E.0.6 Closeness and betweenness centrality (Freeman 1978)

In social network analysis, centrality methods find the most important nodes based on current network structure. We choose two common centrality definitions in Freeman (1978), closeness and betweenness centralities. Closeness centrality claims that nodes with shorter path length to others are more important. Betweenness centrality claims the more important nodes are part of more shortest paths in the network.

1.5.2 E.0.7 PageRank (Page et al. 1999)

It is a well-known node ranking algorithm without using node attributes. Under Markov Chain framework, we repeatedly update ranking scores using the following rule until convergence,

$$\begin{aligned} \varvec{\pi }^{(t+1)}&= (1 - d) \frac{1}{N} \varvec{1} + d \varvec{Q} \varvec{\pi }^{(t)}, \end{aligned}$$

(19)

where vector $\varvec{\pi }$ is the ranking score set of all N nodes, $\varvec{Q} = [ q_{ij} = \frac{1}{n_{j}} ]$ is the transition matrix, $\varvec{1}$ is a vector of all 1’s, and d is the damping factor normally set to 0.85.

1.5.3 E.0.8 Weighted PageRank (WPR) (Xing and Ghorbani 2004)

This variant of PageRank modifies the transition matrix from uniformly distributed weights to a weight distribution proportional to in-degree and out-degree of pointed nodes. Its update rule is

$$\begin{aligned} \varvec{\pi }^{(t+1)} = (1 - d) \frac{1}{N} \varvec{1} + d \varvec{Q} \varvec{\pi }^{(t)} \text { and } q_{ij} = \frac{1}{\zeta _{j}} \frac{m_{i}}{\sum _{k \in S_{j}} m_{k} } \frac{n_{i}}{\sum _{k \in S_{j}} n_{k} }, \end{aligned}$$

(20)

where $\zeta _{j} = \sum _{i} q_{ij}$ for $\varvec{Q} = [ q_{ij} ]$.

1.5.4 E.0.9 Semi-supervised PageRank (SSP) (Gao et al. 2011)

It is the state-of-the-art semi-supervised solution to node ranking. It is composed of a supervised part and an unsupervised part. We adopt only its unsupervised component with node attributes. The objective function of its unsupervised component is simplified as below,

$$\begin{aligned} \arg \min _{\varvec{\pi } \ge 0, \varvec{\omega } \ge 0} \Vert (1 - d) \varvec{X}^{\top } \varvec{\omega } + d \varvec{Q} \varvec{\pi } - \varvec{\pi } \Vert _{2}^{2} , \end{aligned}$$

(21)

where matrix $\varvec{X} = [ \varvec{x}_{1} \varvec{x}_{2} \ldots \varvec{x}_{N} ]$ denotes the collection of node attributes, $\varvec{Q} = [ q_{ij} = \frac{1}{n_{j}} ]$ represents the transition matrix, and $\varvec{\omega }$ refers to the weight vector. Note that weights $\varvec{\omega }$ and attributes $\varvec{X}$ should be non-negative. (21) is optimized using projected gradient descent in Gao et al. (2011).

1.5.5 E.0.10 AttriRank (Hsu et al. 2017)

It is the state-of-the-art unsupervised general approach to node ranking with node attributes. We follow the setup written in the original paper for parameter setting and selection. The update equation is as below,

$$\begin{aligned} \varvec{\pi }^{(t+1)} = (1 - d) \frac{1}{N} \varvec{r} + d \varvec{Q} \varvec{\pi }^{(t)} \text { and } r_{i} = \frac{1}{\zeta } \sum _{j \in V} e^{- \gamma \Vert \varvec{x}_{i} - \varvec{x}_{j} \Vert _{2}^{2}}, \end{aligned}$$

(22)

where vector $\varvec{r} = [ r_{i} ]$ encodes the information of node attributes using the sum of Radial Basis Function (RBF) kernels, $\zeta = \sum _{i} r_{i}$, and $\gamma$ is the parameter of RBF kernel.