Temporal probabilistic measure for link prediction in collaborative networks

Abstract

Link prediction addresses the problem of finding potential links that may form in the future. Existing state of art techniques exploit network topology for computing probability of future link formation. We are interested in using Graphical models for link prediction. Graphical models use higher order topological information underlying a graph for computing Co-occurrence probability of the nodes pertaining to missing links. Time information associated with the links plays a major role in future link formation. There have been a few measures like Time-score, Link-score and T_Flow, which utilize temporal information for link prediction. In this work, Time-score is innovatively incorporated into the graphical model framework, yielding a novel measure called Temporal Co-occurrence Probability (TCOP) for link prediction. The new measure is evaluated on four standard benchmark data sets : DBLP, Condmat, HiePh-collab and HiePh-cite network. In the case of DBLP network, TCOP improves AUROC by 12 % over neighborhood based measures and 5 % over existing temporal measures. Further, when combined in a supervised framework, TCOP gives 93 % accuracy. In the case of three other networks, TCOP achieves a significant improvement of 5 % on an average over existing temporal measures and an average of 9 % improvement over neighborhood based measures. We suggest an extension to link prediction problem called Long-term link prediction, and carry out a preliminary investigation. We find TCOP proves to be effective for long-term link prediction.

Appendices

Appendix A: Graphical model framework

The notation of undirected graphical model called Markov Random Field [14] is given in this section.

Notation Given a graph G=(V,E) where random variables(X) are indexed by V, i.e X={X _v },v∈V is probabilistic model, if it satisfies the following properties:

• 1. Pair wise Markov Property : Any two non adjacent variables are conditionally independent.

  $$ X_{u}~ \perp\!\!\!\perp ~X_{v}, if (u,v) \notin E $$

  (3)

• 2. Local Markov Property : A variable is conditionally independent of all the other variables given its neighbors:

  $$ X_{u}~ \perp\!\!\!\perp ~X_{V-\Gamma(v)} ~|~\Gamma(v) $$

  (4)

  where Γ(v) is set of neighbors of v.

• 3. Global Markov Property : Two sets A,B⊆V are conditionally independent, given their separator set S = A∩B in G.

  $$ X_{A} \perp\!\!\!\perp X_{B}~|~X_{S} $$

  (5)

Whenever a general graph exhibits the Markovian properties, it can be factorized over cliques of G.i.e

$$\begin{array}{@{}rcl@{}} P(X= x) &=& \frac{1}{Z}\prod\limits_{C~ is~ a~ Max~ Cliq(G)} \phi_{c}(X_{c})\\ &&\text{where Z is normalizing factor defined by} \\ Z &=& {\sum\limits_{{x\,{\in}\,{X}}}}\,\,\, \prod\limits_{C~ is~ a~ Max~ Cliq(G)} \phi_{c}(X_{c}) \end{array} $$

(6)

where ϕ _c (X _c ) is clique potential of max Clique C. If G is directed, this graphical model is called Bayesian Network and if it is undirected, it is called Markov Random Field(MRF).

Appendix B: Example illustration of TCOP on snapshot of DBLP dataset

The joint probability inference of the nodes 195 and 6587 using belief propagation in junction tree is illustrated here.

The junction tree of Fig. 1b is shown in Fig. 3 and the corresponding clique potential tables computed using Algorithm 1 are shown in Tables 9, 10 and 11.

Fig. 3

Junction tree of Fig 1b

Table 9 $\phi _{C_{1}}$

Table 10 $\phi _{C_{2}}$

Table 11 $\phi _{C_{3}}$

We can see in Fig. 3 that there are 3 cliques and 2 separators.

Cliques :

$$\begin{array}{@{}rcl@{}} &&C_{1}=\{6246, 6587, 37227\} \\ &&C_{2}=\{195, 6246\}~\text{and}\\ &&C_{3}=\{6587, 10497, 37227\} \end{array} $$

Separators:

$$\begin{array}{@{}rcl@{}} &&S(C_{1},C_{2})=\{6246\}\\ &&S(C_{1},C_{3})=\{6587, 37227\} \end{array} $$

There will be one clique potential corresponding to each clique. In the example, the potentials are $\phi _{C_{1}}, \phi _{C_{2}}~ and~ \phi _{C_{3}}$. These potential Tables are shown in Tables 9, 10 and 11.

The belief propagation in junction tree has two steps: Upward belief propagation and downward belief propagation.

1.1 Belief propagation in upward direction

• For each leaf in the junction tree, send a message to its parent. The message is the marginal of its table, summing out any variable not in the separator.

  $$ for~x \in S(C,C^{\prime}),~ \phi_{S(C,C^{\prime})}(x) = \sum\limits_{x\in C^{\prime}, x \notin S}P(x) $$

  (7)

  That means, every leaf clique node C ^′ sends the potential table computed by (7), to its parent clique, separated by separator S(C,C ^′).

  In the above example illustration, Message from C ₂→C ₁ : Marginalize $\phi _{C_{2}}$ to S(C ₁,C ₂) and is shown in Table 12. In the same way, message from C ₃→C ₁ can be obtained by marginalizing $\phi _{C_{3}}$ to S(C ₁,C ₃), i.e,6587, 37227, as shown in Table 13.

• When a parent Clique(say C) receives a message from a child (say C ^′), it multiplies its table by the message table to obtain its new table and thus updates for all child nodes C ^′.

  $$ \phi_{C}(x)= \prod\limits_{C^{\prime}} \phi_{C}(x) \phi_{S(C,C^{\prime})}(x) $$

  (8)

  Continuing the same example as above,

  Update C ₁ by the message from its child C ₂ to get Table 14.

  Update C ₁ by the message from its child C ₃ to get Table 15.

• When a parent receives messages from all its children, it repeats the process. This process continues until the root receives messages from all its children. In the above example, the tree is of height 1. So, therefore, the process stops in one iteration. The final potential of clique C ₁ will be as in Table 16

Table 12 Message from $C_{2} \rightarrow C_{1}$

Table 13 Message from $C_{3} \rightarrow C_{1}$

Table 14 Updated C ₁ after C ₁ receives message from C ₂

Table 15 Updated C ₁ after C ₁ receives message from C ₃

Table 16 FInal Potential table of clique C ₁ after upward pass

After updations to clique potentials are carried out in upward pass, the downward pass is initiated.

1.2 Belief propagation in downward direction

This step reverses upward pass, starting at the root.

• The root(say C) sends a message to each of its children. More specifically, the root divides its current table by the message received from the child through the separator(say S(C,C ^′)), marginalizes the resulting table to the separator, and sends the result to the child.

  $$ \phi_(x)= \sum\limits_{x \in C, x \notin S(C,C^{\prime})}\frac{\phi_{C^{\prime}}(x)}{\phi_{S(C,C^{\prime})}(x)} $$

  (9)

In the example, the message from clique C ₁→C ₂ is computed, first by dividing C ₁ by S(C ₁,C ₂) as shown in Table 17 and then marginalizing it to S(C ₁,C ₂) as shown in Table 18.

Table 17 C ₁/S(C ₁,C ₂)

Table 18 Message from $C_{1} \rightarrow C_{2}$

In the same way, the message from clique C ₁→C ₃ is computed, first by dividing C ₁ by S(C ₁,C ₃) as shown in Table 19 and then marginalizing it to S(C ₁,C ₃) as shown in Table 20.

Table 19 C ₁/S(C ₁,C ₃)

Table 20 Message from $C_{1} \rightarrow C_{3}$

• Each child(C ^′) multiplies its table by its parent’s(C) table and repeats the process (acts as a root) until leaves are reached.

  $$ \phi_{C^{\prime}}(x)=\prod\phi_{C}(x)\phi_{S}(x) $$

  (10)

  Thus, final potential table of clique is obtained by multiplying the table of C ₂ by message obtained by its parent C ₁ through its separator S(C ₁,C ₂), which is shown in table . Similar is the case for getting final potential of clique C ₃ (Tables 21 and 22).

Table 21 Final potential table of clique C ₂

Table 22 Final potential table of clique C ₃

Thus, potential of each clique is calculated as a joint marginal of its variables, by marginalizing(sum out) over the variables that are not needed. Junction Tree Algorithm finishes the upward and downward passes, Potentials will be equal to Marginals. To infer the joint probability of two variables, we will pick those two variables from any cliques, and multiply their potentials.

For instance, to compute the joint probability of nodes 195 and 6587, first pick the clique containing 195, i.e, C ₂ and marginalize the two entries corresponding to the value 1 of node 195, which is 0.2158, and then pick the clique containing node 6587, i.e, C ₃ and marginalize the entries corresponding to value 1 for node 6587, which is 0.01438. The joint probability is simply the product of 0.2158 * 0.01438 = 0.003103204.