Unraveling human social behavior motivations via inverse reinforcement learning-based link prediction

Abstract

Link prediction aims to capture the evolution of network structure, especially in real social networks, which is conducive to friend recommendations, human contact trajectory simulation, and more. However, the challenge of the stochastic social behaviors and the unstable space-time distribution in such networks often leads to unexplainable and inaccurate link predictions. Therefore, taking inspiration from the success of imitation learning in simulating human driver behavior, we propose a dynamic network link prediction method based on inverse reinforcement learning (DN-IRL) to unravel the motivations behind social behaviors in social networks. Specifically, the historical social behaviors (link sequences) and a next behavior (a single link) are regarded as the current environmental state and the action taken by the agent, respectively. Subsequently, the reward function, which is designed to maximize the cumulative expected reward from expert behaviors in the raw data, is optimized and utilized to learn the agent’s social policy. Furthermore, our approach incorporates the neighborhood structure based node embedding and the self-attention modules, enabling sensitivity to network structure and traceability to predicted links. Experimental results on real-world dynamic social networks demonstrate that DN-IRL achieves more accurate and explainable of prediction compared to the baselines.

Proof of Theorem

The neighborhood structure vector maps the neighborhood information of each node included in the adjacency matrix to the vector space with fixed dimensions. In this section, we prove the injectivity of the mapping.

Theorem 1

(Injective mapping of node neighborhood structure) Given any one graph $G=(V,E)$ and its adjacency matrix A, the neighborhood topological vector $X_i=[x_{i1},x_{i2},\ldots ,x_{il}]$ of node $v_i$ are obtained by summing the matrices $\{A,A^2,\ldots ,A^l\}$ by row. If and only if the neighborhood structures of node $v_i$ are isomorphic as the node $v_j$, their neighborhood topological vectors are the same $X_i=X_j$.

Proof

Firstly, the element $x_{il}$ represents the number of different walks with length l from the node i. If $l=2$, $x_{i2}$ is equal to

$$\begin{aligned} \begin{aligned} x_{i2}&= \sum _{m=1}^{n}a_{im}^{(2)},\\ a_{im}^{(2)}&=\sum _{k=1}^{n}a_{ik}^{(1)}\times a_{km}^{(1)}, \end{aligned} \end{aligned}$$

(16)

where $a_{ik}^{_{(1)}}$ and $a_{im}^{_{(2)}}$ are the values of A[i, j] and $A^2[i,j]$, respectively. If only if $a_{ik}^{_{(1)}}=1$ and $a_{km}^{_{(1)}}=1$, $a_{im}^{_{(2)}}=1$, namely, there exists a walk from the node $v_i$ to node $v_m$, $(i\rightarrow k\rightarrow m)$ in the graph G. Therefore, $x_{i2}$ denotes the total number of walks with length 2 from node i to arbitrary nodes. Clearly, the value of $X_i$ is solely determined by the neighborhood structure of node $v_i$. Nodes with symmetrical structures are inherently guaranteed to possess identical neighborhood feature vectors.

Secondly, we emphasize the rationale behind the distinct neighborhood feature vectors corresponding to different node neighborhood structures. Two typical case of the distinct node neighborhood structures are shown in Fig. 11.

Fig. 11

Two case of the difference on the neighborhood structures of nodes

Obviously in the Case 1, the discrepancy in the number of k-th hop neighboring nodes between the nodes 1 and a results in distinct values for $x_{1k}$ and $x_{ak}$. Therefore, we need to focus on the Case 2 where the number of k-th hop neighbors is the same for two nodes, but the neighborhood structures are different.

To formalize the neighborhood structure vector, we assume $d_0$, $d_1$ and $d_2$ represent the root node $v_i$, its neighbor nodes and its 2-hop nodes, respectively. Then, the value of $x_{i4}$ in the neighborhood topological vector $X_i$ is

$$\begin{aligned} x_4^i=\left\{ \begin{array}{lr} (d_0\rightarrow d_1\rightarrow d_2\rightarrow d_1\rightarrow d_2)\\ (d_0\rightarrow d_1\rightarrow d_0\rightarrow d_1\rightarrow d_2)\\ (d_0\rightarrow d_1\rightarrow d_0\rightarrow d_1\rightarrow d_0)\\ (d_0\rightarrow d_1\rightarrow d_2\rightarrow d_1\rightarrow d_0) \end{array} \right\} , \end{aligned}$$

(17)

where $(d_0\rightarrow d_1\rightarrow d_2\rightarrow d_1\rightarrow d_2)$ means all the walks from the root node $v_i$ to the 2-hop nodes when the length of walks is equal to 4. We can observe that the number of walks $(d_0\rightarrow d_1\rightarrow d_2\rightarrow d_1\rightarrow d_2)$ is related to the allocation of 2-hop neighbor nodes. The reason of $x_{14}\ne x_{a4}$ is

$$\begin{aligned} \left\{ \begin{array}{lr} (1\rightarrow 5\rightarrow 7\rightarrow 5\rightarrow 7)\\ (1\rightarrow 6\rightarrow 8\rightarrow 6\rightarrow 8) \end{array} \right\} \ne \left\{ \begin{array}{lr} (a\rightarrow f\rightarrow g\rightarrow f\rightarrow g)\\ (a\rightarrow f\rightarrow g\rightarrow f\rightarrow h)\\ (a\rightarrow f\rightarrow h\rightarrow f\rightarrow h)\\ (a\rightarrow f\rightarrow h\rightarrow f\rightarrow g) \end{array} \right\} , \end{aligned}$$

(18)

Furthermore, we suppose $|d_1|=n,|d_2|=m$ represent the number of neighbor nodes and 2-hop nodes of the node $v_i$, respectively. And the allocation of 2-hop nodes is

$$\begin{aligned} m=k_1+k_2+\dots +k_n,k_s\in \{0,1,\ldots ,m\}, \end{aligned}$$

(19)

where $k_s$ represents the number of 2-hop nodes connected with the s-th neighbor node. Then, Eq. (17) is changed into

$$\begin{aligned} x_{i4}=\sum _{s=1}^{n}k_s^2+nm+n^2+m, \end{aligned}$$

(20)

where $\sum _{s=1}^{n}k_s^2$ is the number of walks $(d_0\rightarrow d_1\rightarrow d_2\rightarrow d_1\rightarrow d_2)$, in which both $d_1^{_{(s)}}\rightarrow d_2$ and $d_2\rightarrow d_1^{_{(s)}}\rightarrow d_2$ have $k_s$ walks, $d_1^{_{(s)}}\in d_1$. For example, $x_{a4}=(2^2+0^2+2^2)+(3\times 4+3^2+4)=33$, $x_{14}=(2^2+1^2+1^2)+(3\times 4+3^2+4)=31$. From Eq. (20), when the nodes $v_i$ and $v_j$ have the same number of 2-hop neighbors but different allocation methods, the necessary condition of $x_{i4}=x_{j4}$ is,

$$\begin{aligned} \begin{aligned} k_{i1}+k_{i2}+\ldots +k_{in}&=k_{j1}+k_{j2}+\ldots +k_{jn}=m,\\ k_{i1}^2+k_{i2}^2+\ldots +k_{in}^2&= k_{j1}^2+k_{j2}^2+\ldots +k_{jn}^2. \end{aligned} \end{aligned}$$

(21)

However, Eq. (21) does have a positive integer solution, for example, if $m=6$, $\{k_{i1},k_{i2},k_{i3},k_{i4}\}=\{2,2,2,0\}$ and $\{k_{j1},k_{j2},k_{j3},k_{j4}\}=\{3,1,1,1\}$. Thus we must consider setting a larger l, such as the value of $x_{i6}$ is

$$\begin{aligned} x_{i6}=\sum _{s=1}^{n}k_s^3+\sum _{s=1}^{n}k_s^2+O(n,m), \end{aligned}$$

(22)

where O(n, m) represents the polynomial containing parameters n and m. The reason for the occurrence of the third power of $\{k_s\}$ in Eq. (22) is due to the special walk shown in Fig. 12. Simply put, there are $k_s$ possibilities each time a random walk moves from the s-th neighboring node to its 2-hop nodes.

Fig. 12

The special walk with $(d_0,\overline{d_1,d_2},\ldots )$

Then, when $\{k_{i1},k_{i2},\ldots ,k_{in}\}\ne \{k_{j1},k_{j2},\ldots ,k_{jn}\}$, $l=2N$, $|d_1|=n$, $|k_i|=|k_j|=m$ and $X_i=X_j$, the following equations must be met:

$$\begin{aligned} {\left\{ \begin{array}{ll} \sum _{s=1}^{n}(k_{is})^2=\sum _{s=1}^{n}(k_{js})^2,\\ \sum _{s=1}^{n}(k_{is})^3=\sum _{s=1}^{n}(k_{js})^3,\\ \hspace{5mm}\vdots \hspace{22mm}\vdots \\ \sum _{s=1}^{n}(k_{is})^N=\sum _{s=1}^{n}(k_{js})^N, \end{array}\right. } \end{aligned}$$

(23)

where $k_{is},k_{js}\subset \mathbf {N^*}$, and Eq. (23) can be expressed as an integer programming problem. More importantly, n and m are are two specific positive integers, both of which are smaller than the total number of nodes in the entire network (typically in the range of hundreds to thousands). Therefore, according to the combination theory, the number of solutions in the feasible solution space is finite, and it is

$$\begin{aligned} \Omega (k_i,k_j)=\left( {\begin{array}{c}\left( {\begin{array}{c}m\\ n\end{array}}\right) \\ 2\end{array}}\right) . \end{aligned}$$

(24)

Furthermore, as the value of l increases, it indicates a higher number of constraints in the integer problem of Eq. (23), resulting in a lower probability of finding feasible solutions. In other words, there is a injective mapping from the neighborhood structure to the neighborhood topological vector $\{x_{i1},x_{i2},\ldots ,x_{in}\}$, when l is set as a sufficiently large integer. $\square$