Link prediction in heterogeneous data via generalized coupled tensor factorization

Abstract

This study deals with missing link prediction, the problem of predicting the existence of missing connections between entities of interest. We approach the problem as filling in missing entries in a relational dataset represented by several matrices and multiway arrays, that will be simply called tensors. Consequently, we address the link prediction problem by data fusion formulated as simultaneous factorization of several observation tensors where latent factors are shared among each observation. Previous studies on joint factorization of such heterogeneous datasets have focused on a single loss function (mainly squared Euclidean distance or Kullback–Leibler-divergence) and specific tensor factorization models (CANDECOMP/PARAFAC and/or Tucker). However, in this paper, we study various alternative tensor models as well as loss functions including the ones already studied in the literature using the generalized coupled tensor factorization framework. Through extensive experiments on two real-world datasets, we demonstrate that (i) joint analysis of data from multiple sources via coupled factorization significantly improves the link prediction performance, (ii) selection of a suitable loss function and a tensor factorization model is crucial for accurate missing link prediction and loss functions that have not been studied for link prediction before may outperform the commonly-used loss functions, (iii) joint factorization of datasets can handle difficult cases, such as the cold start problem that arises when a new entity enters the dataset, and (iv) our approach is scalable to large-scale data.

Appendix

1.1 Computation for common factors

Here, we show the computation for A:

$$\begin{aligned} \varDelta _{A,1}(Q)&= \left[ \sum \limits _{j,k} Q^{i,j,k} \left( B^{j,r} C^{k,r}\right) \right] = Q_1(BC), \\ \varDelta _{A,2}(Q)&= \left[ \sum \limits _{m} Q^{i,m} \left( D^{m,r}\right) \right] = Q_2 D, \\&A \leftarrow A \circ \frac{Q_1(BC) + Q_2 D}{\hat{X}_1^{-p}(BC) + \hat{X}_2^{-p} D}, \end{aligned}$$

and $B$:

$$\begin{aligned} \varDelta _{B,1}(Q)&= \left[ \sum \limits _{i,k} Q^{i,j,k} \left( A^{i,r} C^{k,r}\right) \right] = Q_1(AC), \\ \varDelta _{B,2}(Q)&= \left[ \sum \limits _{n} Q^{j,n}\left( E^{n,r}\right) \right] = Q_2 E, \\&B \leftarrow B \circ \frac{Q_1(AC) + Q_3 E}{\hat{X}_1^{-p}(AC) + \hat{X}_3^{-p} E}, \end{aligned}$$

given in Model 1, Sect. 4.1.

1.2 Computational complexity

We have conducted experiments on tensor completion problem to demonstrate that time complexity of the modeling framework is $O(N)$ for sparse datasets, where N is the number of known entries. We consider two situations in these experiments: (i) $500 \times 500 \times 500$ three-way array with 99 % missing data (1.25 million known values), and (ii) $1,000 \times 1,000 \times 1,000$ three-way array with 98 % missing data (20 million known values). We have used CP tensor factorization model with R = 3 components to generate data, then added 20 % random Gaussian noise. We have then fitted a CP model using EUC distance-based loss function and used the extracted CP factors to reconstruct the data. Figure 17 shows the average tensor completion performance of 10 independent runs in terms of RMSE score. In the $500 \times 500 \times 500$ case, all ten problems have been solved with an RMSE score around 0.20, with computation times ranging between 400 and 500 s and in the $1,000 \times 1,000 \times 1,000$ case, all ten problems are also solved with an RMSE score around 0.20. The computation times have ranged from 8,000 to 12,000 s, approximately 20 times slower than the $500 \times 500 \times 500$ case, which has 16 times more non-missing entries.

Fig. 17

Results of our algorithm for large-scale problems. The means are shown as solid lines