Skip to main content
Log in

DeepRank: improving unsupervised node ranking via link discovery

  • Published:
Data Mining and Knowledge Discovery Aims and scope Submit manuscript

Abstract

This paper proposes an unsupervised node-ranking model that considers not only the attributes of nodes in a graph but also the incompleteness of the graph structure. We formulate the unsupervised ranking task into an optimization task and propose a deep neural network (DNN) structure to solve it. The rich representation capability of the DNN structure together with a novel design of the objectives allow the proposed model to significantly outperform the state-of-the-art ranking solutions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. https://wsdmcupchallenge.azurewebsites.net/.

  2. https://kddcup2016.azurewebsites.net/.

  3. http://snap.stanford.edu/data/cit-HepPh.html.

  4. http://chato.cl/webspam/datasets/uk2007/.

  5. http://socialnetworks.mpi-sws.org/data-wosn2009.html.

  6. The attributes are the logarithm of: (1) degree divided by average degree of neighbors; (2) in-degree; (3) out-degree; (4, 5) the sum and mean of in-degrees of direct successors; (6, 7) the sum and mean of out-degree of direct predecessors; (8, 9, 10) the number of successors at distance \(k \in \{ 2, 3, 4 \}\); (11, 12, 13) the number of successors at distance k divided by the number of successors at distance \(k - 1\).

References

  • Backstrom L, Leskovec J (2011) Supervised random walks: predicting and recommending links in social networks. In: Proceedings of the fourth ACM international conference on web search and data mining (WSDM’11). ACM, New York, NY, USA, pp 635–644. https://doi.org/10.1145/1935826.1935914

  • Bogolubsky L, Dvurechensky P, Gasnikov A, Gusev G, Nesterov Y, Raigorodskii AM, Tikhonov A, Zhukovskii M (2016) Learning supervised pagerank with gradient-based and gradient-free optimization methods. In: Lee DD, Sugiyama M, Luxburg UV, Guyon I, Garnett R (eds) Advances in neural information processing systems 29. Curran Associates, Inc., pp 4914–4922. http://papers.nips.cc/paper/6565-learning-supervised-pagerank-with-gradient-based-and-gradient-free-optimization-methods.pdf

  • Bromley J, Guyon I, LeCun Y, Säckinger E, Shah R (1993) Signature verification using a “siamese” time delay neural network. In: Proceedings of the 6th international conference on neural information processing systems (NIPS’93). Morgan Kaufmann, San Francisco, CA, USA, pp 737–744. http://dl.acm.org/citation.cfm?id=2987189.2987282

  • Chang H, Cohn D, McCallum A (2000) Learning to create customized authority lists. In: Langley P (ed) Proceedings of the seventeenth international conference on machine learning (ICML 2000), Stanford University, Stanford, CA, USA, June 29–July 2, 2000. Morgan Kaufmann, pp 127–134

  • Clevert D, Unterthiner T, Hochreiter S (2016) Fast and accurate deep network learning by exponential linear units (elus). In: International conference on learning representations arXiv:1511.07289

  • Freeman LC (1978) Centrality in social networks conceptual clarification. Soc Netw 1(3):215–239. https://doi.org/10.1016/0378-8733(78)90021-7

    Article  Google Scholar 

  • Gantner Z, Drumond L, Freudenthaler C, Schmidt-Thieme L (2011) Personalized ranking for non-uniformly sampled items. In: Proceedings of the 2011 international conference on KDD Cup 2011—Volume 18, JMLR.org (KDDCUP’11), pp 231–247. http://dl.acm.org/citation.cfm?id=3000375.3000390

  • Gao B, Liu TY, Wei W, Wang T, Li H (2011) Semi-supervised ranking on very large graphs with rich metadata. In: Proceedings of the 17th ACM SIGKDD international conference on knowledge discovery and data mining (KDD’11). ACM, New York, NY, USA, pp 96–104. https://doi.org/10.1145/2020408.2020430

  • Gehrke J, Ginsparg P, Kleinberg J (2003) Overview of the 2003 KDD cup. SIGKDD Explor Newsl 5(2):149–151. https://doi.org/10.1145/980972.980992

    Article  Google Scholar 

  • Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: Gordon G, Dunson D, Dudk M (eds) Proceedings of the fourteenth international conference on artificial intelligence and statistics, PMLR, Fort Lauderdale, FL, USA, proceedings of machine learning research, vol 15, pp 315–323. http://proceedings.mlr.press/v15/glorot11a.html

  • Gyöngyi Z, Garcia-Molina H, Pedersen J (2004) Combating web spam with trustrank. In: Proceedings of the thirtieth international conference on very large data bases—volume 30, VLDB Endowment (VLDB’04), pp 576–587. http://dl.acm.org/citation.cfm?id=1316689.1316740

  • He X, Liao L, Zhang H, Nie L, Hu X, Chua TS (2017) Neural collaborative filtering. In: Proceedings of the 26th international conference on World Wide Web, International World Wide Web Conferences Steering Committee (WWW’17), Republic and Canton of Geneva, Switzerland, pp 173–182. https://doi.org/10.1145/3038912.3052569

  • Heidemann J, Klier M, Probst F (2010) Identifying key users in online social networks: a pagerank based approach. In: Sabherwal R, Sumner M (eds) Proceedings of the international conference on information systems (ICIS 2010), Saint Louis, Missouri, USA, December 12–15, 2010. Association for Information Systems, p 79. http://aisel.aisnet.org/icis2010_submissions/79

  • Hsu CC, Lai YA, Chen WH, Feng MH, Lin SD (2017) Unsupervised ranking using graph structures and node attributes. In: Proceedings of the tenth ACM international conference on web search and data mining (WSDM’17). ACM, New York, NY, USA, pp 771–779. https://doi.org/10.1145/3018661.3018668

  • Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: International conference on learning representations arXiv:1412.6980

  • Kleinberg JM (1999) Authoritative sources in a hyperlinked environment. J ACM 46(5):604–632. https://doi.org/10.1145/324133.324140

    Article  MathSciNet  MATH  Google Scholar 

  • Lai YA, Hsu CC, Chen WH, Yeh MY, Lin SD (2017) Prune: preserving proximity and global ranking for network embedding. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in neural information processing systems 30. Curran Associates, Inc., pp 5257–5266. http://papers.nips.cc/paper/7110-prune-preserving-proximity-and-global-ranking-for-network-embedding.pdf

  • Liben-Nowell D, Kleinberg J (2007) The link-prediction problem for social networks. J Am Soc Inf Sci Technol 58(7):1019–1031. https://doi.org/10.1002/asi.v58:7

    Article  Google Scholar 

  • Lichtenwalter RN, Lussier JT, Chawla NV (2010) New perspectives and methods in link prediction. In: Proceedings of the 16th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, New York, NY, USA (KDD’10), pp 243–252. https://doi.org/10.1145/1835804.1835837

  • Lü L, Zhou T (2011) Link prediction in complex networks: a survey. Phys A Stat Mech Its Appl 390(6):1150–1170. https://doi.org/10.1016/j.physa.2010.11.027

    Article  Google Scholar 

  • Martínez V, Berzal F, Cubero JC (2016) A survey of link prediction in complex networks. ACM Comput Surv 49(4):69:1–69:33. https://doi.org/10.1145/3012704

    Google Scholar 

  • Martins AFT, Astudillo RF (2016) From softmax to sparsemax: a sparse model of attention and multi-label classification. In: Proceedings of the 33rd international conference on international conference on machine learning—volume 48, JMLR.org, ICML’16, pp 1614–1623. http://dl.acm.org/citation.cfm?id=3045390.3045561

  • Menon AK, Elkan C (2011) Link prediction via matrix factorization. In: Proceedings of the 2011 European conference on machine learning and knowledge discovery in databases—volume part II (ECML PKDD’11). Springer, Berlin, pp 437–452. http://dl.acm.org/citation.cfm?id=2034117.2034146

  • Newman MEJ (2001) Clustering and preferential attachment in growing networks. Phys Rev E 64:025102. https://doi.org/10.1103/PhysRevE.64.025102

    Article  Google Scholar 

  • Page L, Brin S, Motwani R, Winograd T (1999) The pagerank citation ranking: bringing order to the web. Technical Report 1999-66, Stanford InfoLab. http://ilpubs.stanford.edu:8090/422/, previous number = SIDL-WP-1999-0120

  • Rendle S, Freudenthaler C (2014) Improving pairwise learning for item recommendation from implicit feedback. In: Proceedings of the 7th ACM international conference on web search and data mining (WSDM’14). ACM, New York, NY, USA, pp 273–282. https://doi.org/10.1145/2556195.2556248

  • Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958. http://jmlr.org/papers/v15/srivastava14a.html

  • Stern DH, Herbrich R, Graepel T (2009) Matchbox: Large scale online bayesian recommendations. In: Proceedings of the 18th international conference on World Wide Web (WWW’09). ACM, New York, NY, USA, pp 111–120. https://doi.org/10.1145/1526709.1526725

  • Tsoi AC, Morini G, Scarselli F, Hagenbuchner M, Maggini M (2003) Adaptive ranking of web pages. In: Proceedings of the 12th international conference on World Wide Web (WWW’03). ACM, New York, NY, USA, pp 356–365. https://doi.org/10.1145/775152.775203

  • Viswanath B, Mislove A, Cha M, Gummadi KP (2009) On the evolution of user interaction in facebook. In: Proceedings of the 2Nd ACM workshop on online social networks (WOSN’09). ACM, New York, NY, USA, pp 37–42. https://doi.org/10.1145/1592665.1592675

  • Wang Y, Tong Y, Zeng M (2013) Ranking scientific articles by exploiting citations, authors, journals, and time information. In: desJardins M, Littman ML (eds) Proceedings of the twenty-seventh AAAI conference on artificial intelligence, July 14–18, 2013, Bellevue, Washington, USA. AAAI Press. http://www.aaai.org/ocs/index.php/AAAI/AAAI13/paper/view/6363

  • Wang P, Xu B, Wu Y, Zhou X (2015) Link prediction in social networks: the state-of-the-art. Sci China Inf Sci 58(1):1–38. https://doi.org/10.1007/s11432-014-5237-y

    Google Scholar 

  • Xing W, Ghorbani A (2004) Weighted pagerank algorithm. In: Proceedings. In: Second annual conference on communication networks and services research, 2004, pp 305–314. https://doi.org/10.1109/DNSR.2004.1344743

  • Zhai S, Zhang ZM (2015) Dropout training of matrix factorization and autoencoder for link prediction in sparse graphs. In: Venkatasubramanian S, Ye J (eds) Proceedings of the 2015 SIAM international conference on data mining, Vancouver, BC, Canada, April 30–May 2, 2015. SIAM, pp 451–459. https://doi.org/10.1137/1.9781611974010.51

  • Zhukovskiy M, Gusev G, Serdyukov P (2014) Supervised nested pagerank. In: Proceedings of the 23rd ACM international conference on information and knowledge management (CIKM’14). ACM, New York, NY, USA, pp 1059–1068. https://doi.org/10.1145/2661829.2661969

Download references

Acknowledgements

This material is based upon work supported by the Air Force Office of Scientific Research, AOARD under Award Number FA2386-17-1-4038, and Taiwan Ministry of Science and Technology (MOST) under Grant Number 106-2218-E-002-042.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Chin-Chi Hsu.

Additional information

Responsible editor: Jesse Davis, Elisa Fromont, Derek Greene, Bjorn Bringmann.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendices

1.1 Appendix A: Notations

See Table 4.

Table 4 Commonly used notations

1.2 Appendix B: DeepRank pseudo code

figure a
figure b

1.3 Appendix C: Derivation of an upper bound of PageRank objective function

$$\begin{aligned}&\sum _{j \in V} \left( \sum _{i \in P_{j}} \frac{\pi _{i}}{n_{i}} - \pi _{j} \right) ^{2} + \lambda \left( \sum _{j \in V} \pi _{j} - s \right) ^{2} \nonumber \\&\quad = \sum _{j \in V} \left( \sum _{i \in P_{j}} \frac{\pi _{i}}{n_{i}} - \pi _{j} \right) ^{2} + \lambda \left( \sum _{j \in V} \pi _{j} \right) ^{2} - 2 \lambda s \sum _{j \in V} \pi _{j} + \lambda s^{2} \nonumber \\&\quad \le \sum _{j \in V} \left( \sum _{i \in P_{j}} \frac{\pi _{i}}{n_{i}} - \pi _{j} \right) ^{2} + 2 \lambda s \sum _{j \in V} \left( \sum _{i \in P_{j}} \frac{\pi _{i}}{n_{i}} - \pi _{j} \right) + \lambda \left( \sum _{j \in V} \pi _{j} \right) ^{2} + \lambda s^{2} . \end{aligned}$$
(16)

On the right-hand side of the inequality, we add a term \(0 \le \Delta = 2 \lambda s \sum _{j \in V} \sum _{i \in P_{j}} \frac{\pi _{i}}{n_{i}}\). The inequality holds due to non-negative \(\lambda , s, \pi _{i} \forall i\) and \(n_{i} \forall i\). Completing the square to the upper bound (16), we have:

$$\begin{aligned} \sum _{j \in V} \left( \sum _{i \in P_{j}} \frac{\pi _{i}}{n_{i}} - \pi _{j} + \lambda s \right) ^{2} + \lambda \left( \sum _{j \in V} \pi _{j} \right) ^{2} + \lambda s^{2} - \sum _{j \in V} \lambda ^{2} s^{2}. \end{aligned}$$
(17)

Definitely \(\Delta \) is the difference between the upper bound and the original objective function. For ease of presentation, we use matrix representations to derive \(\Delta \). Let vector \(\varvec{\pi } \in [0, \infty ) ^{N}\) be the ranking score vector for all N nodes, while \(\varvec{1}\) represents a N-dimensional constant vector of all 1’s. Matrix \(\varvec{Q} \in [0, \infty ) ^{N \times N}\) denotes a transition matrix where each entry \(q_{ij} = n_{i}^{-1}\) for row i and column j. By the definition of \(\varvec{Q}\), the entry sum of each row of \(\varvec{Q}\) is exactly 1, that is, \(\varvec{Q} \varvec{1} = \varvec{1}\). Having the matrix representations, we derive \(\Delta \) as follows,

$$\begin{aligned} \Delta = 2 \lambda s \sum _{j \in V} \sum _{i \in P_{j}} \frac{\pi _{i}}{n_{i}} = 2 \lambda s \varvec{1}^{\top } \left( \varvec{Q}^{\top } \varvec{\pi } \right) = 2 \lambda s \left( \varvec{1}^{\top } \varvec{Q}^{\top } \right) \varvec{\pi } = 2 \lambda s \varvec{1}^{\top } \varvec{\pi } \!=\! 2 \lambda s \sum _{j \in V} \pi _{j}. \end{aligned}$$
(18)

1.4 Appendix D: Proof for the reduction of node ranking

Suppose that for all link \((i, j) \in E\), the inequality \(\frac{\pi _{j}}{m_{j}} \ge \frac{\pi _{i}}{n_{i}}\) holds. Then for any arbitrary node j, \(\frac{\pi _{j}}{m_{j}} \ge \frac{\pi _{i}}{n_{i}}\) for all direct predecessors i of j. That is, \(\frac{\pi _{j}}{m_{j}}\) must be no less than the average of \(\frac{\pi _{i}}{n_{i}}\) of all direct predecessors \(i \in P_{j}\). Given \(m_{j} = | P_{j} |\), we have:

$$\begin{aligned} \frac{\pi _{j}}{m_{j}} \ge \frac{\pi _{i}}{n_{i}} \text { } \forall \text { } i \in P_{j} \implies \frac{\pi _{j}}{m_{j}} \ge \underbrace{\frac{1}{m_{j}} \sum _{i \in P_{j}} \frac{\pi _{i}}{n_{i}}}_{\text {Average}} \implies \pi _{j} \ge \sum _{i \in P_{j}} \frac{\pi _{i}}{n_{i}} . \end{aligned}$$

1.5 Appendix E: Introduction of competitors

1.5.1 E.0.6 Closeness and betweenness centrality (Freeman 1978)

In social network analysis, centrality methods find the most important nodes based on current network structure. We choose two common centrality definitions in Freeman (1978), closeness and betweenness centralities. Closeness centrality claims that nodes with shorter path length to others are more important. Betweenness centrality claims the more important nodes are part of more shortest paths in the network.

1.5.2 E.0.7 PageRank (Page et al. 1999)

It is a well-known node ranking algorithm without using node attributes. Under Markov Chain framework, we repeatedly update ranking scores using the following rule until convergence,

$$\begin{aligned} \varvec{\pi }^{(t+1)}&= (1 - d) \frac{1}{N} \varvec{1} + d \varvec{Q} \varvec{\pi }^{(t)}, \end{aligned}$$
(19)

where vector \(\varvec{\pi }\) is the ranking score set of all N nodes, \(\varvec{Q} = [ q_{ij} = \frac{1}{n_{j}} ]\) is the transition matrix, \(\varvec{1}\) is a vector of all 1’s, and d is the damping factor normally set to 0.85.

1.5.3 E.0.8 Weighted PageRank (WPR) (Xing and Ghorbani 2004)

This variant of PageRank modifies the transition matrix from uniformly distributed weights to a weight distribution proportional to in-degree and out-degree of pointed nodes. Its update rule is

$$\begin{aligned} \varvec{\pi }^{(t+1)} = (1 - d) \frac{1}{N} \varvec{1} + d \varvec{Q} \varvec{\pi }^{(t)} \text { and } q_{ij} = \frac{1}{\zeta _{j}} \frac{m_{i}}{\sum _{k \in S_{j}} m_{k} } \frac{n_{i}}{\sum _{k \in S_{j}} n_{k} }, \end{aligned}$$
(20)

where \(\zeta _{j} = \sum _{i} q_{ij}\) for \(\varvec{Q} = [ q_{ij} ]\).

1.5.4 E.0.9 Semi-supervised PageRank (SSP) (Gao et al. 2011)

It is the state-of-the-art semi-supervised solution to node ranking. It is composed of a supervised part and an unsupervised part. We adopt only its unsupervised component with node attributes. The objective function of its unsupervised component is simplified as below,

$$\begin{aligned} \arg \min _{\varvec{\pi } \ge 0, \varvec{\omega } \ge 0} \Vert (1 - d) \varvec{X}^{\top } \varvec{\omega } + d \varvec{Q} \varvec{\pi } - \varvec{\pi } \Vert _{2}^{2} , \end{aligned}$$
(21)

where matrix \(\varvec{X} = [ \varvec{x}_{1} \varvec{x}_{2} \ldots \varvec{x}_{N} ] \) denotes the collection of node attributes, \(\varvec{Q} = [ q_{ij} = \frac{1}{n_{j}} ]\) represents the transition matrix, and \(\varvec{\omega }\) refers to the weight vector. Note that weights \(\varvec{\omega }\) and attributes \(\varvec{X}\) should be non-negative. (21) is optimized using projected gradient descent in Gao et al. (2011).

1.5.5 E.0.10 AttriRank (Hsu et al. 2017)

It is the state-of-the-art unsupervised general approach to node ranking with node attributes. We follow the setup written in the original paper for parameter setting and selection. The update equation is as below,

$$\begin{aligned} \varvec{\pi }^{(t+1)} = (1 - d) \frac{1}{N} \varvec{r} + d \varvec{Q} \varvec{\pi }^{(t)} \text { and } r_{i} = \frac{1}{\zeta } \sum _{j \in V} e^{- \gamma \Vert \varvec{x}_{i} - \varvec{x}_{j} \Vert _{2}^{2}}, \end{aligned}$$
(22)

where vector \(\varvec{r} = [ r_{i} ]\) encodes the information of node attributes using the sum of Radial Basis Function (RBF) kernels, \(\zeta = \sum _{i} r_{i}\), and \(\gamma \) is the parameter of RBF kernel.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lai, YA., Hsu, CC., Chen, WH. et al. DeepRank: improving unsupervised node ranking via link discovery. Data Min Knowl Disc 33, 474–498 (2019). https://doi.org/10.1007/s10618-018-0601-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10618-018-0601-y

Keywords

Navigation