ABSTRACT
Network embedding aims to preserve vertex similarity in an embedding space. Existing approaches usually define the similarity by direct links or common neighborhoods between nodes, i.e. structural equivalence. However, vertexes which reside in different parts of the network may have similar roles or positions, i.e. regular equivalence, which is largely ignored by the literature of network embedding. Regular equivalence is defined in a recursive way that two regularly equivalent vertexes have network neighbors which are also regularly equivalent. Accordingly, we propose a new approach named Deep Recursive Network Embedding (DRNE) to learn network embeddings with regular equivalence. More specifically, we propose a layer normalized LSTM to represent each node by aggregating the representations of their neighborhoods in a recursive way. We theoretically prove that some popular and typical centrality measures which are consistent with regular equivalence are optimal solutions of our model. This is also demonstrated by empirical results that the learned node representations can well predict the indexes of regular equivalence and related centrality scores. Furthermore, the learned node representations can be directly used for end applications like structural role classification in networks, and the experimental results show that our method can consistently outperform centrality-based methods and other state-of-the-art network embedding methods.
Supplemental Material
- Hervé Abdi . 2007. The Kendall rank correlation coefficient. Encyclopedia of Measurement and Statistics. Sage, Thousand Oaks, CA (2007), 508--510.Google Scholar
- Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton . 2016. Layer normalization. arXiv preprint arXiv:1607.06450 (2016).Google Scholar
- Joonhyun Bae and Sangwook Kim . 2014. Identifying and ranking influential spreaders in complex networks by neighborhood coreness. Physica A: Statistical Mechanics and its Applications Vol. 395 (2014), 549--559.Google Scholar
- Marc Barthelemy . 2004. Betweenness centrality in large complex networks. The European Physical Journal B-Condensed Matter and Complex Systems Vol. 38, 2 (2004), 163--168.Google ScholarCross Ref
- Phillip Bonacich . 2007. Some unique properties of eigenvector centrality. Social networks Vol. 29, 4 (2007), 555--564.Google Scholar
- Phillip Bonacich and Paulette Lloyd . 2015. Eigenvector centrality and structural zeroes and ones: When is a neighbor not a neighbor? Social Networks Vol. 43 (2015), 86--90.Google ScholarCross Ref
- Stephen P Borgatti and Martin G Everett . 1993. Two algorithms for computing regular equivalence. Social networks Vol. 15, 4 (1993), 361--376.Google Scholar
- Peng Cui, Xiao Wang, Jian Pei, and Wenwu Zhu . 2017. A Survey on Network Embedding. arXiv preprint arXiv:1711.08752 (2017).Google Scholar
- Young-Ho Eom and Hang-Hyun Jo . 2015. Tail-scope: Using friends to estimate heavy tails of degree distributions in large-scale complex networks. Scientific reports Vol. 5 (2015).Google Scholar
- Pablo M Gleiser and Leon Danon . 2003. Community structure in jazz. Advances in complex systems Vol. 6, 04 (2003), 565--573.Google Scholar
- Xavier Glorot, Antoine Bordes, and Yoshua Bengio . 2011. Deep sparse rectifier neural networks. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics. 315--323.Google Scholar
- Aditya Grover and Jure Leskovec . 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 855--864. Google ScholarDigital Library
- Keith Henderson, Brian Gallagher, Tina Eliassi-Rad, Hanghang Tong, Sugato Basu, Leman Akoglu, Danai Koutra, Christos Faloutsos, and Lei Li . 2012. Rolx: structural role extraction & mining in large graphs KDD. ACM, 1231--1239. Google ScholarDigital Library
- Sepp Hochreiter and Jürgen Schmidhuber . 1997. Long short-term memory. Neural computation Vol. 9, 8 (1997), 1735--1780. Google ScholarDigital Library
- Xiao Huang, Jundong Li, and Xia Hu . 2017. Label informed attributed network embedding. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining. ACM, 731--739. Google ScholarDigital Library
- Diederik Kingma and Jimmy Ba . 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
- Maksim Kitsak, Lazaros K Gallos, Shlomo Havlin, Fredrik Liljeros, Lev Muchnik, H Eugene Stanley, and Hernán A Makse . 2010. Identification of influential spreaders in complex networks. arXiv preprint arXiv:1001.5285 (2010).Google Scholar
- Elizabeth A Leicht, Petter Holme, and Mark EJ Newman . 2006. Vertex similarity in networks. Physical Review E Vol. 73, 2 (2006), 026120.Google ScholarCross Ref
- Jundong Li, Harsh Dani, Xia Hu, Jiliang Tang, Yi Chang, and Huan Liu . 2017. Attributed network embedding for learning in a dynamic environment Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. ACM, 387--396. Google ScholarDigital Library
- Jian-Hong Lin, Qiang Guo, Wen-Zhao Dong, Li-Ying Tang, and Jian-Guo Liu . 2014. Identifying the node spreading influence with largest k-core values. Physics Letters A Vol. 378, 45 (2014), 3279--3284.Google ScholarCross Ref
- Linyuan Lü, Tao Zhou, Qian-Ming Zhang, and H Eugene Stanley . 2016. The H-index of a network node and its relation to degree and coreness. Nature communications Vol. 7 (2016), 10168.Google Scholar
- Dijun Luo, Feiping Nie, Heng Huang, and Chris H Ding . 2011. Cauchy graph embedding. In Proceedings of the 28th International Conference on Machine Learning (ICML-11). 553--560. Google ScholarDigital Library
- Jianxin Ma, Peng Cui, and Wenwu Zhu . 2018. DepthLGP: Learning Embeddings of Out-of-Sample Nodes in Dynamic Networks. (2018).Google Scholar
- Tomávs Mikolov, Martin Karafiát, Lukávs Burget, Jan vCernockỳ, and Sanjeev Khudanpur . 2010. Recurrent neural network based language model. In Eleventh Annual Conference of the International Speech Communication Association.Google ScholarCross Ref
- Eisha Nathan and David A Bader . 2017. A Dynamic Algorithm for Updating Katz Centrality in Graphs Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017. ACM, 149--154. Google ScholarDigital Library
- Kazuya Okamoto, Wei Chen, and Xiang-Yang Li . 2008. Ranking of closeness centrality for large-scale social networks. Lecture Notes in Computer Science Vol. 5059 (2008), 186--195. Google ScholarDigital Library
- Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd . 1999. The PageRank citation ranking: Bringing order to the web. Technical Report. Stanford InfoLab.Google Scholar
- Bryan Perozzi, Rami Al-Rfou, and Steven Skiena . 2014. Deepwalk: Online learning of social representations Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 701--710. Google ScholarDigital Library
- Leonardo FR Ribeiro, Pedro HP Saverese, and Daniel R Figueiredo . 2017. struc2vec: Learning node representations from structural identity Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 385--394. Google ScholarDigital Library
- Ryan A Rossi and Nesreen K Ahmed . 2015. Role discovery in networks. IEEE Transactions on Knowledge and Data Engineering Vol. 27, 4 (2015), 1112--1131.Google ScholarDigital Library
- Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad . 2008. Collective classification in network data. AI magazine Vol. 29, 3 (2008), 93.Google Scholar
- Hava T Siegelmann and Eduardo D Sontag . 1995. On the computational power of neural nets. Journal of computer and system sciences Vol. 50, 1 (1995), 132--150. Google ScholarDigital Library
- Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei . 2015. Line: Large-scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 1067--1077. Google ScholarDigital Library
- Ke Tu, Peng Cui, Xiao Wang, Fei Wang, and Wenwu Zhu . 2017. Structural Deep Embedding for Hyper-Networks. arXiv preprint arXiv:1711.10146 (2017).Google Scholar
- Daixin Wang, Peng Cui, and Wenwu Zhu . 2016. Structural deep network embedding. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 1225--1234. Google ScholarDigital Library
- Xiao Wang, Peng Cui, Jing Wang, Jian Pei, Wenwu Zhu, and Shiqiang Yang . 2017. Community Preserving Network Embedding.. In AAAI. 203--209.Google Scholar
- Paul J Werbos . 1990. Backpropagation through time: what it does and how to do it. Proc. IEEE Vol. 78, 10 (1990), 1550--1560.Google ScholarCross Ref
- Reza Zafarani and Huan Liu . 2009. Social computing data repository at ASU.Google Scholar
- Ziwei Zhang, Peng Cui, Jian Pei, Xiao Wang, and Wenwu Zhu . 2017. TIMERS: Error-Bounded SVD Restart on Dynamic Networks. arXiv preprint arXiv:1711.09541 (2017).Google Scholar
- Dingyuan Zhu, Peng Cui, Ziwei Zhang, Jian Pei, and Wenwu Zhu . 2018. High-order Proximity Preserved Embedding For Dynamic Networks. IEEE Transactions on Knowledge and Data Engineering (2018).Google ScholarDigital Library
Index Terms
- Deep Recursive Network Embedding with Regular Equivalence
Recommendations
Structural Deep Network Embedding
KDD '16: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data MiningNetwork embedding is an important method to learn low-dimensional representations of vertexes in networks, aiming to capture and preserve the network structure. Almost all the existing network embedding methods adopt shallow models. However, since the ...
Logical characterizations of regular equivalence in weighted social networks
Social network analysis is a methodology used extensively in social science. Classical social networks can only represent the qualitative relationships between actors, but weighted social networks can describe the degrees of connection between actors. ...
Co-Regularized Deep Multi-Network Embedding
WWW '18: Proceedings of the 2018 World Wide Web ConferenceNetwork embedding aims to learn a low-dimensional vector representation for each node in the social and information networks, with the constraint to preserve network structures. Most existing methods focus on single network embedding, ignoring the ...
Comments