skip to main content
10.1145/3336191.3371843acmconferencesArticle/Chapter ViewAbstractPublication PageswsdmConference Proceedingsconference-collections
research-article

A Structural Graph Representation Learning Framework

Published: 22 January 2020 Publication History

Abstract

The success of many graph-based machine learning tasks highly depends on an appropriate representation learned from the graph data. Most work has focused on learning node embeddings that preserve proximity as opposed to structural role-based embeddings that preserve the structural similarity among nodes. These methods fail to capture higher-order structural dependencies and connectivity patterns that are crucial for structural role-based applications such as visitor stitching from web logs. In this work, we formulate higher-order network representation learning and describe a general framework called HONE for learning such structural node embeddings from networks via the subgraph patterns (network motifs, graphlet orbits/positions) in a nodes neighborhood. A general diffusion mechanism is introduced in HONE along with a space-efficient approach that avoids explicit construction of the k-step motif-based matrices using a k-step linear operator. Furthermore, HONE is shown to be fast and efficient with a worst-case time complexity that is nearly-linear in the number of edges. The experiments demonstrate the effectiveness of HONE for a number of important tasks including link prediction and visitor stitching from large web log data.

References

[1]
Nesreen K. Ahmed, Jennifer Neville, Ryan A. Rossi, and Nick Duffield. 2015. Efficient Graphlet Counting for Large Networks. In ICDM. 10.
[2]
Nesreen K. Ahmed, Ryan A. Rossi, Theodore L. Willke, and Rong Zhou. 2017. Edge Role Discovery via Higher-Order Structures . PAKDD. 291--303.
[3]
Nesreen K. Ahmed, Ryan A. Rossi, Rong Zhou, John Boaz Lee, Xiangnan Kong, Theodore L. Willke, and Hoda Eldardiry. 2018. Learning Role-based Graph Embeddings. In arXiv:1802.02896 .
[4]
Nesreen K. Ahmed, Theodore L. Willke, and Ryan A. Rossi. 2016. Estimation of Local Subgraph Counts. In BigData. 586--595.
[5]
Alex Arenas, Alberto Fernandez, Santo Fortunato, and Sergio Gomez. 2008. Motif-based communities in complex networks. J Phys A Math Theor., Vol. 41, 22 (2008).
[6]
Austin R Benson, David F Gleich, and Jure Leskovec. 2016. Higher-order organization of complex networks. Science, Vol. 353, 6295 (2016), 163--166.
[7]
Aleksandar Bojchevski and Stephan Günnemann. 2017. Deep Gaussian Embedding of Attributed Graphs: Unsupervised Inductive Learning via Ranking. arXiv:1707.03815 (2017).
[8]
S.P. Borgatti and M.G. Everett. 1992. Notions of position in social network analysis. Sociological methodology, Vol. 22, 1 (1992), 1--35.
[9]
Shaosheng Cao, Wei Lu, and Qiongkai Xu. 2015. GraRep: Learning graph representations with global structural information. In CIKM. ACM, 891--900.
[10]
Sandro Cavallari, Vincent W Zheng, Hongyun Cai, Kevin Chen-Chuan Chang, and Erik Cambria. 2017. Learning community embedding with community detection and node embedding on graphs. In CIKM . 377--386.
[11]
Shiyu Chang, Wei Han, Jiliang Tang, Guo-Jun Qi, Charu C Aggarwal, and Thomas S Huang. 2015. Heterogeneous network embedding via deep architectures. In SIGKDD . 119--128.
[12]
Pin-Yu Chen, Baichuan Zhang, Mohammad Al Hasan, and Alfred O Hero. 2015. Incremental method for spectral clustering of increasing orders. In arXiv:1512.07349 .
[13]
Winnie Cheng, Chris Greaves, and Martin Warren. 2006. From n-gram to skipgram to concgram. Int. J. of Corp. Linguistics, Vol. 11, 4 (2006), 411--433.
[14]
Michael Collins, Sanjoy Dasgupta, and Robert E Schapire. 2002. A generalization of principal components analysis to the exponential family. In NIPS . 617--624.
[15]
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In NIPS . 3844--3852.
[16]
Yuxiao Dong, Nitesh V Chawla, and Ananthram Swami. 2017. metapath2vec: Scalable Representation Learning for Heterogeneous Networks. In SIGKDD .
[17]
S. Fortunato. 2010. Community detection in graphs. Phy. Rep., Vol. 486, 3--5 (2010).
[18]
Gene H Golub and Charles F Van Loan. 2012. Matrix computations .JHU Press.
[19]
Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. SIGKDD. 855--864.
[20]
Nathan Halko, Per-Gunnar Martinsson, and Joel A Tropp. 2011. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM review, Vol. 53, 2 (2011), 217--288.
[21]
Mikael Henaff, Joan Bruna, and Yann LeCun. 2015. Deep convolutional networks on graph-structured data. arXiv:1506.05163 (2015).
[22]
Xiao Huang, Jundong Li, and Xia Hu. 2017a. Accelerated attributed network embedding. In SDM .
[23]
Xiao Huang, Jundong Li, and Xia Hu. 2017b. Label informed attributed network embedding. In WSDM .
[24]
Jingu Kim, Yunlong He, and Haesun Park. 2014. Algorithms for nonnegative matrix and tensor factorizations: A unified view based on block coordinate descent framework. Journal of Global Optimization, Vol. 58, 2 (2014), 285--319.
[25]
Sungchul Kim, Nikhil Kini, Jay Pujara, Eunyee Koh, and Lise Getoor. 2017. Probabilistic visitor stitching on cross-device web logs. In WWW. 1581--1589.
[26]
Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In ICLR .
[27]
Jiongqian Liang, Peter Jacobs, and Srinivasan Parthasarathy. 2017. SEANO: Semi-supervised Embedding in Attributed Networks with Outliers. In arXiv:1703.08100 .
[28]
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In ICLR Workshop . 10.
[29]
Cameron Musco and Christopher Musco. 2015. Randomized block krylov methods for stronger and faster approximate singular value decomposition. In Advances in Neural Information Processing Systems . 1396--1404.
[30]
Giang Hoang Nguyen, John Boaz Lee, Ryan A. Rossi, Nesreen K. Ahmed, Eunyee Koh, and Sungchul Kim. 2018. Continuous-Time Dynamic Network Embeddings. In WWW BigNet .
[31]
Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov. 2016. Learning Convolutional Neural Networks for Graphs. arXiv:1605.05273 .
[32]
Jinoh Oh, Wook-Shin Han, Hwanjo Yu, and Xiaoqian Jiang. 2015. Fast and robust parallel SGD matrix factorization. SIGKDD. ACM, 865--874.
[33]
Mingdong Ou, Peng Cui, Jian Pei, Ziwei Zhang, and Wenwu Zhu. 2016. Asymmetric transitivity preserving graph embedding. In SIGKDD . 1105--1114.
[34]
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In SIGKDD . 701--710.
[35]
Natavs a Prvz ulj. 2007. Biological network comparison using graphlet degree distribution. Bioinfo., Vol. 23, 2 (2007), e177--e183.
[36]
Mahmudur Rahman, Tanay Kumar Saha, Mohammad Al Hasan, Kevin S Xu, and Chandan K Reddy. 2018. DyLink2Vec: Effective Feature Representation for Link Prediction in Dynamic Networks. arXiv:1804.05755 (2018).
[37]
Leonardo F.R. Ribeiro, Pedro H.P. Saverese, and Daniel R. Figueiredo. 2017. Struc2Vec: Learning Node Representations from Structural Identity. In SIGKDD .
[38]
Vladimir Rokhlin, Arthur Szlam, and Mark Tygert. 2009. A randomized algorithm for principal component analysis. SIAM J. Matrix Anal. Appl., Vol. 31, 3 (2009).
[39]
Ryan A. Rossi and Nesreen K. Ahmed. 2015a. The Network Data Repository with Interactive Graph Analytics and Visualization. In AAAI . 4292--4293. http://networkrepository.com
[40]
Ryan A. Rossi and Nesreen K. Ahmed. 2015b. Role Discovery in Networks. Transactions on Knowledge and Data Engineering, Vol. 27, 4 (April 2015), 1112--1131.
[41]
Ryan A. Rossi, Di Jin, Sungchul Kim, Nesreen K. Ahmed, Danai Koutra, and John Boaz Lee. 2019. From Community to Role-based Graph Embeddings. In arXiv:1908.08572 .
[42]
Ryan A. Rossi, Luke K. McDowell, David W. Aha, and Jennifer Neville. 2012. Transforming graph data for statistical relational learning. Journal of Artificial Intelligence Research, Vol. 45, 1 (2012), 363--441.
[43]
Ryan A. Rossi, Rong Zhou, and Nesreen K. Ahmed. 2018. Deep Inductive Graph Representation Learning. In IEEE Transactions on Knowledge and Data Engineering (TKDE). 14.
[44]
Tanay Kumar Saha, Thomas Williams, Mohammad Al Hasan, Shafiq Joty, and Nicholas K Varberg. 2018. Models for Capturing Temporal Smoothness in Evolving Networks for Learning Latent Representation of Nodes. In arXiv:1804.05816 .
[45]
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2009. The graph neural network model. TNNLS, Vol. 20, 1 (2009), 61--80.
[46]
Chuan Shi, Xiangnan Kong, Yue Huang, S Yu Philip, and Bin Wu. 2014. HeteSim: A General Framework for Relevance Measure in Heterogeneous Networks . TKDE, Vol. 26, 10 (2014), 2479--2492.
[47]
Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. LINE: Large-scale Information Network Embedding . WWW. 1067--1077.
[48]
Lei Tang and Huan Liu. 2011. Leveraging social media networks for classification. Data Mining and Knowledge Discovery, Vol. 23, 3 (2011), 447--478.
[49]
S Vichy N Vishwanathan, Nicol N Schraudolph, Risi Kondor, and Karsten M Borgwardt. 2010. Graph kernels. JMLR, Vol. 11 (2010), 1201--1242.
[50]
Daixin Wang, Peng Cui, and Wenwu Zhu. 2016. Structural deep network embedding. In SIGKDD. 1225--1234.
[51]
Jason Weston, Frédéric Ratle, and Ronan Collobert. 2008. Deep learning via semi-supervised embedding. In ICML. 1168--1175.
[52]
Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Y Chang. 2015. Network Representation Learning with Rich Text Information. In IJCAI .
[53]
Zhilin Yang, William W Cohen, and Ruslan Salakhutdinov. 2016. Revisiting semi-supervised learning with graph embeddings. arXiv:1603.08861 (2016).
[54]
Hyokun Yun, Hsiang-Fu Yu, Cho-Jui Hsieh, SVN Vishwanathan, and Inderjit Dhillon. 2014. NOMAD: Non-locking, stOchastic Multi-machine algorithm for Asynchronous and Decentralized matrix completion. VLDB, Vol. 7, 11 (2014), 975--986.

Cited By

View all
  • (2025)An analysis of graph neural networks for fake review detection: A systematic literature reviewNeurocomputing10.1016/j.neucom.2025.129341(129341)Online publication date: Jan-2025
  • (2024)Representation Learning of Temporal Graphs with Structural RolesProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671854(654-665)Online publication date: 25-Aug-2024
  • (2024)Representation Learning on Heterostructures via Heterogeneous Anonymous WalksIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.323400535:7(9538-9552)Online publication date: Jul-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
WSDM '20: Proceedings of the 13th International Conference on Web Search and Data Mining
January 2020
950 pages
ISBN:9781450368223
DOI:10.1145/3336191
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 January 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. graphlets
  2. higher-order node embeddings
  3. network motifs
  4. network representation learning
  5. role discovery
  6. role-based embeddings
  7. roles
  8. structural embeddings
  9. structural node embeddings
  10. structural similarity

Qualifiers

  • Research-article

Conference

WSDM '20

Acceptance Rates

Overall Acceptance Rate 498 of 2,863 submissions, 17%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)58
  • Downloads (Last 6 weeks)3
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)An analysis of graph neural networks for fake review detection: A systematic literature reviewNeurocomputing10.1016/j.neucom.2025.129341(129341)Online publication date: Jan-2025
  • (2024)Representation Learning of Temporal Graphs with Structural RolesProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671854(654-665)Online publication date: 25-Aug-2024
  • (2024)Representation Learning on Heterostructures via Heterogeneous Anonymous WalksIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.323400535:7(9538-9552)Online publication date: Jul-2024
  • (2024)A Structure-Aware Graph Representation Learning Optimization2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650572(1-9)Online publication date: 30-Jun-2024
  • (2024)A deep feature interaction and fusion model for fake review detection: Advocating heterogeneous graph convolutional networkNeurocomputing10.1016/j.neucom.2024.128097598(128097)Online publication date: Sep-2024
  • (2024)Unified node, edge and motif learning networks for graphsEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.109354138(109354)Online publication date: Dec-2024
  • (2024)Graph contrastive learning with cross-encoder for community discoveryApplied Intelligence10.1007/s10489-024-05287-354:2(2211-2224)Online publication date: 1-Feb-2024
  • (2023)Temporal SIR-GN: Efficient and Effective Structural Representation Learning for Temporal GraphsProceedings of the VLDB Endowment10.14778/3598581.359858316:9(2075-2089)Online publication date: 10-Jul-2023
  • (2023)A Survey on Graph Representation Learning MethodsACM Transactions on Intelligent Systems and Technology10.1145/363351815:1(1-55)Online publication date: 28-Nov-2023
  • (2023)Generative hypergraph models and spectral embeddingScientific Reports10.1038/s41598-023-27565-913:1Online publication date: 11-Jan-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media