skip to main content
10.1145/3336191.3371781acmconferencesArticle/Chapter ViewAbstractPublication PageswsdmConference Proceedingsconference-collections
research-article

Initialization for Network Embedding: A Graph Partition Approach

Published: 22 January 2020 Publication History

Abstract

Network embedding has been intensively studied in the literature and widely used in various applications, such as link prediction and node classification. While previous work focus on the design of new algorithms or are tailored for various problem settings, the discussion of initialization strategies in the learning process is often missed. In this work, we address this important issue of initialization for network embedding that could dramatically improve the performance of the algorithms on both effectiveness and efficiency. Specifically, we first exploit the graph partition technique that divides the graph into several disjoint subsets, and then construct an abstract graph based on the partitions. We obtain the initialization of the embedding for each node in the graph by computing the network embedding on the abstract graph, which is much smaller than the input graph, and then propagating the embedding among the nodes in the input graph. With extensive experiments on various datasets, we demonstrate that our initialization technique significantly improves the performance of the state-of-the-art algorithms on the evaluations of link prediction and node classification by up to 7.76% and 8.74% respectively. Besides, we show that the technique of initialization reduces the running time of the state-of-the-arts by at least 20%.

References

[1]
James Bergstra, Ré mi Bardenet, Yoshua Bengio, and Balá zs Ké gl. 2011. Algorithms for Hyper-Parameter Optimization. In Advances in Neural Information Processing Systems 24: NIPS 2011, Granada, Spain. 2546--2554.
[2]
Aydin Bulucc, Henning Meyerhenke, Ilya Safro, Peter Sanders, and Christian Schulz. 2016. Recent Advances in Graph Partitioning. In Algorithm Engineering - Selected Results and Surveys. 117--158.
[3]
HongYun Cai, Vincent W. Zheng, and Kevin Chen-Chuan Chang. 2017. A Comprehensive Survey of Graph Embedding: Problems, Techniques and Applications. CoRR, Vol. abs/1709.07604 (2017). http://arxiv.org/abs/1709.07604
[4]
Shaosheng Cao, Wei Lu, and Qiongkai Xu. 2015. GraRep: Learning Graph Representations with Global Structural Information. In Proceedings of the 24th ACM International Conference on Information and Knowledge Management, CIKM 2015, Melbourne, VIC, Australia, October 19 - 23, 2015 . 891--900.
[5]
Haochen Chen, Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2018a. A Tutorial on Network Embeddings. CoRR, Vol. abs/1808.02590 (2018).
[6]
Haochen Chen, Bryan Perozzi, Yifan Hu, and Steven Skiena. 2018b. HARP: Hierarchical Representation Learning for Networks. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, USA, February 2--7, 2018 .
[7]
Peng Cui, Xiao Wang, Jian Pei, and Wenwu Zhu. 2017. A Survey on Network Embedding. CoRR, Vol. abs/1711.08752 (2017).
[8]
Yoav Goldberg and Omer Levy. 2014. word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method. CoRR, Vol. abs/1402.3722 (2014). arxiv: 1402.3722 http://arxiv.org/abs/1402.3722
[9]
Palash Goyal and Emilio Ferrara. 2018. Graph embedding techniques, applications, and performance: A survey. Knowl.-Based Syst., Vol. 151 (2018), 78--94.
[10]
Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable Feature Learning for Networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13--17, 2016. 855--864.
[11]
William L. Hamilton, Rex Ying, and Jure Leskovec. 2017a. Representation Learning on Graphs: Methods and Applications. IEEE Data Eng. Bull., Vol. 40, 3 (2017), 52--74.
[12]
William L. Hamilton, Zhitao Ying, and Jure Leskovec. 2017b. Inductive Representation Learning on Large Graphs. In Advances in Neural Information Processing Systems 30: NIPS 2017, Long Beach, CA, USA. 1025--1035.
[13]
George Karypis and Vipin Kumar. 1995. Analysis of Multilevel Graph Partitioning. In Proceedings Supercomputing, San Diego, CA, USA, December 4--8, 1995. 29.
[14]
George Karypis and Vipin Kumar. 1998. A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs. SIAM J. Scientific Computing, Vol. 20, 1 (1998), 359--392.
[15]
Jure Leskovec, Deepayan Chakrabarti, Jon M. Kleinberg, and Christos Faloutsos. 2005. Realistic, Mathematically Tractable Graph Generation and Evolution, Using Kronecker Multiplication. In Knowledge Discovery in Databases: PKDD 2005, Porto, Portugal, October 3--7, 2005, Proceedings. 133--145.
[16]
Jure Leskovec and Rok Sosivc. 2016. SNAP: A General-Purpose Network Analysis and Graph-Mining Library. ACM Transactions on Intelligent Systems and Technology (TIST), Vol. 8, 1 (2016), 1.
[17]
Jiongqian Liang, Saket Gurukar, and Srinivasan Parthasarathy. 2018. MILE: A Multi-Level Framework for Scalable Graph Embedding. CoRR, Vol. abs/1802.09612 (2018).
[18]
Yao Ma, Zhaochun Ren, Ziheng Jiang, Jiliang Tang, and Dawei Yin. 2018. Multi-Dimensional Network Embedding with Hierarchical Structure. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, WSDM 2018, Marina Del Rey, CA, USA, February 5--9, 2018 . 387--395.
[19]
Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In Advances in Neural Information Processing Systems 26: NIPS 2013, Nevada, United States. 3111--3119.
[20]
Dmytro Mishkin and Jiri Matas. 2015. All you need is a good init. CoRR, Vol. abs/1511.06422 (2015). arxiv: 1511.06422 http://arxiv.org/abs/1511.06422
[21]
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: online learning of social representations. In The 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '14, New York, NY, USA - August 24 - 27, 2014. 701--710.
[22]
Jiezhong Qiu, Yuxiao Dong, Hao Ma, Jian Li, Kuansan Wang, and Jie Tang. 2018. Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, WSDM 2018, Marina Del Rey, CA, USA, February 5--9, 2018 . 459--467.
[23]
Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. LINE: Large-scale Information Network Embedding. In Proceedings of the 24th International Conference on World Wide Web, WWW 2015, Florence, Italy, May 18--22, 2015 . 1067--1077.
[24]
Robert Endre Tarjan. 1974. A Note on Finding the Bridges of a Graph. Inf. Process. Lett., Vol. 2, 6 (1974), 160--161.
[25]
Ke Tu, Peng Cui, Xiao Wang, Fei Wang, and Wenwu Zhu. 2018. Structural Deep Embedding for Hyper-Networks. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, USA, February 2--7, 2018 .
[26]
Michael D. Vose. 1991. A Linear Algorithm For Generating Random Numbers With a Given Distribution. IEEE Trans. Software Eng., Vol. 17, 9 (1991), 972--975.
[27]
Daixin Wang, Peng Cui, and Wenwu Zhu. 2016. Structural Deep Network Embedding. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13--17, 2016. 1225--1234.
[28]
Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Y. Chang. 2015. Network Representation Learning with Rich Text Information. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Argentina, July 25--31, 2015. 2111--2117.
[29]
Daokun Zhang, Jie Yin, Xingquan Zhu, and Chengqi Zhang. 2018. Network Representation Learning: A Survey. CoRR, Vol. abs/1801.05852 (2018).

Cited By

View all
  • (2024)Node Embedding Preserving Graph SummarizationACM Transactions on Knowledge Discovery from Data10.1145/364950518:6(1-19)Online publication date: 12-Apr-2024
  • (2024)Information Diffusion Meets Invitation MechanismCompanion Proceedings of the ACM on Web Conference 202410.1145/3589335.3648337(383-392)Online publication date: 13-May-2024
  • (2024)PlatoD2GL: An Efficient Dynamic Deep Graph Learning System for Graph Neural Network Training on Billion-Scale Graphs2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00191(2421-2434)Online publication date: 13-May-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
WSDM '20: Proceedings of the 13th International Conference on Web Search and Data Mining
January 2020
950 pages
ISBN:9781450368223
DOI:10.1145/3336191
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 January 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. graph partition
  2. hyperparameter learning
  3. initialization
  4. network embedding

Qualifiers

  • Research-article

Conference

WSDM '20

Acceptance Rates

Overall Acceptance Rate 498 of 2,863 submissions, 17%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)16
  • Downloads (Last 6 weeks)3
Reflects downloads up to 25 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Node Embedding Preserving Graph SummarizationACM Transactions on Knowledge Discovery from Data10.1145/364950518:6(1-19)Online publication date: 12-Apr-2024
  • (2024)Information Diffusion Meets Invitation MechanismCompanion Proceedings of the ACM on Web Conference 202410.1145/3589335.3648337(383-392)Online publication date: 13-May-2024
  • (2024)PlatoD2GL: An Efficient Dynamic Deep Graph Learning System for Graph Neural Network Training on Billion-Scale Graphs2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00191(2421-2434)Online publication date: 13-May-2024
  • (2024) E 2 GCL: Efficient and Expressive Contrastive Learning on Graph Neural Networks 2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00071(859-873)Online publication date: 13-May-2024
  • (2023)Bridged-GNN: Knowledge Bridge Learning for Effective Knowledge TransferProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614796(99-109)Online publication date: 21-Oct-2023
  • (2023)Predicting the Silent Majority on Graphs: Knowledge Transferable Graph Neural NetworkProceedings of the ACM Web Conference 202310.1145/3543507.3583287(274-285)Online publication date: 30-Apr-2023
  • (2023)MM-GNN: Mix-Moment Graph Neural Network towards Modeling Neighborhood Feature DistributionProceedings of the Sixteenth ACM International Conference on Web Search and Data Mining10.1145/3539597.3570457(132-140)Online publication date: 27-Feb-2023
  • (2023)Simplifying Graph-based Collaborative Filtering for RecommendationProceedings of the Sixteenth ACM International Conference on Web Search and Data Mining10.1145/3539597.3570451(60-68)Online publication date: 27-Feb-2023
  • (2023)Interpretable Signed Link Prediction With Signed Infomax Hyperbolic GraphIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2021.313903535:4(3991-4002)Online publication date: 1-Apr-2023
  • (2023)Efficient Network Representation Learning via Cluster SimilarityData Science and Engineering10.1007/s41019-023-00222-x8:3(279-291)Online publication date: 12-Sep-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media