Dongjie Li
ABSTRACT
Graph embedding is the resultful method to map the graph in the low-dimensional vector space. Now most existing embedding methods to learn nodes representations mainly focus on obtaining nodes adjacent and feature information, but they ignore the state that there is also semantic information between nodes. Therefore, it is proposed a graph embedding method, which introduces point mutual information to compute the semantic similarity between nodes, the basic idea is to count the probability of two nodes appearing simultaneously in a sentence. And it learns representations by modeling the sum of the squares of the difference between point-wise mutual information and the inner product of node vectors, and theoretically shows that using point mutual information can also obtain a log-linear relationship between graph topological by leveraging the invariance property of difference between nodes. Finally, the study selects 5 social networks datasets for node classifications, clustering tasks, and compares them with 6 graph embedding methods and 4 methods based on graph neural network, the result demonstrates that the direct method does not negatively impact accuracy on many downstream applications, and outperforms all the baseline methods. In addition, the computation complexity of our method is lower than the worst-case.
- Will H, Zhitao Ying, Jure L. Inductive representation learning on large graphs// In 31st Conference on Neural Information Processing Systems (NIPS), Long Beach, CA, USA, 2017:1025–1035.Google Scholar
- Fenyu Hu, Yanqiao, ShuWu,Liang wang,Tieniu Tan.Hierarchical Graph Convolutional Networks for Semi-supervised Node Classification.arXiv:1902.06667v4. 2017.Google Scholar
- Kipf and Welling Thomas N Kipf and Max Welling. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308, 2016.Google Scholar
- Shirui Pan, Ruiqi Hu, Guodong Long, Jing Jiang, Lina Yao, and Chengqi Zhang. Adversarially regularized graph autoencoder for graph embedding. In International Joint Conference on Artifificial Intelligence, pages 2609–2615, 2018.Google Scholar
- Qiu J, Dong Y, Ma H, Netsmf: Large-scale network embedding as sparse matrix factorization//Proceedings of the 28th international conference on World Wide Web. San Francisco, USA, 2019: 1509-1520.Google Scholar
- Zhang J, Dong Y, Wang Y, ProNE: fast and scalable network representation learning//Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China. 2019: 4278-4284.Google Scholar
- Yang C, Liu Z, Zhao D, Network representation learning with rich text information//Twenty-Fourth International Joint Conference on Artificial Intelligence. Buenos Aires, Argentina, 2015: 2111-2117.Google Scholar
- B. Perozzi, R. Al-Rfou, and S. Skiena, Deepwalk: Online learning of social representations, in Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2014:701–710.Google Scholar
- A. Grover and J. Leskovec, node2vec: Scalable feature learning for networks,” in Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and datamining.ACM,2016:855-864.Google Scholar
- Bryan Perozzi, Vivek Kulkarni, Haochen Chen, and Steven Skiena. Don't Walk, Skip! : online learning of multi-scale network embeddings. //In Proceedings of the 2017 IEEEACM International.Conference on Advances in Social Networks Analysis and Mining.2017: 258–265.Google Scholar
- Benedek Rozemberczki and Rik Sarkar. Fast Sequence-Based Embedding with Diffusion Graphs. In International Workshop on Complex Networks. Springer, 2018:99–107.Google ScholarCross Ref
- C. Yang, Z. Liu, D. Zhao, M. Sun, and E. Y. Chang, “Network representation learning with rich text information,” //in Twenty-Furth International Joint Conference on Artificial Intelligence (IJCAI), Buenos Aires, Argentina,2015:2111–2117.Google Scholar
- L. Tang and H. Liu, “Relational learning via latent social dimensions,” // Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2009:817–826.Google Scholar
- Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. Line: Large scale information network embedding. In Proceedings of the 24th international conference on world wide web, pages 1067–1077, Florence, Italy, May 2015. ACM.Google ScholarDigital Library
- Hamilton, W., Ying, Z., and Leskovec, J. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, pp. 1024–1034, 2017.Google ScholarDigital Library
- Petar Veliˇckovi´c, William Fedus, William L Hamilton, Pietro Li`o, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. In International Conference on Learning Representations, 2019.Google Scholar
- Ribeiro, L., Saverese, P., and Figueiredo, D. Struc2vec: Learning node representations from structural identity. In International Conference on Knowledge Discovery and Data Mining, pp 385394, 2017.Google ScholarDigital Library
- You, J., Ying, R., and Leskovec, J. Position-aware graph neural networks. In International Conference on Machine Learning, pp. 7134–7143, 2019.Google Scholar
- Chun Wang, Shirui Pan, Guodong Long, Xingquan Zhu, and Jing Jiang. Mgae: Marginalized graph autoencoder for graph clustering. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pages 889–898. ACM, 2017.Google Scholar
- Garcia Duran, A. and Niepert, M. Learning graph representations with embedding propagation. In Advances in Neural Information Processing Systems, pp. 5119–5130. 2017.Google Scholar
- Park, J., Lee, M., Chang, H. J., Lee, K., and Choi, J. Y. Symmetric graph convolutional autoencoder for unsupervised graph representation learning. In International Conference on Computer Vision, pp. 6519–6528, 2019.Google ScholarCross Ref
- Petar Veliˇckovi´c, William Fedus, William L Hamilton, Pietro Li`o, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. In International Conference on Learning Representations, 2019.Google Scholar
- Tian, Y., Krishnan, D., and Isola, P. Contrastive multiview coding. arXiv preprint arXiv:1906.05849, 2019.Google Scholar
- Li, Y., Gu, C., Dullien, T., Vinyals, O., and Kohli, P. Graph matching networks for learning the similarity of graph structured objects. In International Conference on Machine Learning, pp. 3835–3845, 2019.Google Scholar
- Sun, F.-Y., Hoffman, J., Verma, V., and Tang, J. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In International Conference on Learning Representations, 2020.Google Scholar
- Hjelm, R. D., Fedorov, A., Lavoie-Marchildon, S., Grewal, K., Bachman, P., Trischler, A., and Bengio, Y. Learning deep representations by mutual information estimation and maximization.Google Scholar
- Sun, F.-Y., Hoffman, J., Verma, V., and Tang, J. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In International Conference on Learning Representations, 2020.Google Scholar
- O. Levy and Y. Goldberg, “Neural word embedding as implicit matrix factorization,” in Advances in neural information processing systems, 2014, pp. 2177–2185.Google Scholar
- J. Pennington, R. Socher, and C. Manning, “Glove: Global vectors for word representation,” in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, pp. 1532–1543.Google ScholarCross Ref
- Chenyi Zhuang,Qiang Ma.Dual Graph Convolutional Networks for Graph-Based Semi-Supervised Classification.//the 2018 World Wide Web Conference.2018:499-508.Google Scholar
- Jure Leskovec and Andrej Krevl. SNAP Datasets: Stanford Large Network Dataset Collection. http://snap.stanford.edu/data.2014.Google Scholar
- Benedek Rozemberczki, Carl Allen, and Rik Sarkar. Multi-scale Attributed Node Embedding. arXiv preprint arXiv:1909.1302.2019.Google Scholar
- Benedek Rozemberczki, Oliver Kiss, and Rik Sarkar. Karate Club: An API Oriented Open-Source Python Framework for Unsupervised Learning on Graphs.arXiv:2003.04819v3 [cs.LG] 7 Aug 2020.Google Scholar
Index Terms
- Network Embedding Method Based on Semantic Information
Recommendations
Embedding into Bipartite Graphs
The conjecture of Bollobás and Komlós, recently proved by Böttcher, Schacht, and Taraz [Math. Ann., 343 (2009), pp. 175-205], implies that for any $\gamma>0$, every balanced bipartite graph on $2n$ vertices with bounded degree and sublinear bandwidth ...
Separator-based graph embedding into multidimensional grids with small edge-congestion
We study the problem of embedding a guest graph with minimum edge-congestion into a multidimensional grid with the same size as that of the guest graph. Based on a well-known notion of graph separators, we show that an embedding with a smaller edge-...
Discrete embedding for attributed graphs
AbstractAttributed graphs refer to graphs where both node links and node attributes are observable for analysis. Attributed graph embedding enables joint representation learning of node links and node attributes. Different from classical graph embedding ...
Comments