Dongjie Li
Abstract
With the success of Graph Neural Network (GNN) in network data, some GNN-based representation learning methods for networks have emerged recently. Variational Graph Autoencoder (VGAE) is a basic GNN framework for network representation. Its purpose is to well preserve the topology and node attribute information of the network to learn node representation, but it only reconstructs network topology, and does not consider the reconstruction of node features. This strategy will make node representation can not well reserve node features information, impairing the ability of the VGAE method to learn higher quality representations. To solve this problem, we arise a new network representation method to improve the VGAE method for well retaining both node features and network structure information. The method utilizes adversarial mutual information learning to maximize the mutual information (MI) of node features and node representations during the encoding process of the variational autoencoder, which forces the variational encoder to get the representation containing the most informative node features. The method consists of three parts: a variational graph autoencoder includes a variational encoder (MI generator (G)) and a decoder, a positive MI sample module (maximizing MI module), and an MI discriminator (D). Furthermore, we explain why maximizing MI between node features and node representation can reconstruct node attributes. Finally, we conduct experiments on seven public representative datasets for nodes classification, nodes clustering, and graph visualization tasks. Experimental results demonstrate that the proposed algorithm significantly outperforms current popular network representation algorithms on these tasks. The best improvement is 17.13% than the VGAE method.
- [1] . 2006. Community detection as an inference problem. Physical Review E74, 3 (2006), 035–102.Google Scholar
- [2] . 2017. Semi-supervised classification with graph convolutional networks. In Proceedings of the ICLR.Google Scholar
- [3] . 2018. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the SIGKDD Graph Convolutional Neural Networks for Web-scale Recommender Systems. 974–983.Google ScholarDigital Library
- [4] . 2015. GraRep: Learning graph representations with global structural information. In Proceedings of the GraRep: Learning Graph Representations with Global Structural Information. 891–900.Google ScholarDigital Library
- [5] . 2018. Community detection in attributed graphs: An embedding approach. In Proceedings of the AAAIConference on Artificial Intelligence. 338–345.Google ScholarCross Ref
- [6] . 2015. Network representation learning with rich text information. In Proceedings of the 24th International Joint Conference on Artificial Intelligence. 2111–2117.Google ScholarDigital Library
- [7] . 2019. Netsmf: Large-scale network embedding as sparse matrix factorization. In Proceedings of the 28th International Conference on World Wide Web. San Francisco. 1509–1520.Google ScholarDigital Library
- [8] . 2019. ProNE: fast and scalable network representation learning. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao. 4278–4284.Google ScholarCross Ref
- [9] . 2014. Deep walk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 701–710.Google ScholarDigital Library
- [10] . 2016. Node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 855–864.Google ScholarDigital Library
- [11] . 2015. Line: Large scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web. Florence, Italy. 1067–1077.Google ScholarDigital Library
- [12] . 2017. Don’t walk, skip! Online learning of multi-scale network embeddings. In Proceedings of the 2017 IEEE/ACM International Conference.Google Scholar
- [13] . 2018. Fast sequence-based embedding with diffusion graphs. In Proceedings of the International Workshop on Complex Networks. Springer (2018). 99–107.Google Scholar
- [14] . 2017. Mgae: Marginalized graph autoencoder for graph clustering. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 889–898.Google ScholarDigital Library
- [15] . 2017. Struc2vec: Learning node representations from structural identity. In Proceedings of the International Conference on Knowledge Discovery and Data Mining. 385–394.Google ScholarDigital Library
- [16] . 2016. Structural deep network embedding. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery \(and\) Data Mining. 1225–1234.Google ScholarDigital Library
- [17] . 2018. Graph attention networks. In Proceedings of the International Conference on Learning Representations.Google Scholar
- [18] . 2017. Inductive representation learning on large graphs. In Proceedings of the Advances in Neural Information Processing Systems 1024–1034.Google Scholar
- [19] . 2016. Variational graph auto-encoders. In Proceedings of the NIPS Workshop on Bayesian Deep Learning.Google Scholar
- [20] . 2018. Adversarially regularized graph autoencoder for graph embedding. In Proceedings of the IJCAI. 2609–2615.Google ScholarCross Ref
- [21] . 2018. Learning deep network representations with adversarially regularized autoencoders. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. London, United Kingdom 2663–2671.Google ScholarDigital Library
- [22] . 2014. Auto-Encoding Variational Bayes. arXiv:1312.6114v10. Retrieved from https://arxiv.org/abs/1312.6114v10Google Scholar
- [23] . 2019. Position-aware graph neural networks. In Proceedings of the International Conference on Machine Learning. 7134–7143.Google Scholar
- [24] . 2005. Shortest-path kernels on graphs. In Proceedings of the International Conference on Data Mining.Google ScholarDigital Library
- [25] . 2009. Efficient graphlet kernels for large graph comparison. In Proceedings of the Artificial Intelligence and Statistics. 488–495.Google Scholar
- [26] . 2011. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research. 2539–2561.Google Scholar
- [27] . 2015. Deep graph kernels. In Proceedings of the International Conference on Knowledge Discovery and Data Mining. 1365–1374.Google ScholarDigital Library
- [28] . 2016. The multiscale laplacian graph kernel. In Proceedings of the Advances in Neural Information Processing Systems. 2990–2998.Google Scholar
- [29] . 2019. Symmetric graph convolutional autoencoder for unsupervised graph representation learning. In Proceedings of the International Conference on Computer Vision. 6519–6528.Google ScholarCross Ref
- [30] . 2019. Deep graph infomax. In Proceedings of the International Conference on Learning Representations.Google Scholar
- [31] . 2020. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In Proceedings of the International Conference on Learning Representations.Google Scholar
- [32] . 2019. Learning deep representations by mutual information estimation and maximization. In Proceedings of the International Conference on Learning Representations.Google Scholar
- [33] . 2014. Generative adversarial nets. In Proceedings of the Advances in Neural Information Processing Systems. 2672–2680.Google Scholar
- [34] . 2016. F-GAN: Training generative neural samplers using variational divergence minimization. In Proceedings of the 30th International Conference on Neural Information Processing Systems. 271–279.Google Scholar
- [35] Retrieved from https://spaces.ac.cn/archives/6088.Google Scholar
- [36] . 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Proceedings of the Advances in Neural Information Processing Systems. 384–852.Google Scholar
- [37] . 2017. Geometric deep learning on graphs and manifolds using mixture model cnns. In Proceedings of the Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- [38] . 2017. Accelerated attributed network embedding. In Proceedings of the 2017 SIAM International Conference on Data Mining. Houston, Texas. 633–641.Google Scholar
- [39] . 2018. Attributed social network embedding. IEEE Transactions on Knowledge and Data Engineering. 2257–2270.Google Scholar
- [40] . 2020. Adversarial mutual information learning for network embedding. In Proceedings of the 29th International Joint Conference on Artificial Intelligence.Google ScholarCross Ref
- [41] . 2015. Weekly-shared deep transfer networks for heterogeneous-domain knowledge propagation. ACM International Conference on Multimedia. 35–44.Google Scholar
- [42] . 2016. Generalized deep transfer networks for heterogeneous-domain knowledge propagation. ACM Transactions on Multimedia Computing, Communications, and Applications 12, 4s (2016), 68.Google ScholarDigital Library
- [43] . 2020. BiANE: Bipartite attributed network embedding. International ACM SIGIR Conference on Research and Development in Information Retrieval.Google Scholar
- [44] . 2021. Motif-preserving dynamic attributed network embedding. International World Wide Web Conferences.Google Scholar
- [45] . 2020. SNEQ: Semi-supervised attributed network embedding with attention-based quantisation. In Proceedings of the AAAI Conference on Artificial Intelligence. 4091–4098.Google ScholarCross Ref
Index Terms
- Variational Graph Autoencoder with Adversarial Mutual Information Learning for Network Representation Learning
Recommendations
Adversarial learning based residual variational graph normalized autoencoder for network representation
AbstractWith the success of Graph Neural Networks (GNNs) on non-Euclidean data, some GNN-based approaches to network representation learning have emerged in recent years. Graph autoencoder (GAE) and variational GAE (VGAE) are unsupervised ...
Adversarial correlated autoencoder for unsupervised multi-view representation learning
AbstractTo eliminate the view discrepancy of multi-view data due to different distributions, the key is to learn the common representation for multi-view data in many practical applications. To achieve the end, we propose a novel unsupervised ...
Text-enhanced network representation learning
AbstractNetwork representation learning called NRL for short aims at embedding various networks into low-dimensional continuous distributed vector spaces. Most existing representation learning methods focus on learning representations purely based on the ...
Comments