Abstract
Independent of graph augmentation techniques, graph autoencoders (GAEs) have yielded promising results in the realm of self-supervised learning. However, GAEs tend to over-emphasize proximity information at the expense of structural information, leading to relatively poor performance on some downstream tasks such as node classification. To address this issue, we propose a novel GAE framework via community, named Community aware Masked Graph AutoEncoder(ComMGAE). Since community represents a high-order structure of a graph, characterized by a group of densely connected nodes, ComMGAE can import structural information from community semantics to the self-supervised generative learning on the graph. Firstly, we partition the community structure by using the community detection algorithm and calculate the community strength. Then we introduce a novel community-guided graph masking strategy to learn more about the graph structure during the encoding process. Moreover, we mask and reconstruct both the structure and attribution of the graph and employ a graph neural network as the decoder to enrich learning representations with compressed information. Finally, experimental results on node classification demonstrate that the proposed ComMGAE preserves both the graph topology and semantic information effectively and outperforms other state-of-the-art baselines on a series of benchmarks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ashish, V., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30, pp. 5998–6008 (2017)
Xiao, W., et al.: Heterogeneous Graph Attention Network, The Web Conference, abs/1903.07293.: 2022-2032 (2019)
You, J., Liu, B., Ying, Z., Pande, V., Leskovec, J.: Graph convolutional policy network for OAL-directed molecular graph generation. In: Proceedings of the 32nd International Conference Neural Information Processing. System, pp. 6410–6421 (2018)
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning, vol. 119, pp. 1597–1607 (2020)
He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Computer Vision and Pattern Recognition, pp. 9726–9735 (2020)
You, J., Ying, R., Ren, X., Hamilton, W., Leskovec, J.: GraphRNN: generating realistic graphs with deep auto-regressive models. In: Proceedings of the 35th International Conference on Machine Learning, pp. 5708–5717 (2018)
Girvan, M., Newman, M.E.J.: Community structure in social and biological networks. Proc. Natl. Acad. Sci. 99(12), 7821–7826 (2002)
Petar, V., William, F., Hamilton, W.L., H., Pietro, L., Yoshua, B., Hjelm, R.D.: Deep Graph Infomax. In: International Conference on Learning Representations, abs/1809.10341 (2019)
Jintang, L., et al.: MaskGAE: masked graph modeling meets graph autoencoders, arXiv preprint arXiv:2205.10053 (2022)
Chun, W., Shirui, P., Guodong, L., Xingquan, Z., Jing, J.: MGAE: marginalized graph autoencoder for graph clustering. In: International Conference on Information and Knowledge Management, pp. 889–898 (2017)
Grill, J.-B.,et al.: Bootstrap your own latent: A new approach to self-supervised learning. In: NeurIPS (2020)
Kipf, T.N., Welling, M.: Variational graph auto-encoders. arXiv preprint arXiv:1611.07308 (2016)
Zhenyu, H., et al.: GraphMAE: self-supervised masked graph autoencoders. In: ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 594–604 (2022)
Yanqiao, Z., Yichen, X., Feng, Y., Qiang, L., Shu, W., Liang, W.: Graph Contrastive Learning with Adaptive Augmentation, The Web Conference, abs/2010.14945: 2069-2080 (2021)
Yuning, Y., Tianlong, C., Yongduo, S., Ting, C., Zhangyang, W., Yang, S.: graph contrastive learning with augmentations. In: Conference on Neural Information Processing Systems, vol. 33, pp. 5812–5823 (2020)
Manessi, F., Rozza, A.: Graph-based neural network models with multiple self-supervised auxiliary tasks. Pattern Recognition Letters (2021)
Nix, D.A., Weigend, A.S.: Estimating the mean and variance of the target probability distribution. In: Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN’94), Orlando, FL, USA, 1994, vol. 1, pp. 55–60 (1994). https://doi.org/10.1109/ICNN.1994.374138.
Traag, V.A., Van Dooren, P., Nesterov, Y.: Narrow scope for resolution-limit-free community detection. Phys. Rev. E 84, 016114 (2011). https://doi.org/10.1103/PhysRevE.84.016114
Traag, V.A., Waltman, L. & van Eck, N.J. From Louvain to Leiden: guaranteeing well-connected communities. Sci Rep 9, 5233 (2019). https://doi.org/10.1038/s41598-019-41695-z
Sobolevsky, S., Campari, R., Belyi, A., Ratti, C.: General optimization technique for high-quality community detection in complex networks. Phys. Rev. E 90(1), 012811 (2014)
Zhilin, Y., William, C., Ruslan, S.: Revisiting Semi-Supervised Learning with Graph Embeddings. Computing Research Repository, abs/1603.08861.: 40–48 (2016)
C. Lee, G., Kurt D., B., Steve, L.: CiteSeer: an automatic citation indexing system. In: ACM International Conference on Digital Libraries, pp. 89–98 (1998)
Shchur, L., Mumme, M., Bojchevski, A., Gunnemann, S.: Pitfalls of graph neural network evaluation. CoRR, abs/1811.05868 (2018)
Hu, W., et al.: Open graph benchmark: datasets for machine learning on graphs. In: Conference on Neural Information Processing Systems, vol. 33, pp. 22118–22133 (2020)
Ryan, R., Nesreen, A.: The network data repository with interactive graph analytics and visualization. In: AAAI Conference on Artificial Intelligence, pp. 4292–4293 (2015)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXv preprint arXv:1710.10903 (2017)
Keyulu, X., Weihua, H., Jure, L., Stefanie, J.: How Powerful are Graph Neural Networks ?. In: International Conference on Learning Representations, abs/1810.00826 (2019)
Shantanu, T., Corentin, T., Mohammad Gheshlaghi, A., Rémi, M., Petar, V., Michal, V.: Bootstrapped representation learning on graphs. In: International Conference on Learning Representations, abs/2102.06514 (2021)
Fan-Yun, S., Jordan, H., Vikas, V., Jian, T.: InfoGraph: unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In: International Conference on Learning Representations, abs/1908.01000 (2020)
Jun, X., Lirong, W., Jintao, C., Bozhen, H., Li, S.Z.: SimGRACE: a simple framework for graph contrastive learning without data augmentation. In: The Web Conference, pp. 1070–1079 (2022)
Hassani, K., Khasahmadi Amir, H.: Contrastive multi-view representation learning on graphs. Comput. Res. Repository 1, 4116–4126 (2020)
Schulman, J., Moritz, P., Levine, S., Jordan, M.I., Abbeel, P.: High-dimensional continuous control using generalized advantage estimation. In: International Conference on Learning Representations, abs/1506.02438 (2015)
Bhagat, S., Cormode, G., Muthukrishnan, S.: Node classification in social networks. In: Aggarwal, C. (eds.) Social network data analytics, pp. 115–148. Springer, Cham (2011). https://doi.org/10.1007/978-1-4419-8462-3_5
Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines. TIST 2011, 1–27 (2011)
Zhang, H., Wu, Q., Yan, J., Wipf, D., Yu, P.S.: From canonical correlation analysis to self-supervised graph neural networks. In: NeurIPS (2021)
Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., Mei, Q.: Line: large-scale information network embedding. In: Proceedings of the 24th International Conference on World Wide Web, pp. 1067–1077. International World Wide Web Conferences Steering Committee (2015)
Han, C., Ziwen, Z., Yuhua, L., Yixiong, Z., Ruixuan, L., Rui, Z.: CSGCL: Community-Strength-Enhanced Graph Contrastive Learning, Computing Research Repository, abs/2305.04658: 2059-2067 (2023)
Pan, S., Hu, R., Long, G., Jiang, J., Yao, L., Zhang, C.: Adversarially regularized graph autoencoder for graph embedding. In: IJCAI. ijcai.org, pp. 2609–2615 (2018)
Duran, A.G., Niepert, M.: Learning graph representations with embedding propagation. In: NeurIPS (2017)
Park, J., Lee, M., Chang, H.J., Lee, K., Choi, J.Y.: Symmetric graph convolutional autoencoder for unsupervised graph representation learning. In: ICCV (2019)
Salehi, A., Davulcu, H.: Graph attention auto-encoders. In: ICTAI. IEEE (2020)
Hassani, K., Ahmadi, A.H.K.: Contrastive multi-view representation learning on graphs. In: ICML, Vol. 119. PMLR, pp. 4116–4126 (2020)
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT (1). Association for Computational Linguistics, pp. 4171–4186 (2019)
He, K., Chen, X., Xie, S., Li, Y., Dollá¡r, P., Girshick, R.B.: Masked autoencoders are scalable vision learners. In: CVPR, pp. 15979–15988. IEEE (2022)
Acknowledgement
This work is supported by the National Natural Science Foundation of China (No.623061062), and Natural Science Foundation of Hubei Province under Grant 2023AFB377.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jiang, G., Jin, X., Luo, M., Chen, J., Huang, Z., Wang, J. (2024). ComMGAE: Community Aware Masked Graph AutoEncoder. In: Wand, M., Malinovská, K., Schmidhuber, J., Tetko, I.V. (eds) Artificial Neural Networks and Machine Learning – ICANN 2024. ICANN 2024. Lecture Notes in Computer Science, vol 15020. Springer, Cham. https://doi.org/10.1007/978-3-031-72344-5_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-72344-5_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72343-8
Online ISBN: 978-3-031-72344-5
eBook Packages: Computer ScienceComputer Science (R0)