Abstract
Recently, graph contrastive learning (GCL) has emerged as a dominant technique for unsupervised graph representation learning. Existing GCL-based approaches typically adopt data augmentation to generate two contrastive views, and maximize the similarity between representations that share the same semantics in these two views. However, there are two weaknesses in existing methods: 1) They are highly dependent on the selection of augmentation modes, such as random node and edge perturbation as well as node feature masking, which may lead to changes in the structure and semantic information of the graph. 2) Existing augmentation strategies fail to adequately model local and global information. Graph-based generative models can use graph topology and semantic information to reconstruct node features, so as to avoid random perturbation of features. Therefore, we design a novel graph contrastive learning approach based on generative model augmentation that minimizes damage to the original graph and incorporates local-global information into the augmented view. Specifically, we first obtain the diffusion graph with global information by diffusion method. Based on the structural and semantic information of the original graph and the diffused graph, local and global augmentations are performed using generative models respectively. Then, we fuse the local and global information into the final augmented view. Extensive experiments illustrate the superior performance of model over state-of-the-art methods.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hassani, K., Khasahmadi, A.H.: Contrastive multi-view representation learning on graphs. In: International Conference on Machine Learning, pp. 4116–4126. PMLR (2020)
Klicpera, J., Weißenberger, S., Günnemann, S.: Diffusion improves graph learning. arXiv preprint arXiv:1911.05485 (2019)
Kondor, R.I., Lafferty, J.: Diffusion kernels on graphs and other discrete structures. In: Proceedings of the 19th International Conference on Machine Learning. vol. 2002, pp. 315–322 (2002)
Lee, N., Lee, J., Park, C.: Augmentation-free self-supervised learning on graphs. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 36, pp. 7372–7380 (2022)
Liu, S., et al.: Local augmentation for graph neural networks. In: International Conference on Machine Learning, pp. 14054–14072. PMLR (2022)
Van der Maaten, L., Hinton, G.: Visualizing data using t-sne. J. Mach. Learn. Res. 9(11) (2008)
Namata, G., London, B., Getoor, L., Huang, B., Edu, U.: Query-driven active surveying for collective classification. In: 10th International Workshop on Mining and Learning with Graphs. vol. 8, p. 1 (2012)
Page, L., Brin, S., Motwani, R., Winograd, T.: The Pagerank Citation Ranking: Bringing Order To The Web. Tech. rep, Stanford InfoLab (1999)
Peng, Z., et al.: Graph representation learning via graphical mutual information maximization. In: Proceedings of The Web Conference 2020, pp. 259–270 (2020)
Sen, P., Namata, G., Bilgic, M., Getoor, L., Galligher, B., Eliassi-Rad, T.: Collective classification in network data. AI Mag. 29(3), 93–93 (2008)
Velickovic, P., Fedus, W., Hamilton, W.L., Liò, P., Bengio, Y., Hjelm, R.D.: Deep graph infomax. ICLR (Poster) 2(3), 4 (2019)
Xia, J., Wu, L., Chen, J., Hu, B., Li, S.Z.: Simgrace: A simple framework for graph contrastive learning without data augmentation. In: Proceedings of the ACM Web Conference 2022. pp. 1070–1079 (2022)
Xie, Y., Xu, Z., Zhang, J., Wang, Z., Ji, S.: Self-supervised learning of graph neural networks: A unified review. IEEE Transactions on Pattern Analysis and Machine Intelligence (2022)
Yang, Z., Cohen, W., Salakhudinov, R.: Revisiting semi-supervised learning with graph embeddings. In: International Conference on Machine Learning, pp. 40–48. PMLR (2016)
Zhao, J., Dong, Y., Ding, M., Kharlamov, E., Tang, J.: Adaptive diffusion in graph neural networks. Adv. Neural. Inf. Process. Syst. 34, 23321–23333 (2021)
Zhu, X., Ghahramani, Z., Lafferty, J.D.: Semi-supervised learning using gaussian fields and harmonic functions. In: Proceedings of the 20th International Conference on Machine learning (ICML-03), pp. 912–919 (2003)
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., Wang, L.: Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131 (2020)
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., Wang, L.: Graph contrastive learning with adaptive augmentation. In: Proceedings of the Web Conference 2021, pp. 2069–2080 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jin, D., Wang, Z., Huo, C., Yu, Z., He, D., Huang, Y. (2023). Local-Global Fusion Augmented Graph Contrastive Learning Based on Generative Models. In: Jin, Z., Jiang, Y., Buchmann, R.A., Bi, Y., Ghiran, AM., Ma, W. (eds) Knowledge Science, Engineering and Management. KSEM 2023. Lecture Notes in Computer Science(), vol 14120. Springer, Cham. https://doi.org/10.1007/978-3-031-40292-0_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-40292-0_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-40291-3
Online ISBN: 978-3-031-40292-0
eBook Packages: Computer ScienceComputer Science (R0)