Skip to main content
Springer Nature Link
Log in

Enhancing Heterogeneous Graph Contrastive Learning with Strongly Correlated Subgraphs

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14452))

Included in the following conference series:

  • 866 Accesses

Abstract

Graph contrastive learning maximizes the mutual information between the embedding representations of the same data instances in different augmented views of a graph, obtaining feature representations for graph data in an unsupervised manner without the need for manual labeling. Most existing node-level graph contrastive learning models only consider embeddings of the same node in different views as positive sample pairs, ignoring rich inherent neighboring relation and resulting in certain contrastive information loss. To address this issue, we propose a heterogeneous graph contrastive learning model that incorporates strongly correlated subgraph features. We design a contrastive learning framework suitable for heterogeneous graphs and introduce high-level neighborhood information during the contrasting process. Specifically, our model selects a strongly correlated subgraph for each target node in the heterogeneous graph based on both topological structure information and node attribute feature information. In the calculation of contrastive loss, we perform feature shifting operations on positive and negative samples based on subgraph encoding to enhance the model’s ability to discriminate between approximate samples. We conduct node classification and ablation experiments on multiple public heterogeneous datasets and the results verify the effectiveness of the research contributions of our model.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/Andy-Border/NSHE.

  2. 2.

    https://docs.dgl.ai/generated/dgl.nn.pytorch.HeteroGraphConv.html.

  3. 3.

    https://github.com/PyGCL/PyGCL.

  4. 4.

    https://github.com/optuna/optuna.

References

  1. Cao, S., Lu, W., Xu, Q.: GraRep: learning graph representations with global structural information. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, Melbourne, Australia, pp. 891–900. ACM (2015). https://doi.org/10.1145/2806416.2806512

  2. Chami, I., Ying, Z., Ré, C., Leskovec, J.: Hyperbolic graph convolutional neural networks. In: Wallach, H., Larochelle, H., Beygelzimer, A., dAlché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems. Curran Associates, Inc. (2019)

    Google Scholar 

  3. Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: Proceedings of the 37th International Conference on Machine Learning. PMLR (2020)

    Google Scholar 

  4. Dong, Y., Chawla, N.V., Swami, A.: metapath2vec: scalable representation learning for heterogeneous networks. In: The 23rd ACM SIGKDD International Conference (2017)

    Google Scholar 

  5. Fu, T., Lee, W.C., Lei, Z.: HIN2Vec: explore meta-paths in heterogeneous information networks for representation learning. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, Singapore, Singapore, pp. 1797–1806. ACM (2017). https://doi.org/10.1145/3132847.3132953

  6. Gasteiger, J., Bojchevski, A., Günnemann, S.: Predict then propagate: graph neural networks meet personalized PageRank. In: International Conference on Learning Representations (2019)

    Google Scholar 

  7. Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., Dahl, G.E.: Neural message passing for quantum chemistry. In: Proceedings of the 34th International Conference on Machine Learning, ICML 2017, vol. 70. JMLR.org (2017)

    Google Scholar 

  8. Gori, M., Monfardini, G., Scarselli, F.: A new model for learning in graph domains. In: Proceedings of the 2005 IEEE International Joint Conference on Neural Networks. IEEE (2005). https://doi.org/10.1109/IJCNN.2005.1555942

  9. Grover, A., Leskovec, J.: Node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM (2016). https://doi.org/10.1145/2939672.2939754

  10. Gulcehre, C., et al.: Hyperbolic attention networks. In: 2018 International Conference on Learning Representations (2018)

    Google Scholar 

  11. Gutmann, M., Hyvärinen, A.: Noise-contrastive estimation: a new estimation principle for unnormalized statistical models. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. JMLR Workshop and Conference Proceedings (2010)

    Google Scholar 

  12. Hamilton, W.L., Ying, R., Leskovec, J.: Inductive representation learning on large graphs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS 2017. Curran Associates Inc. (2017)

    Google Scholar 

  13. Han, H., et al.: OpenHGNN: an open source toolkit for heterogeneous graph neural network. In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management. ACM (2022). https://doi.org/10.1145/3511808.3557664

  14. Hassani, K., Khasahmadi, A.H.: Contrastive multi-view representation learning on graphs. In: Proceedings of the 37th International Conference on Machine Learning, ICML 2020. JMLR.org (2020)

    Google Scholar 

  15. He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729–9738 (2020)

    Google Scholar 

  16. Hu, B., Fang, Y., Shi, C.: Adversarial learning on heterogeneous information networks. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM (2019). https://doi.org/10.1145/3292500.3330970

  17. Jin, M., Zheng, Y., Li, Y.F., Gong, C., Zhou, C., Pan, S.: Multi-scale contrastive siamese networks for self-supervised graph representation learning. In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence Organization (2021). https://doi.org/10.24963/ijcai.2021/204

  18. Ju, W., et al.: Unsupervised graph-level representation learning with hierarchical contrasts. Neural Netw. (2023). https://doi.org/10.1016/j.neunet.2022.11.019

    Article  Google Scholar 

  19. Khrulkov, V., Mirvakhabova, L., Ustinova, E., Oseledets, I., Lempitsky, V.: Hyperbolic image embeddings. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2020). https://doi.org/10.1109/CVPR42600.2020.00645

  20. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: 5th International Conference on Learning Representations, ICLR 2017, Conference Track Proceedings, Toulon, France, 24–26 April 2017. OpenReview.net (2017)

    Google Scholar 

  21. Li, X., Ding, D., Kao, B., Sun, Y., Mamoulis, N.: Leveraging meta-path contexts for classification in heterogeneous information networks. In: 2021 IEEE 37th International Conference on Data Engineering (ICDE). IEEE (2021). https://doi.org/10.1109/ICDE51399.2021.00084

  22. Liu, J., Yang, M., Zhou, M., Feng, S., Fournier-Viger, P.: Enhancing hyperbolic graph embeddings via contrastive learning. In: 2nd Workshop on Self-Supervised Learning, NeurIPS 2021. arXiv arXiv:2201.08554, 35th Conference on Neural Information Processing Systems, NeurIPS 2021 (2022)

  23. Liu, Q., Nickel, M., Kiela, D.: Hyperbolic graph neural networks. In: Advances in Neural Information Processing Systems. Curran Associates, Inc. (2019)

    Google Scholar 

  24. Nickel, M., Kiela, D.: Poincaré embeddings for learning hierarchical representations. In: Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc. (2017)

    Google Scholar 

  25. Page, L., Brin, S., Motwani, R., Winograd, T.: The PageRank citation ranking: bringing order to the web (1999). http://ilpubs.stanford.edu:8090/422/

  26. Perozzi, B., Al-Rfou, R., Skiena, S.: DeepWalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM (2014). https://doi.org/10.1145/2623330.2623732

  27. Ren, Y., Liu, B., Huang, C., Dai, P., Bo, L., Zhang, J.: HDGI: an unsupervised graph neural network for representation learning in heterogeneous graph. In: AAAI Workshop (2020)

    Google Scholar 

  28. Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE Trans. Neural Netw. 20, 61–80 (2009). https://doi.org/10.1109/TNN.2008.2005605

    Article  Google Scholar 

  29. Shi, C., Hu, B., Zhao, W.X., Yu, P.S.: Heterogeneous information network embedding for recommendation. IEEE Trans. Knowl. Data Eng. (2019). https://doi.org/10.1109/TKDE.2018.2833443

    Article  Google Scholar 

  30. Tan, Z., Ding, K., Guo, R., Liu, H.: Supervised graph contrastive learning for few-shot node classification. In: Amini, M.R., Canu, S., Fischer, A., Guns, T., Kralj Novak, P., Tsoumakas, G. (eds.) Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2022. LNCS, vol. 13714, pp. 394–411. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-26390-3_24

  31. Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., Mei, Q.: LINE: large-scale information network embedding. In: Proceedings of the 24th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee (2015). https://doi.org/10.1145/2736277.2741093

  32. Thakoor, S., et al.: Large-scale representation learning on graphs via bootstrapping. In: International Conference on Learning Representations (2022)

    Google Scholar 

  33. Tu, K., Cui, P., Wang, X., Wang, F., Zhu, W.: Structural deep embedding for hyper-networks. Proc. AAAI Conf. Artif. Intell. 32(1), 426–433 (2018). AAAI’18/IAAI’18/EAAI’18, AAAI Press (2018)

    Google Scholar 

  34. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. In: 2018 International Conference on Learning Representations (2018)

    Google Scholar 

  35. Veličković, P., Fedus, W., Hamilton, W.L., Liò, P., Bengio, Y., Hjelm, R.D.: Deep graph infomax. In: International Conference on Learning Representations (2022)

    Google Scholar 

  36. Wang, C., Sun, D., Bai, Y.: PiPAD: pipelined and parallel dynamic GNN training on GPUs. In: Proceedings of the 28th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming. ACM (2023). https://doi.org/10.1145/3572848.3577487

  37. Wang, M., et al.: Deep graph library: a graph-centric, highly-performant package for graph neural networks (2019)

    Google Scholar 

  38. Wang, P., Agarwal, K., Ham, C., Choudhury, S., Reddy, C.K.: Self-supervised learning of contextual embeddings for link prediction in heterogeneous networks. In: 2021 Proceedings of the Web Conference. ACM (2021). https://doi.org/10.1145/3442381.3450060

  39. Wang, X., et al.: Heterogeneous graph attention network. In: The World Wide Web Conference. ACM (2019). https://doi.org/10.1145/3308558.3313562

  40. Wang, X., Zhang, Y., Shi, C.: Hyperbolic heterogeneous information network embedding. Proc. AAAI Conf. Artif. Intell. (2019). https://doi.org/10.1609/aaai.v33i01.33015337

    Article  Google Scholar 

  41. Wang, Y., Wang, W., Liang, Y., Cai, Y., Liu, J., Hooi, B.: NodeAug: semi-supervised node classification with data augmentation. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM (2020). https://doi.org/10.1145/3394486.3403063

  42. Wu, F., Souza, A., Zhang, T., Fifty, C., Yu, T., Weinberger, K.: Simplifying graph convolutional networks. In: International Conference on Machine Learning, PMLR 2019, pp. 6861–6871. PMLR (2019)

    Google Scholar 

  43. Wu, Z., Xiong, Y., Yu, S.X., Lin, D.: Unsupervised feature learning via non-parametric instance discrimination. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)

    Google Scholar 

  44. Yang, C., Zhang, J., Han, J.: Neural embedding propagation on heterogeneous networks. In: 2019 IEEE International Conference on Data Mining (ICDM). IEEE (2019)

    Google Scholar 

  45. Yang, Y., Guan, Z., Li, J., Zhao, W., Cui, J., Wang, Q.: Interpretable and efficient heterogeneous graph convolutional network. IEEE Trans. Knowl. Data Eng. (2021). https://doi.org/10.1109/TKDE.2021.3101356

    Article  Google Scholar 

  46. You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., Shen, Y.: Graph contrastive learning with augmentations. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33. Curran Associates, Inc. (2020)

    Google Scholar 

  47. Zeng, J., Xie, P.: Contrastive self-supervised learning for graph classification. Proc. AAAI Conf. Artif. Intell. (2021). https://doi.org/10.1609/aaai.v35i12.17293

    Article  Google Scholar 

  48. Zhang, J., Dong, Y., Wang, Y., Tang, J., Ding, M.: ProNE: fast and scalable network representation learning. In: IJCAI 2019, vol. 19, pp. 4278–4284 (2019)

    Google Scholar 

  49. Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., Wang, L.: Deep graph contrastive representation learning. arXiv arXiv:2006.04131 [cs, stat] (2020)

  50. Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., Wang, L.: Graph contrastive learning with adaptive augmentation. In: Proceedings of the Web Conference 2021. ACM (2021). https://doi.org/10.1145/3442381.3449802

  51. Zhu, Y., Zhou, D., Xiao, J., Jiang, X., Chen, X., Liu, Q.: HyperText: endowing FastText with hyperbolic geometry. In: Findings of the Association for Computational Linguistics, EMNLP 2020. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.findings-emnlp.104

Download references

Author information

Authors and Affiliations

  1. State Key Laboratory of Software Development Environment, Beihang University, Beijing, China

    Yanxi Liu & Bo Lang

  2. Zhongguancun Laboratory, Beijing, China

    Bo Lang

Authors
  1. Yanxi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bo Lang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanxi Liu .

Editor information

Editors and Affiliations

  1. Central South University, Changsha, China

    Biao Luo

  2. Chinese Academy of Sciences, Beijing, China

    Long Cheng

  3. Zhejiang University, Hangzhou, China

    Zheng-Guang Wu

  4. Guangdong University of Technology, Guangzhou, China

    Hongyi Li

  5. UNSW Sydney, Sydney, NSW, Australia

    Chaojie Li

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liu, Y., Lang, B. (2024). Enhancing Heterogeneous Graph Contrastive Learning with Strongly Correlated Subgraphs. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14452. Springer, Singapore. https://doi.org/10.1007/978-981-99-8076-5_7

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8076-5_7

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8075-8

  • Online ISBN: 978-981-99-8076-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics