Skip to main content
SpringerLink
Log in

Multi-view Self-supervised Heterogeneous Graph Embedding

  • Conference paper
  • First Online:
Machine Learning and Knowledge Discovery in Databases. Research Track (ECML PKDD 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12976))

Abstract

Graph mining tasks often suffer from the lack of supervision from labeled information due to the intrinsic sparseness of graphs and the high cost of manual annotation. To alleviate this issue, inspired by recent advances of self-supervised learning (SSL) on computer vision and natural language processing, graph self-supervised learning methods have been proposed and achieved remarkable performance by utilizing unlabeled information. However, most existing graph SSL methods focus on homogeneous graphs, ignoring the ubiquitous heterogeneity of real-world graphs where nodes and edges are of multiple types. Therefore, directly applying existing graph SSL methods to heterogeneous graphs can not fully capture the rich semantics and their correlations in heterogeneous graphs. In light of this, we investigate self-supervised learning on heterogeneous graphs and propose a novel model named Multi-View Self-supervised heterogeneous graph Embedding (MVSE). By encoding information from different views defined by meta-paths and optimizing both intra-view and inter-view contrastive learning tasks, MVSE comprehensively utilizes unlabeled information and learns node embeddings. Extensive experiments are conducted on various tasks to show the effectiveness of the proposed framework.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/Andy-Border/MVSE.

References

  1. Bordes, A., Usunier, N., Garca-Durán, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: NIPS, pp. 2787–2795 (2013)

    Google Scholar 

  2. Chen, H., Yin, H., Wang, W., Wang, H., Nguyen, Q.V.H., Li, X.: PME: projected metric embedding on heterogeneous networks for link prediction. In: KDD, pp. 1177–1186 (2018)

    Google Scholar 

  3. Dong, Y., Chawla, N.V., Swami, A.: metapath2vec: scalable representation learning for heterogeneous networks. In: KDD, pp. 135–144 (2017)

    Google Scholar 

  4. Fan, S., et al.: Metapath-guided heterogeneous graph neural network for intent recommendation. In: KDD, pp. 2478–2486 (2019)

    Google Scholar 

  5. Fu, T.Y., Lee, W.C., Lei, Z.: HIN2Vec: explore meta-paths in heterogeneous information networks for representation learning. In: CIKM, pp. 1797–1806 (2017)

    Google Scholar 

  6. Fu, X., Zhang, J., Meng, Z., King, I.: MAGNN: metapath aggregated graph neural network for heterogeneous graph embedding. In: WWW, pp. 2331–2341 (2020)

    Google Scholar 

  7. Hamilton, W.L., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: NIPS, pp. 1024–1034 (2017)

    Google Scholar 

  8. Hassani, K., Ahmadi, A.H.K.: Contrastive multi-view representation learning on graphs. In: ICML, pp. 4116–4126 (2020)

    Google Scholar 

  9. He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.B.: Momentum contrast for unsupervised visual representation learning. In: CVPR, pp. 9726–9735 (2020)

    Google Scholar 

  10. He, Y., Song, Y., Li, J., Ji, C., Peng, J., Peng, H.: HeteSpaceyWalk: a heterogeneous spacey random walk for heterogeneous information network embedding. In: CIKM, pp. 639–648 (2019)

    Google Scholar 

  11. Hong, H., Guo, H., Lin, Y., Yang, X., Li, Z., Ye, J.: An attention-based graph neural network for heterogeneous structural learning. In: AAAI, pp. 4132–4139 (2020)

    Google Scholar 

  12. Hu, B., Fang, Y., Shi, C.: Adversarial learning on heterogeneous information networks. In: KDD, pp. 120–129 (2019)

    Google Scholar 

  13. Hu, B., Zhang, Z., Shi, C., Zhou, J., Li, X., Qi, Y.: Cash-out user detection based on attributed heterogeneous information network with a hierarchical attention mechanism. In: AAAI, pp. 946–953 (2019)

    Google Scholar 

  14. Hu, L., Yang, T., Shi, C., Ji, H., Li, X.: Heterogeneous graph attention networks for semi-supervised short text classification. In: EMNLP-IJCNLP, pp. 4820–4829 (2019)

    Google Scholar 

  15. Hu, W., et al.: Strategies for pre-training graph neural networks. In: ICLR (2020)

    Google Scholar 

  16. Hu, Z., Dong, Y., Wang, K., Chang, K.W., Sun, Y.: GPT-GNN: generative pretraining of graph neural networks. In: KDD, pp. 1857–1867 (2020)

    Google Scholar 

  17. Hu, Z., Dong, Y., Wang, K., Sun, Y.: Heterogeneous graph transformer. In: WWW, pp. 2704–2710 (2020)

    Google Scholar 

  18. Hussein, R., Yang, D., Cudré-Mauroux, P.: Are meta-paths necessary?: Revisiting heterogeneous graph embeddings. In: CIKM, pp. 437–446 (2018)

    Google Scholar 

  19. Hwang, D., Park, J., Kwon, S., Kim, K.M., Ha, J.W., Kim, H.J.: Self-supervised auxiliary learning with meta-paths for heterogeneous graphs. arXiv preprint arXiv:2007.08294 (2020)

  20. Jin, W., et al.: Self-supervised learning on graphs: deep insights and new direction. arXiv preprint arXiv:2006.10141 (2020)

  21. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)

    Google Scholar 

  22. Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. In: ICLR (2014)

    Google Scholar 

  23. Kipf, T.N., Welling, M.: Variational graph auto-encoders. arXiv preprint arXiv:1611.07308 (2016)

  24. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: ICLR (2017)

    Google Scholar 

  25. Li, A., Qin, Z., Liu, R., Yang, Y., Li, D.: Spam review detection with graph convolutional networks. In: CIKM, pp. 2703–2711 (2019)

    Google Scholar 

  26. Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: AAAI, pp. 2181–2187 (2015)

    Google Scholar 

  27. Liu, X., et al.: Self-supervised learning: Generative or contrastive. arXiv preprint arXiv:2006.08218 (2020)

  28. Lu, Y., Shi, C., Hu, L., Liu, Z.: Relation structure-aware heterogeneous information network embedding. In: AAAI, pp. 4456–4463 (2019)

    Google Scholar 

  29. Maaten, L.V.D., Hinton, G.: Visualizing data using t-SNE. JMLR 9(11), 2579–2605 (2008)

    Google Scholar 

  30. Oord, A.V.D., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)

  31. Perozzi, B., Al-Rfou, R., Skiena, S.: DeepWalk: online learning of social representations. In: KDD, pp. 701–710 (2014)

    Google Scholar 

  32. Qiu, J., et al.: GCC: graph contrastive coding for graph neural network pre-training. In: KDD, pp. 1150–1160 (2020)

    Google Scholar 

  33. Qiu, J., Tang, J., Ma, H., Dong, Y., Wang, K., Tang, J.: DeepInf: social influence prediction with deep learning. In: KDD, pp. 2110–2119 (2018)

    Google Scholar 

  34. Sun, Y., Han, J., Yan, X., Yu, P.S., Wu, T.: PathSim: meta path-based top-K similarity search in heterogeneous information networks. VLDB 4(11), 992–1003 (2011)

    Google Scholar 

  35. Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: ICML. pp. 2071–2080 (2016)

    Google Scholar 

  36. Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. In: ICLR (2018)

    Google Scholar 

  37. Velickovic, P., Fedus, W., Hamilton, W.L., Liò, P., Bengio, Y., Hjelm, R.D.: Deep graph infomax. In: ICLR (2019)

    Google Scholar 

  38. Wang, X., Bo, D., Shi, C., Fan, S., Ye, Y., Yu, P.S.: A survey on heterogeneous graph embedding: methods, techniques, applications and sources. arXiv preprint arXiv:2011.14867 (2020)

  39. Wang, X., et al.: Heterogeneous graph attention network. In: WWW, pp. 2022–2032 (2019)

    Google Scholar 

  40. Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: AAAI, pp. 1112–1119 (2014)

    Google Scholar 

  41. Xu, K., Hu, W., Leskovec, J., Jegelka, S.: How powerful are graph neural networks? In: ICLR (2019)

    Google Scholar 

  42. Yang, B., Yih, W., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. In: ICLR (2015)

    Google Scholar 

  43. Yang, C., Xiao, Y., Zhang, Y., Sun, Y., Han, J.: Heterogeneous network representation learning: a unified framework with survey and benchmark. TKDE (2020)

    Google Scholar 

  44. You, J., Ying, R., Ren, X., Hamilton, W.L., Leskovec, J.: GraphRNN: generating realistic graphs with deep auto-regressive models. In: ICML, pp. 5694–5703 (2018)

    Google Scholar 

  45. Yun, S., Jeong, M., Kim, R., Kang, J., Kim, H.J.: Graph transformer networks. In: NIPS, pp. 11960–11970 (2019)

    Google Scholar 

  46. Zhang, C., Song, D., Huang, C., Swami, A., Chawla, N.V.: Heterogeneous graph neural network. In: KDD, pp. 793–803 (2019)

    Google Scholar 

  47. Zhang, C., Swami, A., Chawla, N.V.: SHNE: representation learning for semantic-associated heterogeneous networks. In: WSDM, pp. 690–698 (2019)

    Google Scholar 

  48. Zhao, J., Wang, X., Shi, C., Liu, Z., Ye, Y.: Network schema preserving heterogeneous information network embedding. In: IJCAI, pp. 1366–1372 (2020)

    Google Scholar 

  49. Zhao, K., et al.: Deep adversarial completion for sparse heterogeneous information network embedding. In: WWW, pp. 508–518 (2020)

    Google Scholar 

Download references

Acknowledgements

This work is partially supported by the NSF under grants IIS-2107172, IIS-2027127, IIS-2040144, CNS-2034470, IIS-1951504, CNS-1940859, CNS-1814825, OAC-1940855, and the NIJ 2018-75-CX-0032.

Author information

Authors and Affiliations

  1. Case Western Reserve University, Cleveland, OH, USA

    Jianan Zhao, Qianlong Wen, Shiyu Sun & Yanfang Ye

  2. Brandeis University, Waltham, MA, USA

    Chuxu Zhang

Authors
  1. Jianan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qianlong Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shiyu Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yanfang Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chuxu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yanfang Ye or Chuxu Zhang .

Editor information

Editors and Affiliations

  1. ELLIS - The European Laboratory for Learning and Intelligent Systems, Alicante, Spain

    Nuria Oliver

  2. ETHZ and EPFL, Zürich, Switzerland

    Fernando Pérez-Cruz

  3. Johannes Gutenberg University of Mainz, Mainz, Germany

    Stefan Kramer

  4. École Polytechnique, Palaiseau, France

    Jesse Read

  5. Basque Center for Applied Mathematics, Bilbao, Spain

    Jose A. Lozano

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhao, J., Wen, Q., Sun, S., Ye, Y., Zhang, C. (2021). Multi-view Self-supervised Heterogeneous Graph Embedding. In: Oliver, N., Pérez-Cruz, F., Kramer, S., Read, J., Lozano, J.A. (eds) Machine Learning and Knowledge Discovery in Databases. Research Track. ECML PKDD 2021. Lecture Notes in Computer Science(), vol 12976. Springer, Cham. https://doi.org/10.1007/978-3-030-86520-7_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86520-7_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86519-1

  • Online ISBN: 978-3-030-86520-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation