Skip to main content
SpringerLink
Log in

MG2Vec+: A multi-headed graph attention network for multigraph embedding

  • Regular paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Representation learning of graphs in the form of graph embeddings is an extensively studies area, especially for simple networks, to help with different downstream applications such as node clustering, link prediction, and node classification. In this paper, we propose MG2Vec+, a method that generates node embeddings for a multigraph, a network structure comprising multiple types of edges between pairs of nodes. MG2Vec+ uses multi-headed attention layers to aggregate multiple types of edge-relations that can exist among nodes. The parameters are learned using a graph likelihood loss function which ensures that the sum of attention scores for high-priority nodes is larger as compared to low-priority nodes. We compare MG2Vec+ with nine existing baseline methods after modifying them to our setting on four real-world datasets. MG2Vec+ outperforms the competing methods when evaluated on two downstream tasks: (1) link prediction, and (2) multi-class node classification. It is able to achieve a 5.88% higher AUC-ROC score than the best baseline for link prediction and 9.52% higher classification accuracy than the best baseline for the multi-class node classification task. The superiority of MG2Vec+ can be explained by its principled way of capturing multi-relational contexts and learning them in an unsupervised manner with the same set of parameters using graph likelihood loss.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. Multilayer network is a stacked representation of multiple single layers. Multidimensional network is a special type of multilayer network which is edge-homogeneous, i.e., each layer represents a particular type of relationship among nodes.

  2. http://data.europa.eu/euodp/en/data/dataset/cordisfp7projects.

  3. https://web.hike.in/login.

References

  1. Abu-El-Haija S, Perozzi B, Al-Rfou R, Alemi A (2017a) Watch your step: learning graph embeddings through attention. CoRR. arXiv:1710.09599

  2. Adamic LA, Adar E (2001) Friends and neighbors on the web. Soc Netw 25:211–230

    Article  Google Scholar 

  3. Backstrom L, Leskovec J (2011) Supervised random walks: predicting and recommending links in social networks. In: WSDM, pp 635–644

  4. Bhatia V, Rani R (2018) Dfuzzy: a deep learning-based fuzzy clustering model for large graphs. Knowl Inf Syst 57(1):159–181. https://doi.org/10.1007/s10115-018-1156-3

    Article  Google Scholar 

  5. Bruna J, Zaremba W, Szlam A, LeCun Y (2013) Spectral networks and locally connected networks on graphs. CoRR. arXiv:1312.6203

  6. Chakraborty T, Kumar S, Goyal P, Ganguly N, Mukherjee A (2014) Towards a stratified learning approach to predict future citation counts. In: JCDL, pp 351–360

  7. Chang S, Han W, Tang J, Qi GJ, Aggarwal CC, Huang TS (2015) Heterogeneous network embedding via deep architectures. In: ACM SIGKDD, pp 119–128

  8. Chen S, Niu S, Akoglu L, Kovačević J, Faloutsos C (2017) Fast, warped graph embedding: unifying framework and one-click algorithm. arXiv:1702.05764

  9. Chen H, Yin H, Wang W, Wang H, Nguyen QVH, Li X (2018) PME: projected metric embedding on heterogeneous networks for link prediction. In: ACM SIGKDD, ACM, pp 1177–1186

  10. Cui Z, Park N, Chakraborty T (2020) Incremental community discovery via latent network representation and probabilistic inference. Knowl Inf Syst 62(6):2281–2300. https://doi.org/10.1007/s10115-019-01422-6

    Article  Google Scholar 

  11. Dong Y, Chawla NV, Swami A (2017) Metapath2vec: scalable representation learning for heterogeneous networks. In: ACM SIGKDD, pp 135–144

  12. Grover A, Leskovec J (2016) Node2vec: scalable feature learning for networks. In: ACM SIGKDD, New York, NY, USA, pp 855–864

  13. Guo Q, Cozzo E, Zheng Z, Moreno Y (2016) Lévy random walks on multiplex networks. Sci Rep 6

  14. Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. CoRR. arXiv:1706.02216

  15. Kipf TN, Welling M (2016) Variational graph auto-encoders. NIPS 2016. arXiv:1611.07308

  16. Li Y, Tarlow D, Brockschmidt M, Zemel R (2015) Gated graph sequence neural networks. arXiv:1511.05493

  17. Liao L, He X, Zhang H, Chua T (2018) Attributed social network embedding. IEEE Trans Knowl Data Eng 30(12):2257–2270

    Article  Google Scholar 

  18. Liben-Nowell D, Kleinberg J (2007) The link-prediction problem for social networks. JASIST 58(7):1019–1031

    Article  Google Scholar 

  19. Liu W, Chen PY, Yeung S, Suzumura T, Chen L (2017) Principled multilayer network embedding. In: ICDMW pp 134–141

  20. Liu G, Guo J, Zuo Y, Wu J, Ry Guo (2020) Fraud detection via behavioral sequence embedding. Knowl Inf Syst 62(7):2685–2708. https://doi.org/10.1007/s10115-019-01433-3

    Article  Google Scholar 

  21. Ma Y, Ren Z, Jiang Z, Tang J, Yin D (2018) Multi-dimensional network embedding with hierarchical structure. In: WSDM, pp 387–395

  22. Mikolov T, Chen K, Corrado G, Dean J (2013a) Efficient estimation of word representations in vector space. CoRR. arXiv:1301.3781

  23. Mikolov T, Sutskever I, Chen K, Corrado G, Dean J (2013b) Distributed representations of words and phrases and their compositionality. In: NIPS, pp 3111–3119

  24. Newman MEJ (2001) Clustering and preferential attachment in growing networks. Phys Rev E 64:025102

    Article  Google Scholar 

  25. Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: online learning of social representations. In: ACM SIGKDD, New York, NY, USA, pp 701–710

  26. Roy A, Kumar V, Mukherjee D, Chakraborty T (2020) Learning multigraph node embeddings using guided lévy flights. In: Pacific–Asia conference on knowledge discovery and data mining. Springer, Berlin, pp 524–537

  27. Schlichtkrull M, Kipf TN, Bloem P, van den Berg R, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: Gangemi A, Navigli R, Vidal ME, Hitzler P, Troncy R, Hollink L, Tordai A, Alam M (eds) The semantic web. Springer, Cham, pp 593–607

    Chapter  Google Scholar 

  28. Shi C, Hu B, Zhao WX, Philip SY (2019) Heterogeneous information network embedding for recommendation. IEEE TKDE 31(2):357–370

    Google Scholar 

  29. Snijders TA, Pattison PE, Robins GL, Handcock MS (2006) New specifications for exponential random graph models. Sociol Methodol 36(1):99–153

    Article  Google Scholar 

  30. Tang L, Liu H (2010) Leveraging social media networks for classification. Data Min Knowl Discov 23:447–478

    Article  MathSciNet  MATH  Google Scholar 

  31. Tang J, Qu M, Wang M, Zhang M, Yan J, Mei Q (2015) Line: large-scale information network embedding. In: WWW, pp 1067–1077

  32. Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. CoRR. arXiv:1710.10903

  33. Verbrugge LM (1979) Multiplexity in adult friendships. Soc Forces 57(4):1286–1309

    Article  Google Scholar 

  34. Verma J, Gupta S, Mukherjee D, Chakraborty T (2019) Heterogeneous edge embeddings for friend recommendation. arXiv:1902.03124

  35. You J, Ying R, Ren X, Hamilton WL, Leskovec J (2018) Graphrnn: a deep generative model for graphs. CoRR. arXiv:1802.08773

  36. Zhang H, Qiu L, Yi L, Song Y (2018) Scalable multiplex network embedding. In: IJCAI, pp 3082–3088

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, IIIT-Delhi, New Delhi, India

    Aman Roy, Shravika Mittal & Tanmoy Chakraborty

Authors
  1. Aman Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shravika Mittal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tanmoy Chakraborty
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aman Roy.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Roy, A., Mittal, S. & Chakraborty, T. MG2Vec+: A multi-headed graph attention network for multigraph embedding. Knowl Inf Syst 65, 111–132 (2023). https://doi.org/10.1007/s10115-022-01706-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-022-01706-4

Keywords

Search

Navigation