Skip to main content
SpringerLink
Log in

Multi-level self-adaptive prototypical networks for few-shot node classification on attributed networks

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Attributed networks, such as social networks, citation networks, and traffic networks, are ubiquitous nowadays. Node classification is an essential analysis task on attributed networks that has attracted increasing research attention. However, the conventional node classification methods usually require a large number of labeled training instances, which is labor-consuming and cannot be applied to real-world attributed networks. Thus, various meta-learning methods have been proposed to tackle the problem of insufficient labeled instances. However, few-shot learning for graph-structured data is still under-explored due to two reasons: (1) In the few-shot scenario, equally treating each node makes the model vulnerable to noisy instances; (2) The adjacency matrix and attributed features suffer from the data sparsity and not all dimensions in the feature space are discriminative to support final classification. To solve the above problems, we propose a novel graph meta-learning framework: Multi-Level Self-Adaptive Prototypical Networks (MSPN). The proposed MSPN leverages multi-level adaptive learning to highlight the most expressive nodes and features and rectify the prototype to obtain a more accurate one. The episodic training strategy mimics the real testing scenario and transfers knowledge from previous tasks to unseen target tasks. Extensive experimental results demonstrate that the MSPN is superior to the other methods on few-shot node classification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Borgwardt KM, Ong CS, Schönauer S, Vishwanathan S, Smola AJ, Kriegel HP (2005) Protein function prediction via graph kernels. Bioinformatics 21:i47–i56

    Article  Google Scholar 

  2. Bruna J, Zaremba W, Szlam A, Lecun Y (2013) Spectral networks and locally connected networks on graphs. http://arxiv.org/abs/1312.6203

  3. Cao S, Lu W, Xu Q (2015) Grarep: Learning graph representations with global structural information. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp. 891–900

  4. Chen J, Ma T, Xiao C (2018) Fastgcn: fast learning with graph convolutional networks via importance sampling. http://arxiv.org/abs/1801.10247

  5. Cohen-Shapira N, Rokach L, Shapira B, Katz G, Vainshtein R (2019) Autogrd: Model recommendation through graphical dataset representation. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management,pp. 821–830

  6. Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. Adv Neural Inf Process Syst 25:3837–3845

    Google Scholar 

  7. Ding K, Li J, Agarwal N, Liu H (2020) Inductive anomaly detection on attributed networks. In: 29th International Joint Conference on Artificial Intelligence, IJCAI 2020, pp. 1288–1294. International Joint Conferences on Artificial Intelligence

  8. Ding K, Li J, Bhanushali R, Liu H (2019) Deep anomaly detection on attributed networks. In: Proceedings of the 2019 SIAM International Conference on Data Mining, pp. 594–602. SIAM

  9. Ding K, Wang J, Li J, Shu K, Liu C, Liu H (2020) Graph prototypical networks for few-shot learning on attributed networks. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 295–304

  10. Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: International Conference on Machine Learning, pp. 1126–1135. PMLR

  11. Gao H, Huang H (2018) Deep attributed network embedding. In: Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI))

  12. Gao H, Wang Z, Ji S (2018) Large-scale learnable graph convolutional networks. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1416–1424

  13. Garcia V, Bruna J (2017) Few-shot learning with graph neural networks. http://arxiv.org/abs/1711.04043

  14. Grover A, Leskovec J (2016) node2vec: Scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 855–864

  15. Hamilton W.L, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. http://arxiv.org/abs/1706.02216

  16. Huang X, Li J, Hu X (2017) Label informed attributed network embedding. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, pp. 731–739

  17. Joshi V, Peters M, Hopkins M (2018) Extending a parser to distant domains using a few dozen partially annotated examples. http://arxiv.org/abs/1805.06556

  18. Kaise, Ł, Nachum O, Roy A, Bengio S (2017) Learning to remember rare events. http://arxiv.org/abs/1703.03129

  19. Kipf T.N, Welling M (2017) Semi-supervised classification with graph convolutional networks. In Proceedings of the 6th International Conference on Learning Representations

  20. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  21. Lee K, Maji S, Ravichandran A, Soatto S (2019) Meta-learning with differentiable convex optimization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10657–10665

  22. Li J, Cheng K, Wu L, Liu H (2018) Streaming link prediction on dynamic attributed networks. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pp. 369–377

  23. Li R, Wang S, Zhu F, Huang J (2018) Adaptive graph convolutional neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32

  24. Liu Y, Lee J, Park M, Kim S, Yang E, Hwang S.J, Yang Y (2019) Learning to propagate labels: Transductive propagation network for few-shot learning. In: In Proceedings of the7th International Conference on Learning Representation

  25. McAuley J, Pandey R, Leskovec J (2015) Inferring networks of substitutable and complementary products. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794

  26. Mishra N, Rohaninejad M, Chen X, Abbeel P (2018) A simple neural attentive meta-learner. In: In Proceedings of the 6th International Conference on Learning Representation

  27. Nichol A, Achiam J, Schulman J (2018) On first-order meta-learning algorithms. http://arxiv.org/abs/1803.02999

  28. Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: Online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701–710

  29. Pfeiffer III J.J, Moreno S, La Fond T, Neville J, Gallagher B (2014) Attributed graph models: Modeling network structure with correlated attributes. In: Proceedings of the 23rd International Conference on World wide web, pp. 831–842

  30. Ravi S, Larochelle H (2017) Optimization as a model for few-shot learning. In: In Proceedings of the 5th International Conference on Learning Representation

  31. Ren M, Triantafillou E, Ravi S, Snell J, Swersky K, Tenenbaum J.B, Larochelle H, Zemel R.S (2018) Meta-learning for semi-supervised few-shot classification. In: In Proceedings of the 6th International Conference on Learning Representation

  32. Rios A, Kavuluru R (2018) Few-shot and zero-shot multi-label learning for structured label spaces. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing, vol. 2018, p. 3132. NIH Public Access

  33. Santoro A, Bartunov S, Botvinick M, Wierstra D, Lillicrap T (2016) Meta-learning with memory-augmented neural networks. In: International Conference on Machine Learning, pp. 1842–1850. PMLR

  34. Scarselli F, Gori M, Tsoi A, Hagenbuchner M, Monfardini G (2009) The graph neural network model. IEEE Trans Neural Netw 20(1):61

    Article  Google Scholar 

  35. Sen P, Namata G, Bilgic M, Getoor L, Galligher B, Eliassi-Rad T (2008) Collective classification in network data. AI Magazine 29(3):93–93

    Article  Google Scholar 

  36. Snell J, Swersky K, Zemel R.S (2017) Prototypical networks for few-shot learning. http://arxiv.org/abs/1703.05175

  37. Sung F, Yang Y, Zhang L, Xiang T, Torr P.H, Hospedales T.M (2018) Learning to compare: Relation network for few-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1199–1208

  38. Tang J, Qu M, Wang M, Zhang M, Yan J, Mei Q (2015) Line: Large-scale information network embedding. In: Proceedings of the 24th International Conference on World Wide Web, pp. 1067–1077

  39. Tang J, Zhang J, Yao L, Li J, Zhang L, Su Z (2008) Arnetminer: extraction and mining of academic social networks. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 990–998

  40. Velickovic P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2017) Graph attention networks.http://arxiv.org/abs/1710.10903

  41. Vinyals O, Blundell C, Lillicrap T, Wierstra D et al (2016) Matching networks for one shot learning. Adv Neural Inf Process Syst 29:3630–3638

    Google Scholar 

  42. Wang J, Ding K, Hong L, Liu H, Caverlee J (2020) Next-item recommendation with sequential hypergraphs. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1101–1110

  43. Wang N, Luo M, Ding K, Zhang L, Li J, Zheng Q(2020) Graph few-shot learning with attribute matching. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 1545–1554

  44. Wang X, Cui P, Wang J, Pei J, Zhu W, Yang S (2017) Community preserving network embedding. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31, pp. 203–209

  45. Wu F, Souza A, Zhang T, Fifty C, Yu T, Weinberger K (2019) Simplifying graph convolutional networks. In: International Conference on Machine Learning, pp. 6861–6871. PMLR

  46. Zhang J, Zhao C, Ni B, Xu M, Yang X (2019) Variational few-shot learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1685–1694

  47. Zhang L, Liu J, Luo M, Chang X, Zheng Q (2018) Deep semisupervised zero-shot learning with maximum mean discrepancy. Neural Comput 30(5):1426–1447

    Article  MathSciNet  MATH  Google Scholar 

  48. Zhang S, Tong H (2016) Final: Fast attributed network alignment. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1345–1354

  49. Zhou F, Cao C, Zhang K, Trajcevski G, Zhong T, Geng J (2019) Meta-gnn: On few-shot node classification in graph meta-learning. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 2357–2360

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (62192784, 62172056, 62272058) and CCF-Tencent Open Fund (RAGR20220125).

Author information

Authors and Affiliations

  1. Beijing Key Laboratory of Intelligent Telecommunication Software and Multimedia, School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, 100876, China

    Xin Xu, Junping Du & Zhe Xue

Authors
  1. Xin Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junping Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhe Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Xin Xu or Junping Du.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, X., Du, J. & Xue, Z. Multi-level self-adaptive prototypical networks for few-shot node classification on attributed networks. Neural Comput & Applic 35, 9131–9144 (2023). https://doi.org/10.1007/s00521-022-08175-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-08175-4

Keywords

Search

Navigation