Skip to main content
SpringerLink
Log in

Temporal resonant graph network for representation learning on dynamic graphs

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Generating dynamic node representation via limited samples at a time is a challenging task. Meanwhile, the human brain can quickly capture dynamic features without much information in a changing environment. Motivated by the idea of brain functions from the adaptive resonance theory (ART), in this paper, we designed and built a dynamic model for node representation generation. Based on the data flow of the ART model, we replaced the classic ART components with deep learnable modules. Our proposed method is suitable for capturing different graph evolution events even with very sparse changes. Experimental results on several synthetic and real-world dynamic graphs for many vital applications show the superiority of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Angadi UB, Venkatesulu M (2010) Fuzzyart neural network for protein classification. J Bioinform Comput Biol 8(5):825–841

    Article  Google Scholar 

  2. Bastas N, Semertzidis T, Axenopoulos A, Daras P (2019) evolve2vec: Learning network representations using temporal unfolding. In: Proceedings of international conference on multimedia modeling (MMM), Lect Notes Comput Sci, vol 11295. pp 447–458

  3. Belkin M, Niyogi P (2001) Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Proceedings of advances in neural information processing systems (NIPS), pp 585–591

  4. Biggiogera J, Boateng G, Hilpert P, Vowels MJ, Bodenmann G, Neysari M, Nussbeck F, Kowatsch T (2021) BERT Meets LIWC: exploring state-of-the-art language models for predicting communication behavior in couples’ conflict interactions arXiv:2106.01536

  5. Chehreghani MH (2021) Dynamical algorithms for data mining and machine learning over dynamic graphs. Wiley Interdiscip Rev Data Min Knowl Discov 11(2)

  6. Cherng S, Fang C, Chen C, Chen S (2009) Critical motion detection of nearby moving vehicles in a vision-based driver-assistance system. IEEE Trans Intell Transp Syst 10(1):70–82

    Article  Google Scholar 

  7. Foulds JR, DuBois C, Asuncion AU, Butts CT, Smyth P (2011) A dynamic relational infinite feature model for longitudinal social networks. In: Proceedings of international conference on artificial intelligence and statistics (AISTATS), JMLR Proceedings, vol 15. pp 287-295

  8. Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the international conference on artificial intelligence and statistics, pp 249–256

  9. Goyal P, Kamra N, He X, Liu Y (2018) Dyngem: Deep embedding method for dynamic graphs, axXiv:1805.11273

  10. Goyal P, Chhetri SR, Canedo A (2020) Dyngraph2vec: Capturing network dynamics using dynamic graph representation learning. Knowl Based Syst, vol 187

  11. Grossberg S (2013) Adaptive resonance theory: How a brain learns to consciously attend, learn, and recognize a changing world. Neural Netw 37:1–47

    Article  Google Scholar 

  12. Grover A, Leskovec J (2016) Node2vec: Scalable feature learning for networks. In: Proceedings of ACM SIGKDD international conference on knowledge discovery and data mining (KDD), pp 855–864

  13. Hamilton WL, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. In: Proceedings of advances in neural information processing systems (NIPS), pp 1024–1034

  14. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778

  15. Heaukulani C, Ghahramani Z (2013) Dynamic probabilistic models for latent feature propagation in social networks. In: Proceedings of international conference on machine learning (ICML), JMLR workshop and conference proceedings, vol 28. pp 275–283

  16. Hilbe JM (2009) Logistic regression models. Chapman and hall/CRC

  17. Hochreiter S, Schmidhuber J (1997a) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  18. Hochreiter S, Schmidhuber J (1997b) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  19. Izquierdo JMC, Almonacid M, Pinzolas M, Ibarrola J (2009) dfasart: Dynamic neural processing in fasart model. Neural Netw 22(4):479–487

    Article  Google Scholar 

  20. Kazemi SM, Goel R, Jain K, Kobyzev I, Sethi A, Forsyth P, Poupart P (2020) Representation learning for dynamic graphs: a survey. J Mach Learn Res 21:70:1–70:73

    MathSciNet  MATH  Google Scholar 

  21. Kim M, Leskovec J (2013) Nonparametric multi-group membership model for dynamic networks. In: Proceedings of advances in neural information processing systems (NIPS), pp 1385–1393

  22. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv

  23. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: Proceedings of international conference on machine learning (ICLR)

  24. Kumar S, Zhang X, Leskovec J (2019) Predicting dynamic embedding trajectory in temporal interaction networks. In: Proceedings of ACM SIGKDD international conference on knowledge discovery and data mining (KDD), pp 1269–1278

  25. Lancichinetti A, Fortunato S (2009) Benchmarks for testing community detection algorithms on directed and weighted graphs with overlapping communities. Phys Rev E 80(1):016118

    Article  Google Scholar 

  26. Lei K, Qin M, Bai B, Zhang G (2018) Adaptive multiple non-negative matrix factorization for temporal link prediction in dynamic networks. In: Proceedings of workshop on network meets AI & ML, pp 28–34

  27. Lei K, Qin M, Bai B, Zhang G, Yang M (2019) GCN-GAN: A non-linear temporal link prediction model for weighted dynamic networks. In: Proceedings of IEEE conference on computer communications (INFOCOM), pp 388–396

  28. Liu Z, Zhou D, He J (2019) Towards explainable representation of time-evolving graphs via spatial-temporal graph attention networks. In: Proceedings of ACM international conference on information and knowledge management (CIKM), pp 2137– 2140

  29. Ma X, Sun P, Wang Y (2018) Graph regularized nonnegative matrix factorization for temporal link prediction in dynamic networks. Phys A Stat Mech Appl 496:121–136

    Article  MATH  Google Scholar 

  30. Meng L, Tan A (2019) II, DCW adaptive resonance theory in social media data clustering - roles, methodologies and applications. Adv Inf Knowl Process, Springer

  31. Mishra S, Singh SS, Mishra S, Biswas B (2021) TCD2: tree-based community detection in dynamic social networks. Expert Syst Appl 169:114493

    Article  Google Scholar 

  32. Pareja A, Domeniconi G, Chen J, Ma T, Suzumura T, Kanezashi H, Kaler T, Schardl TB, Leiserson CE (2020) Evolvegcn: Evolving graph convolutional networks for dynamic graphs. In: Proceedings of the AAAI conference on artificial intelligence (AAAI), pp 5363–5370

  33. Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: online learning of social representations. In: Proceedings of ACM SIGKDD international conference on knowledge discovery and data mining (KDD), pp 701–710

  34. Ribeiro LFR, Saverese PHP, Figueiredo DR (2017) Struc2vec: Learning node representations from structural identity. In: Proceedings of ACM SIGKDD international conference on knowledge discovery and data mining (KDD), pp 385–394

  35. Rossi E, Chamberlain B, Frasca F, Eynard D, Monti F, Bronstein M (2020) Temporal graph networks for deep learning on dynamic graphs. In: Proceedings of ICML workshop on graph representation learning

  36. Sankar A, Wu Y, Gou L, Zhang W, Yang H (2018) Dynamic graph representation learning via self-attention networks. arXiv:1812.09430

  37. da Silva LEB, Elnabarawy I, II DCW (2019) A survey of adaptive resonance theory neural network models for engineering applications. Neural Netw 120:167–203

  38. Sun L, Zhang Z, Zhang J, Wang F, Peng H, Su S, Yu PS (2021) Hyperbolic variational graph neural network for modeling dynamic graphs. In: Proceedings of AAAI conference on artificial intelligence (AAAI), pp 4375–4383

  39. Tang J, Qu M, Wang M, Zhang M, Yan J, Mei Q (2015) LINE: large-scale information network embedding. In: Proceedings of international conference on world wide web (WWW), pp 1067–1077

  40. Tay Y, Luu AT, Hui SC (2017) Non-parametric estimation of multiple embeddings for link prediction on dynamic knowledge graphs. In: Proceedings of AAAI conference on artificial intelligence (AAAI), pp 1243–1249

  41. Tian F, Gao B, Cui Q, Chen E, Liu T (2014) Learning deep representations for graph clustering. In: Proceedings of AAAI conference on artificial intelligence (AAAI), pp 1293–1299

  42. Trivedi R, Farajtabar M, Biswal P, Zha H (2019) Dyrep: Learning representations over dynamic graphs. In: Proceedings of international conference on machine learning (ICLR) (ICLR)

  43. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Proceedings of advances in neural information processing systems (NIPS), pp 5998–6008

  44. Velickovic P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2017) Graph attention networks. arXiv:1710.10903

  45. Wang D, Cui P, Zhu W (2016) Structural deep network embedding. In: Proceedings of ACM SIGKDD international conference on knowledge discovery and data mining (KDD), pp 1225–1234

  46. Wang Y, Chang Y, Liu Y, Leskovec J, Li P (2021) Inductive representation learning in temporal networks via causal anonymous walks. In: Proceedings of international conference on machine learning (ICLR)

  47. Xu D, Ruan C, Körpeoglu E, Kumar S, Achan K (2020) Inductive representation learning on temporal graphs. In: Proceedings of international conference on machine learning (ICLR)

  48. Xu K, Hu W, Leskovec J (2019) Jegelka s. In: Proceedings of international conference on machine learning (ICLR)

  49. Yang H, Cheng J, Su X, Zhang W, Zhao S, Chen X (2021) A spiderweb model for community detection in dynamic networks. Appl Intell 51(7):5157–5188

    Article  Google Scholar 

  50. Yang L, Cao X, He D, Wang C, Wang X, Zhang W (2016) Modularity based community detection with deep learning. In: Proceedings of international joint conference on Artificial Intelligence (IJCAI), pp 2252–2258

  51. You J, Ying R, Leskovec J (2019) Position-aware graph neural networks. In: Proceedings of international conference on machine learning (ICML), Proceedings of Machine Learning Research, vol. 97. pp 7134–7143

  52. Zhu D, Cui P, Zhang Z, Pei J, Zhu W (2018) High-order proximity preserved embedding for dynamic networks. IEEE Trans Knowl Data Eng 30(11):2134–2144

    Google Scholar 

  53. Zhu L, Guo D, Yin J, Steeg GV, Galstyan A (2016) Scalable temporal latent space inference for link prediction in dynamic social networks. IEEE Trans Knowl Data Eng 28(10):2765–2777

    Article  Google Scholar 

  54. Zhuang D, Chang JM, Li M (2021) Dynamo: Dynamic community detection by incrementally maximizing modularity. IEEE Trans Knowl Data Eng 33(5):1934–1945

    Google Scholar 

Download references

Acknowledgements

This paper was supported by the National Natural Science Foundation of China (U1802271), Science Foundation for Distinguished Young Scholars of Yunnan Province (2019FJ011), Program of Donglu Scholars of Yunnan University, and Program of Yunnan Key Laboratory of Intelligent Systems and Computing (202205AG070003).

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Yunnan Normal University, Kunming, 650500, China

    Zidu Yin

  2. Yunnan Key Laboratory of Intelligent Systems and Computing, Yunnan University, Kunming, 650500, China

    Kun Yue

Authors
  1. Zidu Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kun Yue
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kun Yue.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yin, Z., Yue, K. Temporal resonant graph network for representation learning on dynamic graphs. Appl Intell 53, 7466–7483 (2023). https://doi.org/10.1007/s10489-022-03919-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03919-0

Keywords

Search

Navigation