Skip to main content
SpringerLink
Log in

Multi-scale Directed Graph Convolution Neural Network for Node Classification Task

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1967))

Included in the following conference series:

  • 418 Accesses

Abstract

The existence of problems and objects in the real world which can be naturally modeled by complex graph structure has motivated researchers to combine deep learning techniques with graph theory. Despite the proposal of various spectral-based graph neural networks (GNNs), they still have shortcomings in dealing with directed graph-structured data and aggregating neighborhood information of nodes at larger scales. In this paper, we first improve the Lanczos algorithm by orthogonality checking method and Modified Gram-Schmidt orthogonalization technique. Then, we build a long-scale convolution filter based on the improved Lanczos algorithm and combine it with a short-scale filter based on Chebyshev polynomial truncation to construct a multi-scale directed graph convolution neural network (MSDGCNN) which can aggregate multi-scale neighborhood information of directed graph nodes in larger scales. We validate our improved Lanczos algorithm on the atom classification task of the QM8 quantum chemistry dataset. We also apply the MSDGCNN on various real-world directed graph datasets (including WebKB, Citeseer, Telegram and Cora-ML) for node classification task. The result shows that our improved Lanczos algorithm has much better stability, and the MSDGCNN outperforms other state-of-the-art GNNs on such task of real-world datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bojchevski, A., Günnemann, S.: Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking. arXiv preprint arXiv:1707.03815 (2017)

  2. Bovet, A., Grindrod, P.: The activity of the far right on telegram (2020)

    Google Scholar 

  3. Chen, B.: Molecular Graph Representation Learning and Generation for Drug Discovery. Ph.D. thesis, Massachusetts Institute of Technology (2022)

    Google Scholar 

  4. Chung, F.R.: Spectral graph theory, vol. 92. American Mathematical Soc. (1997)

    Google Scholar 

  5. Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems 29 (2016)

    Google Scholar 

  6. Fanuel, M., Alaiz, C.M., Suykens, J.A.: Magnetic eigenmaps for community detection in directed networks. Phys. Rev. E 95(2), 022302 (2017)

    Article  MathSciNet  Google Scholar 

  7. Furutani, S., Shibahara, T., Akiyama, M., Hato, K., Aida, M.: Graph signal processing for directed graphs based on the hermitian laplacian. In: Brefeld, U., Fromont, E., Hotho, A., Knobbe, A., Maathuis, M., Robardet, C. (eds.) ECML PKDD 2019. LNCS (LNAI), vol. 11906, pp. 447–463. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-46150-8_27

    Chapter  Google Scholar 

  8. Gasteiger, J., Bojchevski, A., Günnemann, S.: Predict then propagate: Graph neural networks meet personalized pagerank. arXiv preprint arXiv:1810.05997 (2018)

  9. Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., Dahl, G.E.: Neural message passing for quantum chemistry. In: International Conference on Machine Learning, pp. 1263–1272. PMLR (2017)

    Google Scholar 

  10. Golub, G.H., Van Loan, C.F.: Matrix computations. JHU press (2013)

    Google Scholar 

  11. Grcar, J.F.: Analyses of the Lanczos Algorithm and of the Approximation Problem in Richardson’s Method. University of Illinois at Urbana-Champaign (1981)

    Google Scholar 

  12. Guo, K., Mohar, B.: Hermitian adjacency matrix of digraphs and mixed graphs. J. Graph Theory 85(1), 217–248 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  13. Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. Advances in neural information processing systems 30 (2017)

    Google Scholar 

  14. Hammond, D.K., Vandergheynst, P., Gribonval, R.: Wavelets on graphs via spectral graph theory. Appl. Comput. Harmon. Anal. 30(2), 129–150 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  15. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  16. He, Y., Perlmutter, M., Reinert, G., Cucuringu, M.: Msgnn: a spectral graph neural network based on a novel magnetic signed laplacian. In: Learning on Graphs Conference, pp. 40–1. PMLR (2022)

    Google Scholar 

  17. Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R.: Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 (2012)

  18. Huang, T., Dong, Y., Ding, M., Yang, Z., Feng, W., Wang, X., Tang, J.: Mixgcf: An improved training method for graph neural network-based recommender systems. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. pp. 665–674 (2021)

    Google Scholar 

  19. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)

  20. Kipf, T.N., Welling, M.: Variational graph auto-encoders. arXiv preprint arXiv:1611.07308 (2016)

  21. Ko, T., Choi, Y., Kim, C.K.: A spectral graph convolution for signed directed graphs via magnetic laplacian. Neural Networks (2023)

    Google Scholar 

  22. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)

    Article  Google Scholar 

  23. Lanczos, C.: An iteration method for the solution of the eigenvalue problem of linear differential and integral operators (1950)

    Google Scholar 

  24. LeClair, A., Haque, S., Wu, L., McMillan, C.: Improved code summarization via a graph neural network. In: Proceedings of the 28th International Conference on Program Comprehension, pp. 184–195 (2020)

    Google Scholar 

  25. Levie, R., Huang, W., Bucci, L., Bronstein, M., Kutyniok, G.: Transferability of spectral graph convolutional neural networks. J. Mach. Learn. Res. 22(1), 12462–12520 (2021)

    MathSciNet  MATH  Google Scholar 

  26. Li, Y., Yu, R., Shahabi, C., Liu, Y.: Diffusion convolutional recurrent neural network: data-driven traffic forecasting. arXiv preprint arXiv:1707.01926 (2017)

  27. Li, Y., Tarlow, D., Brockschmidt, M., Zemel, R.: Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493 (2015)

  28. Liao, R., Zhao, Z., Urtasun, R., Zemel, R.S.: Lanczosnet: multi-scale deep graph convolutional networks. arXiv preprint arXiv:1901.01484 (2019)

  29. Lieb, E.H., Loss, M.: Fluxes, laplacians, and kasteleyn’s theorem. Statistical Mechanics: Selecta of Elliott H. Lieb, pp. 457–483 (2004)

    Google Scholar 

  30. Paige, C.C.: Computational variants of the lanczos method for the eigenproblem. IMA J. Appl. Math. 10(3), 373–381 (1972)

    Article  MathSciNet  MATH  Google Scholar 

  31. Paige, C.C.: Error analysis of the lanczos algorithm for tridiagonalizing a symmetric matrix. IMA J. Appl. Math. 18(3), 341–349 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  32. Parlett, B.N., Scott, D.S.: The lanczos algorithm with selective orthogonalization. Math. Comput. 33(145), 217–238 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  33. Pei, H., Wei, B., Chang, K.C.C., Lei, Y., Yang, B.: Geom-gcn: geometric graph convolutional networks. arXiv preprint arXiv:2002.05287 (2020)

  34. Ramakrishnan, R., Hartmann, M., Tapavicza, E., Von Lilienfeld, O.A.: Electronic spectra from tddft and machine learning in chemical space. J. Chem. Phys. 143(8), 084111 (2015)

    Article  Google Scholar 

  35. Ramsundar, B., Eastman, P., Walters, P., Pande, V., Leswing, K., Wu, Z.: Deep Learning for the Life Sciences. O’Reilly Media (2019)

    Google Scholar 

  36. Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45(11), 2673–2681 (1997)

    Article  Google Scholar 

  37. Shuman, D.I., Narang, S.K., Frossard, P., Ortega, A., Vandergheynst, P.: The emerging field of signal processing on graphs: extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Process. Mag. 30(3), 83–98 (2013)

    Article  Google Scholar 

  38. Simon, H.D.: The lanczos algorithm with partial reorthogonalization. Math. Comput. 42(165), 115–142 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  39. Susnjara, A., Perraudin, N., Kressner, D., Vandergheynst, P.: Accelerated filtering on graphs using lanczos method. arXiv preprint arXiv:1509.04537 (2015)

  40. Tong, Z., Liang, Y., Sun, C., Li, X., Rosenblum, D., Lim, A.: Digraph inception convolutional networks. Adv. Neural. Inf. Process. Syst. 33, 17907–17918 (2020)

    Google Scholar 

  41. Tong, Z., Liang, Y., Sun, C., Rosenblum, D.S., Lim, A.: Directed graph convolutional network. arXiv preprint arXiv:2004.13970 (2020)

  42. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)

  43. Wu, L., Cui, P., Pei, J., Zhao, L., Guo, X.: Graph neural networks: foundation, frontiers and applications. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 4840–4841 (2022)

    Google Scholar 

  44. Wu, Z., et al.: Moleculenet: a benchmark for molecular machine learning. Chem. Sci. 9(2), 513–530 (2018)

    Article  Google Scholar 

  45. Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Philip, S.Y.: A comprehensive survey on graph neural networks. IEEE Trans. Neural Networks Learn. Syst. 32(1), 4–24 (2020)

    Article  MathSciNet  Google Scholar 

  46. Xu, B., Shen, H., Cao, Q., Qiu, Y., Cheng, X.: Graph wavelet neural network. arXiv preprint arXiv:1904.07785 (2019)

  47. Xu, K., Hu, W., Leskovec, J., Jegelka, S.: How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 (2018)

  48. Zhang, C., et al.: Deeptralog: trace-log combined microservice anomaly detection through graph-based deep learning (2022)

    Google Scholar 

  49. Zhang, J., Hui, B., Harn, P.W., Sun, M.T., Ku, W.S.: Mgc: a complex-valued graph convolutional network for directed graphs. arXiv e-prints pp. arXiv-2110 (2021)

    Google Scholar 

  50. Zhang, X., He, Y., Brugnone, N., Perlmutter, M., Hirn, M.: Magnet: a neural network for directed graphs. Adv. Neural. Inf. Process. Syst. 34, 27003–27015 (2021)

    Google Scholar 

  51. Zhou, J., et al.: Graph neural networks: a review of methods and applications. AI Open 1, 57–81 (2020)

    Article  Google Scholar 

Download references

Acknowledgements

This work is financially supported by: The National Key R &D Program of China (No. 2020YFB1712600); The Fundamental Research Funds for Central University (No. 3072022QBZ0601); and The State Key Laboratory of Underwater Robotics Technology at Harbin Engineering University (No. KY70100200052).

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Harbin Engineering University, Harbin, China

    Fengming Li, Dong Xu, Fangwei Liu & Yulong Meng

  2. Modeling and Emulation in E-Government National Engineering Laboratory, Harbin Engineering University, Harbin, China

    Dong Xu & Yulong Meng

  3. Jiangsu JARI Technology Group Co. Ltd, Lianyungang, China

    Xinyu Liu

Authors
  1. Fengming Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dong Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fangwei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yulong Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xinyu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yulong Meng .

Editor information

Editors and Affiliations

  1. Changsha, China

    Biao Luo

  2. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Long Cheng

  3. Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou, China

    Zheng-Guang Wu

  4. School of Automation, Guangdong University of Technology, Guangdong, China

    Hongyi Li

  5. UNSW Sydney, Sydney, NSW, Australia

    Chaojie Li

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, F., Xu, D., Liu, F., Meng, Y., Liu, X. (2024). Multi-scale Directed Graph Convolution Neural Network for Node Classification Task. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1967. Springer, Singapore. https://doi.org/10.1007/978-981-99-8178-6_34

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8178-6_34

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8177-9

  • Online ISBN: 978-981-99-8178-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation