Abstract
Graph neural networks (GNNs) are one of the most effective techniques for node classification tasks. However, standard GNNs strongly depend on the graph homophily assumption, and their accuracy can degrade substantially on heterophilic graphs. A number of recent works have found that using graph topology as an explicit feature can improve classification performance. However, we observe that the sparsity of graphs often limits the amount of information first order connectivity can provide. We propose an embedding method which uses higher-order connectivity information to further improve accuracy, while limiting the amount of extra computational overhead. We further observe that standard features-based GNNs and newer topology-based models each have their own strengths and weaknesses on the same graph, and we introduce a technique to combine information from both types of GNNs to achieve higher accuracy than either type alone. We conduct extensive experiments on graphs with a range of sizes and heterophily levels and show that our proposed GNN architecture achieves state of the art accuracy, especially on highly heterophilic graphs. We also conduct further experiments and ablations to validate the observations underlying our GNN’s design and analyze the importance of different components. Our code is available in https://github.com/lannester666/BoPE-GNN.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bingham, E., Mannila, H.: Random projection in dimensionality reduction: applications to image and text data. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 245–250 (2001)
Bo, D., Wang, X., Shi, C., Shen, H.: Beyond low-frequency information in graph convolutional networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 3950–3957 (2021)
Bojchevski, A., Günnemann, S.: Deep gaussian embedding of graphs: unsupervised inductive learning via ranking. arXiv preprint arXiv:1707.03815 (2017)
Chen, M., Wei, Z., Huang, Z., Ding, B., Li, Y.: Simple and deep graph convolutional networks. In: International Conference on Machine Learning, pp. 1725–1735. PMLR (2020)
Chien, E., Peng, J., Li, P., Milenkovic, O.: Adaptive universal generalized pagerank graph neural network. arXiv preprint arXiv:2006.07988 (2020)
Goyal, P., Ferrara, E.: Graph embedding techniques, applications, and performance: a survey. Knowl.-Based Syst. 151, 78–94 (2018)
Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Kilmer, M.E., Martin, C.D.: Factorization strategies for third-order tensors. Linear Algebra Appl. 435(3), 641–658 (2011)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Li, X., et al.: Finding global homophily in graph neural networks when meeting heterophily. In: International Conference on Machine Learning, pp. 13242–13256. PMLR (2022)
Lim, D., et al.: Large scale learning on non-homophilous graphs: new benchmarks and strong simple methods. In: Advances in Neural Information Processing Systems, vol. 34, pp. 20887–20902 (2021)
Liu, H., Hu, B., Wang, X., Shi, C., Zhang, Z., Zhou, J.: Confidence may cheat: self-training on graph neural networks under distribution shift. In: Proceedings of the ACM Web Conference 2022, pp. 1248–1258 (2022)
Luan, S., et al.: Revisiting heterophily for graph neural networks. In: Advances in Neural Information Processing Systems, vol. 35, pp. 1362–1375 (2022)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, vol. 26 (2013)
Pei, H., Wei, B., Chang, K.C.C., Lei, Y., Yang, B.: Geom-GCN: geometric graph convolutional networks. In: International Conference on Learning Representations (2019)
Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701–710 (2014)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. In: International Conference on Learning Representations (2017)
Wang, M., Hu, G.: A novel method for twitter sentiment analysis based on attentional-graph neural network. Information 11(2), 92 (2020)
Wang, X., Liu, H., Shi, C., Yang, C.: Be confident! towards trustworthy graph neural networks via confidence calibration. In: Advances in Neural Information Processing Systems, vol. 34, pp. 23768–23779 (2021)
Wu, F., Souza, A., Zhang, T., Fifty, C., Yu, T., Weinberger, K.: Simplifying graph convolutional networks. In: International Conference on Machine Learning, pp. 6861–6871. PMLR (2019)
Wu, J., Chen, H., Cheng, M., Xiong, H.: Curvagn: curvature-based adaptive graph neural networks for predicting protein-ligand binding affinity. BMC Bioinform. 24(1), 378 (2023)
Yan, Y., Hashemi, M., Swersky, K., Yang, Y., Koutra, D.: Two sides of the same coin: heterophily and oversmoothing in graph convolutional neural networks. In: 2022 IEEE International Conference on Data Mining (ICDM), pp. 1287–1292. IEEE (2022)
Zhang, J., Dong, Y., Wang, Y., Tang, J., Ding, M.: Prone: fast and scalable network representation learning. In: IJCAI, vol. 19, pp. 4278–4284 (2019)
Zheleva, E., Getoor, L.: To join or not to join: the illusion of privacy in social networks with mixed public and private user profiles. In: Proceedings of the 18th International Conference on World Wide Web, pp. 531–540 (2009). https://doi.org/10.1145/1526709.1526781
Zhu, J., Yan, Y., Zhao, L., Heimann, M., Akoglu, L., Koutra, D.: Beyond homophily in graph neural networks: current limitations and effective designs. In: Advances in Neural Information Processing Systems, vol. 33, pp. 7793–7804 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lai, Y., Zhang, T., Fan, R. (2024). Improved Topology Features for Node Classification on Heterophilic Graphs. In: Bifet, A., Davis, J., Krilavičius, T., Kull, M., Ntoutsi, E., Žliobaitė, I. (eds) Machine Learning and Knowledge Discovery in Databases. Research Track. ECML PKDD 2024. Lecture Notes in Computer Science(), vol 14947. Springer, Cham. https://doi.org/10.1007/978-3-031-70368-3_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-70368-3_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-70367-6
Online ISBN: 978-3-031-70368-3
eBook Packages: Computer ScienceComputer Science (R0)
