Elsevier

Neurocomputing

Volume 453, 17 September 2021, Pages 524-537
Neurocomputing

M-GWNN: Multi-granularity graph wavelet neural networks for semi-supervised node classification

https://doi.org/10.1016/j.neucom.2020.10.033Get rights and content

Abstract

Graph convolutional neural networks (GCNs) based on spectral-domain have achieved impressive performance for semi-supervised node classification task. Recently, graph wavelet neural network (GWNN) has made a significant improvement for this task. However, GWNN is usually shallow based on a one- or two-hop neighborhood structure, making it unable to obtain sufficient global information to make it better. But, if GWNN merely stacks too many convolutional layers, it produces the phenomenon of the wavelet convolutional filters over-smoothing. To stack this challenge, we propose Multi-granularity Graph Wavelet Neural Networks (M-GWNN), a novel spectral GCNs architecture that leverages the proposed Louvain-variant algorithm and the jump connection to improve the ability of node representations for semi-supervised node classification. We first repeatedly apply the proposed Louvain-variant algorithm to aggregate nodes into supernodes to build a hierarchy of successively coarser graph, further refine the coarsened graph symmetrically back to the original by utilizing the jump connection. Moreover, during this process, multiple layers of GWNN are applied to propagate information across entire networks. The proposed M-GWNN efficiently captures node features and graph topological structures of varying granularity to obtain global information. Furthermore, M-GWNN effectively employs the jump connection to connect receptive fields of varying granularity to alleviate the speed of over-smoothing. Experiments on four benchmark datasets demonstrate the effectiveness of the proposed M-GWNN. Particularly, when only a few labeled nodes are provided on the NELL dataset, M-GWNN achieves up to an average 5.7% performance improvement compared with state-of-the-art methods.

Introduction

Since many non-Euclidean data that do not have a grid-like structure can only be represented in the form of a graph, extending neural networks to graph structure has attracted considerable attention from researchers. In this paper, we focus on the task of graph-based semi-supervised node classification using graph convolutional neural networks (GCNs) based on spectral-domain. Spectral CNN [1] first attempted to exploit graph Fourier to implement CNN [2] on graphs and maintain the property of CNN, which is the weight sharing parameter used in training. After that, spectral GCNs based on graph Fourier became increasingly popular. Defferrard et al. [3] utilized localized spectral Chebyshev filters to achieve the graph convolution operation for semi-supervised node classification task. Furthermore, GCN [4] demonstrated considerable power and achieved state-of-art performance on this task. Different from GCN [4], graph wavelet neural network (GWNN) [5] is also a spectral graph convolutional neural network that replaces graph Fourier with graph wavelet, which is sparser, more localized, less computationally expensive than Fourier.

However, we found that GWNN is usually shallow and follows a neighborhood aggregation based on one- or two–hop. Owing to the restricted receptive field, GWNN has a difficulty of obtaining sufficient global information to achieve better node representations. But, if GWNN merely stacks too many convolutional layers, it will reduce the performance because of the model produces the phenomenon of over-smoothing. some researches [6], [7] proved that spectral GCNs models repeatedly apply Laplacian smoothing, the features of vertices in each connected component 1 of the graph will converge to the same value, and further cause the performance of the model learning node representations to significantly decrease.

Recently, few embedding methods have proposed hierarchical approaches that capture global topological information of networks to learn node representations. For instance, HARP [8] and MILE [9] repeatedly compresses a network into a series of small networks based on two node collapsing schemes and learn node embeddings for each compressed network by existing network embedding methods. However, because HARP heuristically combines two closed nodes into a new one, the compressed network may not be able to display the global topology of the input network. Other recent approaches, such as graph U-Net [10] and SAGPool [11], which selectively remove some nodes of graph to compress graph and maximize retention of topological information of hierarchical graph.

In this paper, we utilize the concept of community structure existing in real networks [12], [13], [14], [15], [16] as an effective strategy to realize hierarchical graph coarsening to capture global information of the graph. Nodes that exist in a single community have similar types and are closely connected [17]. Therefore, placing nodes in the same community closely in the embedding space is benefit for graph mining tasks [16], such as semi-supervised node classification. To be specific, dividing a graph recursively into a series of coarser graphs (communities) helps capture the similarities between nodes at different levels of proximity: recursive partitioning essentially creates a hierarchy of communities in a network, where (a) nodes in the same community at the top level of the hierarchy represent similar clusters of nodes with higher-order structure and (b) the lower communities in the hierarchy maintain a one-hop (neighbor) relationship between connected pairs of nodes. Therefore, generating embedding vectors from the presence of nodes in the community at different granularities can maintain embedding quality and is suitable for the task of semi-supervised node classification. Moreover, the Louvain algorithm [18] is a state-of-the-art community detection algorithm [19] based on hierarchical modularity optimization [20], which is used to measure the quality of the results of the community detection algorithm, and also can characterize the closeness of the communities. However, there are existing several problems that directly applying the Louvain algorithm into hierarchical graph coarsening: 1) the number of nodes in communities may exhibit significant inhomogeneity in each coarsening; 2) the uncontrollability of the aggregation times of the Louvain algorithm; 3) there are maybe existing several nodes that are not assigned to any community during the coarsening. Therefore, to better fuse GWNN to implement a hierarchical approach, we propose the Louvain-variant algorithm building on Louvain.

Formally, we propose Multi-granularity Graph Wavelet Neural Networks (M-GWNN), a novel spectral GCNs architecture that leverages the proposed Louvain-variant algorithm and the jump connection to improve the ability of node representations for semi-supervised node classification. As pictured in Fig. 1, M-GWNN mainly consists of several graph coarsening and refining procedures and one output layer. M-GWNN first repeatedly applies the proposed Louvain-variant algorithm to aggregate nodes into supernodes, further refines the coarsened graph symmetrically back to the original by utilizing the jump connection. Moreover, during this process, multiple layers of GWNN are applied to propagate information across entire networks. The core idea of the M-GWNN model is effectively widening the receptive field of the wavelet convolutional filters by capturing node features and graph topological information of varying granularity, which employs the proposed Louvain-variant algorithm to build a hierarchy of successively coarser graph. Furthermore, the M-GWNN utilizes the jump connection to connect receptive fields of varying granularity to efficiently alleviate the speed of over-smoothing, which combining the graph coarsening and symmetrical refining procedures.

Overall, the main contributions of the proposed M-GWNN for semi-supervised node classification are summarized as follows:

  • We study GWNN from a multi-granularity perspective, which, to the best of our knowledge, has not been studied before. Compared to related work, we propose the Louvain-variant algorithm to capture graph information of varying granularity to obtain global information of the graph. Furthermore, we employ the jump connection to connect receptive fields of varying granularity to stack the challenge of wavelet convolutional filters over-smoothing.

  • We propose the Louvain-variant algorithm, which is usually used in unsupervised scenarios to solve the semi-supervised problem for the first time. Compared to previous work, a hierarchical clustering method that employs the concept of community structure builds a hierarchy of coarser graph. Moreover, every supernode represents a local topological structure of the original graph, which efficiently captures global information.

  • We conduct extensive evaluation over four standard datasets for semi-supervised node classification task, and comprehensive experiments show the effectiveness of the proposed M-GWNN model. Especially, compared to state-of-the-art methods, M-GWNN achieves an average error reduction of up to 5.7% in the NELL dataset.

Section snippets

Related works

In this section, we briefly review previous work on graph convolutional neural networks, multi-layer graph representation learning, and algorithms based on graph reduction mechanisms.

Notation introduction

Table 1 shows the main notations used in this paper.

Problem definition

We denote a weighted undirected graph G=(V,E,A,X), where |V|=n=|VL|+|VU| is the vertex set of labeled (VL) and unlabeled (VU) nodes in a graph. Generally, |VL||V|, the goal of semi-supervised node classification is to assign labels for all unlabeled nodes VU. E is the edge set, ARn×n is the adjacency matrix of a graph with Ai,j=Aj,i,XRn×f is the graph representations of n data vectors in f dimension at the layer (including original,

Multi-granularity graph wavelet neural networks (M-GWNN)

In this section, we discuss the proposed multi-granularity graph wavelet neural networks (M-GWNN) in detail.

Datasets

To demonstrate the benefit and effectiveness of the proposed M-GWNN on semi-supervised node classification task, we tested it on four standard benchmark datasets, including three Citation Network datasets: Cora, Citeseer, Pubmed (Sen et al. [49]) and Knowledge Graph dataset; the Never-Ending Language Learning system (NELL) (Carlson et al. [50]; and Yang et al. [51]). Table 2 gives an overview of the four datasets, which are described in detail. Following the experimental setup in GCN [4], we

Conclusion

Throughout this paper, to widen the receptive field and alleviate the speed of the wavelet convolutional filters over-smoothing, we propose multi-granularity graph wavelet neural networks (M-GWNN) for semi-supervised node classification. We first aggregate nodes into supernodes utilizing the proposed Louvain-variant algorithm and then symmetrically refine the coarsened graph to the original by the jump connection. M-GWNN obtains node features and graph topological information of varying

CRediT authorship contribution statement

Wenjie Zheng: Conceptualization, Methodology, Software, Data curation, Formal analysis, Writing - original draft, Writing - review & editing. Fulan Qian: Investigation, Writing - review & editing, Formal analysis, Data curation, Visualization, Funding acquisition. Shu Zhao: Validation, Writing - review & editing, Supervision, Funding acquisition. Yanping Zhang: Validation, Writing - review & editing, Supervision, Funding acquisition.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

This work is supported and funded by the National Natural Science Foundation of China (Grant Nos. 61702003, 61673020, 61876001) and Natural Science Foundation of Anhui Province (Grant No. 1808085MF175). Moreover, we would like to thank the reviewers for their constructive comments and suggestions.

Wenjie Zheng is currently pursuing the master degree with the School of Computer Science and Technology, Anhui University, Hefei, China. His current research interests include data mining, graph neural networks and knowledge graphs.

References (56)

  • S. Fortunato

    Community detection in graphs

    Physics Reports

    (2010)
  • D.K. Hammond et al.

    Wavelets on graphs via spectral graph theory

    Applied and Computational Harmonic Analysis

    (2011)
  • W. Sadiq et al.

    Analyzing process models using graph reduction techniques

    Information Systems

    (2000)
  • G. Karypis et al.

    Multilevelk-way partitioning scheme for irregular graphs

    Journal of Parallel and Distributed computing

    (1998)
  • J. Bruna, W. Zaremba, A. Szlam, Y. LeCun, Spectral networks and locally connected networks on graphs, in: International...
  • A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, in:...
  • M. Defferrard, X. Bresson, P. Vandergheynst, Convolutional neural networks on graphs with fast localized spectral...
  • T. N. Kipf, M. Welling, Semi-supervised classification with graph convolutional networks, in: International Conference...
  • B. Xu, H. Shen, Q. Cao, Y. Qiu, X. Cheng, Graph wavelet neural network, in: International Conference on Learning...
  • Q. Li, Z. Han, X.-M. Wu, Deeper insights into graph convolutional networks for semi-supervised learning, in:...
  • G. Li et al.

    Deepgcns: Can gcns go as deep as cnns?

  • H. Chen, B. Perozzi, Y. Hu, S. Skiena, Harp: Hierarchical representation learning for networks, in: Thirty-Second AAAI...
  • J. Liang, S. Gurukar, S. Parthasarathy, Mile: A multi-level framework for scalable graph embedding, arXiv preprint...
  • H. Gao et al.

    Graph u-nets

  • J. Lee et al.

    Self-attention graph pooling

  • D. Meunier et al.

    Hierarchical modularity in human brain functional networks

    Frontiers in Neuroinformatics

    (2009)
  • R. Mall et al.

    Multilevel hierarchical kernel spectral clustering for real-life large scale complex networks

    PloS One

    (2014)
  • G. Fu et al.

    Learning topological representation for networks via hierarchical sampling

  • A.K. Bhowmick et al.

    Louvainne: Hierarchical louvain method for high quality and scalable network embedding

  • M.E. Newman et al.

    Finding and evaluating community structure in networks

    Physical Review E

    (2004)
  • V.D. Blondel et al.

    Fast unfolding of communities in large networks

    Journal of Statistical Mechanics: Theory and Experiment

    (2008)
  • A. Lancichinetti et al.

    Community detection algorithms: a comparative analysis

    Physical Review E

    (2009)
  • A. Clauset et al.

    Finding community structure in very large networks

    Physical Review E

    (2004)
  • C. Zhuang et al.

    Dual graph convolutional networks for graph-based semi-supervised classification

  • S. Vashishth et al.

    Confidence-based graph convolutional networks for semi-supervised learning

  • S. Abu-El-Haija et al.

    N-gcn: Multi-scale graph convolution for semi-supervised node classification

  • S. Qi et al.

    Learning human-object interactions by graph parsing neural networks

  • P. Ghosh et al.

    Stacked spatio-temporal graph convolutional networks for action segmentation

  • Cited by (0)

    Wenjie Zheng is currently pursuing the master degree with the School of Computer Science and Technology, Anhui University, Hefei, China. His current research interests include data mining, graph neural networks and knowledge graphs.

    Fulan Qian received the PhD degree in computer science from Anhui University in 2016. She is now a associate professor in the School of Computer Science and Technology, Anhui University. Her current research interests include granular computing, social network and recommendation system.

    Shu Zhao received the PhD degree in computer science from Anhui University in 2007. She is now a professor in the School of Computer Science and Technology, Anhui University. Her current research interests include quotient space theory, granular computing, social network and machine learning.

    Yanping Zhang is currently a professor in Anhui University. She received the PhD degree from Anhui University in 2005. Her main research interests include computational intelligence, quotient space theory, artificial neural networks and intelligent information processing, machine learning, and so on. She is the Principal Investigator of some 973 projects and the leader of National Natural Science Foundations of China and Anhui Province.

    View full text