M-GWNN: Multi-granularity graph wavelet neural networks for semi-supervised node classification
Introduction
Since many non-Euclidean data that do not have a grid-like structure can only be represented in the form of a graph, extending neural networks to graph structure has attracted considerable attention from researchers. In this paper, we focus on the task of graph-based semi-supervised node classification using graph convolutional neural networks (GCNs) based on spectral-domain. Spectral CNN [1] first attempted to exploit graph Fourier to implement CNN [2] on graphs and maintain the property of CNN, which is the weight sharing parameter used in training. After that, spectral GCNs based on graph Fourier became increasingly popular. Defferrard et al. [3] utilized localized spectral Chebyshev filters to achieve the graph convolution operation for semi-supervised node classification task. Furthermore, GCN [4] demonstrated considerable power and achieved state-of-art performance on this task. Different from GCN [4], graph wavelet neural network (GWNN) [5] is also a spectral graph convolutional neural network that replaces graph Fourier with graph wavelet, which is sparser, more localized, less computationally expensive than Fourier.
However, we found that GWNN is usually shallow and follows a neighborhood aggregation based on one- or two–hop. Owing to the restricted receptive field, GWNN has a difficulty of obtaining sufficient global information to achieve better node representations. But, if GWNN merely stacks too many convolutional layers, it will reduce the performance because of the model produces the phenomenon of over-smoothing. some researches [6], [7] proved that spectral GCNs models repeatedly apply Laplacian smoothing, the features of vertices in each connected component 1 of the graph will converge to the same value, and further cause the performance of the model learning node representations to significantly decrease.
Recently, few embedding methods have proposed hierarchical approaches that capture global topological information of networks to learn node representations. For instance, HARP [8] and MILE [9] repeatedly compresses a network into a series of small networks based on two node collapsing schemes and learn node embeddings for each compressed network by existing network embedding methods. However, because HARP heuristically combines two closed nodes into a new one, the compressed network may not be able to display the global topology of the input network. Other recent approaches, such as graph U-Net [10] and SAGPool [11], which selectively remove some nodes of graph to compress graph and maximize retention of topological information of hierarchical graph.
In this paper, we utilize the concept of community structure existing in real networks [12], [13], [14], [15], [16] as an effective strategy to realize hierarchical graph coarsening to capture global information of the graph. Nodes that exist in a single community have similar types and are closely connected [17]. Therefore, placing nodes in the same community closely in the embedding space is benefit for graph mining tasks [16], such as semi-supervised node classification. To be specific, dividing a graph recursively into a series of coarser graphs (communities) helps capture the similarities between nodes at different levels of proximity: recursive partitioning essentially creates a hierarchy of communities in a network, where (a) nodes in the same community at the top level of the hierarchy represent similar clusters of nodes with higher-order structure and (b) the lower communities in the hierarchy maintain a one-hop (neighbor) relationship between connected pairs of nodes. Therefore, generating embedding vectors from the presence of nodes in the community at different granularities can maintain embedding quality and is suitable for the task of semi-supervised node classification. Moreover, the Louvain algorithm [18] is a state-of-the-art community detection algorithm [19] based on hierarchical modularity optimization [20], which is used to measure the quality of the results of the community detection algorithm, and also can characterize the closeness of the communities. However, there are existing several problems that directly applying the Louvain algorithm into hierarchical graph coarsening: 1) the number of nodes in communities may exhibit significant inhomogeneity in each coarsening; 2) the uncontrollability of the aggregation times of the Louvain algorithm; 3) there are maybe existing several nodes that are not assigned to any community during the coarsening. Therefore, to better fuse GWNN to implement a hierarchical approach, we propose the Louvain-variant algorithm building on Louvain.
Formally, we propose Multi-granularity Graph Wavelet Neural Networks (M-GWNN), a novel spectral GCNs architecture that leverages the proposed Louvain-variant algorithm and the jump connection to improve the ability of node representations for semi-supervised node classification. As pictured in Fig. 1, M-GWNN mainly consists of several graph coarsening and refining procedures and one output layer. M-GWNN first repeatedly applies the proposed Louvain-variant algorithm to aggregate nodes into supernodes, further refines the coarsened graph symmetrically back to the original by utilizing the jump connection. Moreover, during this process, multiple layers of GWNN are applied to propagate information across entire networks. The core idea of the M-GWNN model is effectively widening the receptive field of the wavelet convolutional filters by capturing node features and graph topological information of varying granularity, which employs the proposed Louvain-variant algorithm to build a hierarchy of successively coarser graph. Furthermore, the M-GWNN utilizes the jump connection to connect receptive fields of varying granularity to efficiently alleviate the speed of over-smoothing, which combining the graph coarsening and symmetrical refining procedures.
Overall, the main contributions of the proposed M-GWNN for semi-supervised node classification are summarized as follows:
- •
We study GWNN from a multi-granularity perspective, which, to the best of our knowledge, has not been studied before. Compared to related work, we propose the Louvain-variant algorithm to capture graph information of varying granularity to obtain global information of the graph. Furthermore, we employ the jump connection to connect receptive fields of varying granularity to stack the challenge of wavelet convolutional filters over-smoothing.
- •
We propose the Louvain-variant algorithm, which is usually used in unsupervised scenarios to solve the semi-supervised problem for the first time. Compared to previous work, a hierarchical clustering method that employs the concept of community structure builds a hierarchy of coarser graph. Moreover, every supernode represents a local topological structure of the original graph, which efficiently captures global information.
- •
We conduct extensive evaluation over four standard datasets for semi-supervised node classification task, and comprehensive experiments show the effectiveness of the proposed M-GWNN model. Especially, compared to state-of-the-art methods, M-GWNN achieves an average error reduction of up to 5.7% in the NELL dataset.
Section snippets
Related works
In this section, we briefly review previous work on graph convolutional neural networks, multi-layer graph representation learning, and algorithms based on graph reduction mechanisms.
Notation introduction
Table 1 shows the main notations used in this paper.
Problem definition
We denote a weighted undirected graph , where is the vertex set of labeled and unlabeled nodes in a graph. Generally, , the goal of semi-supervised node classification is to assign labels for all unlabeled nodes . is the edge set, is the adjacency matrix of a graph with is the graph representations of data vectors in dimension at the layer (including original,
Multi-granularity graph wavelet neural networks (M-GWNN)
In this section, we discuss the proposed multi-granularity graph wavelet neural networks (M-GWNN) in detail.
Datasets
To demonstrate the benefit and effectiveness of the proposed M-GWNN on semi-supervised node classification task, we tested it on four standard benchmark datasets, including three Citation Network datasets: Cora, Citeseer, Pubmed (Sen et al. [49]) and Knowledge Graph dataset; the Never-Ending Language Learning system (NELL) (Carlson et al. [50]; and Yang et al. [51]). Table 2 gives an overview of the four datasets, which are described in detail. Following the experimental setup in GCN [4], we
Conclusion
Throughout this paper, to widen the receptive field and alleviate the speed of the wavelet convolutional filters over-smoothing, we propose multi-granularity graph wavelet neural networks (M-GWNN) for semi-supervised node classification. We first aggregate nodes into supernodes utilizing the proposed Louvain-variant algorithm and then symmetrically refine the coarsened graph to the original by the jump connection. M-GWNN obtains node features and graph topological information of varying
CRediT authorship contribution statement
Wenjie Zheng: Conceptualization, Methodology, Software, Data curation, Formal analysis, Writing - original draft, Writing - review & editing. Fulan Qian: Investigation, Writing - review & editing, Formal analysis, Data curation, Visualization, Funding acquisition. Shu Zhao: Validation, Writing - review & editing, Supervision, Funding acquisition. Yanping Zhang: Validation, Writing - review & editing, Supervision, Funding acquisition.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
This work is supported and funded by the National Natural Science Foundation of China (Grant Nos. 61702003, 61673020, 61876001) and Natural Science Foundation of Anhui Province (Grant No. 1808085MF175). Moreover, we would like to thank the reviewers for their constructive comments and suggestions.
Wenjie Zheng is currently pursuing the master degree with the School of Computer Science and Technology, Anhui University, Hefei, China. His current research interests include data mining, graph neural networks and knowledge graphs.
References (56)
Community detection in graphs
Physics Reports
(2010)- et al.
Wavelets on graphs via spectral graph theory
Applied and Computational Harmonic Analysis
(2011) - et al.
Analyzing process models using graph reduction techniques
Information Systems
(2000) - et al.
Multilevelk-way partitioning scheme for irregular graphs
Journal of Parallel and Distributed computing
(1998) - J. Bruna, W. Zaremba, A. Szlam, Y. LeCun, Spectral networks and locally connected networks on graphs, in: International...
- A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, in:...
- M. Defferrard, X. Bresson, P. Vandergheynst, Convolutional neural networks on graphs with fast localized spectral...
- T. N. Kipf, M. Welling, Semi-supervised classification with graph convolutional networks, in: International Conference...
- B. Xu, H. Shen, Q. Cao, Y. Qiu, X. Cheng, Graph wavelet neural network, in: International Conference on Learning...
- Q. Li, Z. Han, X.-M. Wu, Deeper insights into graph convolutional networks for semi-supervised learning, in:...
Deepgcns: Can gcns go as deep as cnns?
Graph u-nets
Self-attention graph pooling
Hierarchical modularity in human brain functional networks
Frontiers in Neuroinformatics
Multilevel hierarchical kernel spectral clustering for real-life large scale complex networks
PloS One
Learning topological representation for networks via hierarchical sampling
Louvainne: Hierarchical louvain method for high quality and scalable network embedding
Finding and evaluating community structure in networks
Physical Review E
Fast unfolding of communities in large networks
Journal of Statistical Mechanics: Theory and Experiment
Community detection algorithms: a comparative analysis
Physical Review E
Finding community structure in very large networks
Physical Review E
Dual graph convolutional networks for graph-based semi-supervised classification
Confidence-based graph convolutional networks for semi-supervised learning
N-gcn: Multi-scale graph convolution for semi-supervised node classification
Learning human-object interactions by graph parsing neural networks
Stacked spatio-temporal graph convolutional networks for action segmentation
Cited by (0)
Wenjie Zheng is currently pursuing the master degree with the School of Computer Science and Technology, Anhui University, Hefei, China. His current research interests include data mining, graph neural networks and knowledge graphs.
Fulan Qian received the PhD degree in computer science from Anhui University in 2016. She is now a associate professor in the School of Computer Science and Technology, Anhui University. Her current research interests include granular computing, social network and recommendation system.
Shu Zhao received the PhD degree in computer science from Anhui University in 2007. She is now a professor in the School of Computer Science and Technology, Anhui University. Her current research interests include quotient space theory, granular computing, social network and machine learning.
Yanping Zhang is currently a professor in Anhui University. She received the PhD degree from Anhui University in 2005. Her main research interests include computational intelligence, quotient space theory, artificial neural networks and intelligent information processing, machine learning, and so on. She is the Principal Investigator of some 973 projects and the leader of National Natural Science Foundations of China and Anhui Province.