Haoyu Geng
ABSTRACT
Spectral methods for graph neural networks (GNNs) have achieved great success. Despite their success, many works have shown that existing approaches are mainly focused on low-frequency information which may not be pertinent to the task at hand. Recent efforts have been made to design new graph filters for wider frequency profiles, but it remains an open problem how to learn multi-scale disentangled node embeddings in the graph Fourier domain. In this paper, we propose a graph (signal) sampling and filtering framework, entitled Pyramid Graph Neural Network (PyGNN), which follows the Downsampling-Filtering-Upsampling-Decoding scheme. To be specific, we develop an ω-bandlimited downsampling approach to split input graph into subgraphs for the reduction of high-frequency components, then perform spectral graph filters on subgraphs to achieve node embeddings with different frequency bands, and propose a Laplacian smoothing-based upsampling approach to extrapolate the node embedding on subgraphs to the full set of vertices on the original graph. In the end, we add frequency-aware gated units to decode node embeddings of different frequencies for downstream tasks. Results on both homophilic and heterophilic graph datasets show its superiority over state-of-the-art methods.
Supplemental Material
- Aamir Anis, Akshay Gadde, and Antonio Ortega. 2016. Efficient sampling set selection for bandlimited graph signals using graph spectral proxies. IEEE Transactions on Signal Processing , Vol. 64, 14 (2016), 3775--3789.Google Scholar
Digital Library
- Muhammet Balcilar, Guillaume Renton, Pierre Héroux, Benoit Gaüzère, Sébastien Adam, and Paul Honeine. 2020. Analyzing the expressive power of graph neural networks in a spectral perspective. In International Conference on Learning Representations (ICLR).Google Scholar
- Filippo Maria Bianchi, Daniele Grattarola, Lorenzo Livi, and Cesare Alippi. 2021. Graph neural networks with convolutional arma filters. IEEE transactions on pattern analysis and machine intelligence (TPAMI) (2021).Google ScholarCross Ref
- Deyu Bo, Xiao Wang, Chuan Shi, and Huawei Shen. 2021. Beyond low-frequency information in graph convolutional networks. In Proceedings of the AAAI conference on artificial intelligence (AAAI), Vol. 35. 3950--3957.Google ScholarCross Ref
- Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann Lecun. 2014. Spectral networks and locally connected networks on graphs. In International Conference on Learning Representations (ICLR).Google Scholar
- Peter J Burt and Edward H Adelson. 1987. The Laplacian pyramid as a compact image code. In Readings in computer vision. Elsevier, 671--679.Google Scholar
- Lei Cai and Shuiwang Ji. 2020. A multi-scale approach for graph link prediction. In Proceedings of the AAAI conference on artificial intelligence (AAAI), Vol. 34. 3308--3315.Google ScholarCross Ref
- Chao Chen, Haoyu Geng, Gang Zeng, Zhaobing Han, Hua Chai, Xiaokang Yang, and Junchi Yan. 2023. Graph Signal Sampling for Inductive One-Bit Matrix Completion: a Closed-form Solution. In The Eleventh International Conference on Learning Representations (ICLR).Google Scholar
- Chao Chen, Dongsheng Li, Junchi Yan, Hanchi Huang, and Xiaokang Yang. 2021. Scalable and Explainable 1-Bit Matrix Completion via Graph Signal Learning. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI).Google ScholarCross Ref
- Haochen Chen, Bryan Perozzi, Yifan Hu, and Steven Skiena. 2018b. Harp: Hierarchical representation learning for networks. In Proceedings of the AAAI conference on artificial intelligence (AAAI), Vol. 32.Google ScholarCross Ref
- Jie Chen, Tengfei Ma, and Cao Xiao. 2018a. FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling. In International Conference on Learning Representations (ICLR).Google Scholar
- Siheng Chen, Rohan Varma, Aliaksei Sandryhaila, and Jelena Kovačević. 2015. Discrete Signal Processing on Graphs: Sampling Theory. IEEE transactions on signal processing, Vol. 63, 24 (2015), 6510--6523.Google Scholar
- Eli Chien, Jianhao Peng, Pan Li, and Olgica Milenkovic. 2021. Adaptive Universal Generalized PageRank Graph Neural Network. In International Conference on Learning Representations (ICLR).Google Scholar
- Fan RK Chung and Fan Chung Graham. 1997. Spectral graph theory. American Mathematical Soc.Google Scholar
- Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Neural Information Processing Systems (NIPS). 3844--3852.Google Scholar
- C Deng, Z Zhao, Y Wang, Z Zhang, and Z Feng. 2020. GraphZoom: A Multi-level Spectral Approach for Accurate and Scalable Graph Embedding. In The International Conference on Learning Representations (ICLR).Google Scholar
- Bin Dong. 2017. Sparse representation on graphs by tight wavelet frames and applications. Applied and Computational Harmonic Analysis, Vol. 42, 3 (2017), 452--479.Google ScholarCross Ref
- Yushun Dong, Kaize Ding, Brian Jalaian, Shuiwang Ji, and Jundong Li. 2021. AdaGNN: Graph Neural Networks with Adaptive Frequency Response Filter. In Proceedings of the ACM International Conference on Information & Knowledge Management (CIKM). 392--401.Google ScholarDigital Library
- David L Donoho, Iain M Johnstone, Gérard Kerkyacharian, and Dominique Picard. 1995. Wavelet shrinkage: asymptopia? Journal of the Royal Statistical Society: Series B (Methodological), Vol. 57, 2 (1995), 301--337.Google ScholarCross Ref
- Matthias Fey and Jan E. Lenssen. 2019. Fast Graph Representation Learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds.Google Scholar
- Hongyang Gao and Shuiwang Ji. 2019. Graph u-nets. In International Conference on Machine Learning (ICML). PMLR, 2083--2092.Google Scholar
- Willem H Haemers. 1995. Interlacing eigenvalues and graphs. Linear Algebra and its applications, Vol. 226 (1995), 593--616.Google Scholar
- William L Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Neural Information Processing Systems (NIPS). 1025--1035.Google Scholar
- David K Hammond, Pierre Vandergheynst, and Rémi Gribonval. 2011. Wavelets on graphs via spectral graph theory. Applied and Computational Harmonic Analysis, Vol. 30, 2 (2011), 129--150.Google ScholarCross Ref
- Mingguo He, Zhewei Wei, Zengfeng Huang, and Hongteng Xu. 2021. BernNet: Learning Arbitrary Graph Spectral Filters via Bernstein Approximation. In Neural Information Processing Systems (NeurIPS).Google Scholar
- Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 639--648.Google ScholarDigital Library
- Yixuan He, Michael Perlmutter, Gesine Reinert, and Mihai Cucuringu. 2022a. Msgnn: A spectral graph neural network based on a novel magnetic signed laplacian. In Learning on Graphs Conference. PMLR, 40-1.Google Scholar
- Yixuan He, Gesine Reinert, and Mihai Cucuringu. 2022b. DIGRAC: Digraph Clustering Based on Flow Imbalance. In Learning on Graphs Conference. PMLR, 21--1.Google Scholar
- Yixuan He, Gesine Reinert, Songchao Wang, and Mihai Cucuringu. 2022c. SSSNET: Semi-Supervised Signed Network Clustering. In Proceedings of the 2022 SIAM International Conference on Data Mining (SDM). SIAM, 244--252.Google ScholarCross Ref
- NT Hoang, Takanori Maehara, and Tsuyoshi Murata. 2021. Revisiting Graph Neural Networks: Graph Filtering Perspective. In 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 8376--8383.Google Scholar
- Zengfeng Huang, Shengzhong Zhang, Chong Xi, Tang Liu, and Min Zhou. 2021. Scaling up graph neural networks via graph coarsening. In Proceedings of the ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD). 675--684.Google ScholarDigital Library
- Elvin Isufi, Andreas Loukas, Andrea Simonetto, and Geert Leus. 2016. Autoregressive moving average graph filtering. IEEE Transactions on Signal Processing, Vol. 65, 2 (2016), 274--288.Google ScholarDigital Library
- Mingxuan Ju, Shifu Hou, Yujie Fan, Jianan Zhao, Yanfang Ye, and Liang Zhao. 2022. Adaptive kernel graph neural network. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 7051--7058.Google ScholarCross Ref
- Rudolph Emil Kalman. 1960. A new approach to linear filtering and prediction problems. Journal of basic Engineering, Vol. 82, 1 (1960), 35--45.Google ScholarCross Ref
- Henry Kenlay, Dorina Thanou, and Xiaowen Dong. 2021. Interpretable stability bounds for spectral graph filters. In International Conference on Machine Learning (ICML). PMLR, 5388--5397.Google Scholar
- Elias Khalil, Hanjun Dai, Yuyu Zhang, Bistra Dilkina, and Le Song. 2017. Learning combinatorial optimization algorithms over graphs. In Neural Information Processing Systems (NeurIPS). 6351--6361.Google Scholar
- Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR).Google Scholar
- Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR).Google Scholar
- Johannes Klicpera, Aleksandar Bojchevski, and Stephan Günnemann. 2018. Predict then Propagate: Graph Neural Networks meet Personalized PageRank. In International Conference on Learning Representations (ICLR).Google Scholar
- Ron Levie, Federico Monti, Xavier Bresson, and Michael M Bronstein. 2018. Cayleynets: Graph convolutional neural networks with complex rational spectral filters. IEEE Transactions on Signal Processing, Vol. 67, 1 (2018), 97--109.Google ScholarDigital Library
- Guohao Li, Matthias Muller, Ali Thabet, and Bernard Ghanem. 2019. Deepgcns: Can gcns go as deep as cnns?. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 9267--9276.Google ScholarCross Ref
- Jia Li, Jiajin Li, Yang Liu, Jianwei Yu, Yueting Li, and Hong Cheng. 2021. Deconvolutional Networks on Graph Data. Neural Information Processing Systems (NeurIPS), Vol. 34 (2021).Google Scholar
- Qimai Li, Zhichao Han, and Xiao-Ming Wu. 2018. Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the AAAI conference on artificial intelligence (AAAI).Google ScholarCross Ref
- Jiaqi Ma, Junwei Deng, and Qiaozhu Mei. 2021. Subgroup generalization and fairness of graph neural networks. Neural Information Processing Systems (NeurIPS), Vol. 34 (2021).Google Scholar
- Stephane G Mallat. 1989. A theory for multiresolution signal decomposition: the wavelet representation. IEEE transactions on pattern analysis and machine intelligence (TPAMI), Vol. 11, 7 (1989), 674--693.Google Scholar
- Antonio Ortega, Pascal Frossard, Jelena Kovačević, José MF Moura, and Pierre Vandergheynst. 2018. Graph signal processing: Overview, challenges, and applications. Proc. IEEE, Vol. 106, 5 (2018), 808--828.Google ScholarCross Ref
- Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. 1999. The PageRank citation ranking: Bringing order to the web. Technical Report. Stanford InfoLab.Google Scholar
- Jiwoong Park, Minsik Lee, Hyung Jin Chang, Kyuewang Lee, and Jin Young Choi. 2019. Symmetric graph convolutional autoencoder for unsupervised graph representation learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 6519--6528.Google ScholarCross Ref
- Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Neural Information Processing Systems (NeurIPS) , Vol. 32 (2019).Google Scholar
- Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. 2019. Geom-GCN: Geometric Graph Convolutional Networks. In International Conference on Learning Representations (ICLR).Google Scholar
- Nathanael Perraudin, Benjamin Ricaud, David I Shuman, and Pierre Vandergheynst. 2018. Global and local uncertainty principles for signals on graphs. APSIPA Transactions on Signal and Information Processing, Vol. 7 (2018).Google ScholarCross Ref
- Isaac Pesenson. 2000. A sampling theorem on homogeneous manifolds. Trans. Amer. Math. Soc., Vol. 352, 9 (2000), 4257--4269.Google ScholarCross Ref
- Isaac Pesenson. 2008. Sampling in Paley-Wiener spaces on combinatorial graphs. Trans. Amer. Math. Soc., Vol. 360 (2008), 5603--5627.Google ScholarCross Ref
- Isaac Pesenson. 2009. Variational splines and Paley-Wiener spaces on combinatorial graphs. Constructive Approximation, Vol. 29, 1 (2009), 1--21.Google ScholarCross Ref
- Jiezhong Qiu, Jian Tang, Hao Ma, Yuxiao Dong, Kuansan Wang, and Jie Tang. 2018. Deepinf: Social influence prediction with deep learning. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD). 2110--2119.Google ScholarDigital Library
- Akie Sakiyama, Yuichi Tanaka, Toshihisa Tanaka, and Antonio Ortega. 2019. Eigendecomposition-free sampling set selection for graph signals. IEEE Transactions on Signal Processing, Vol. 67, 10 (2019), 2679--2692.Google ScholarDigital Library
- David I Shuman, Mohammad Javad Faraji, and Pierre Vandergheynst. 2015. A multiscale pyramid transform for graph signals. IEEE Transactions on Signal Processing, Vol. 64, 8 (2015), 2119--2134.Google ScholarDigital Library
- David I Shuman, Sunil K Narang, Pascal Frossard, Antonio Ortega, and Pierre Vandergheynst. 2013. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Processing Magazine, Vol. 30, 3 (2013), 83--98.Google ScholarCross Ref
- David I Shuman, Benjamin Ricaud, and Pierre Vandergheynst. 2016. Vertex-frequency analysis on graphs. Applied and Computational Harmonic Analysis, Vol. 40, 2 (2016), 260--291.Google ScholarCross Ref
- Harald Steck. 2019. Markov Random Fields for Collaborative Filtering. In Neural Information Processing Systems (NeurIPS). 5474--5485.Google Scholar
- Andrei Nikolaevich Tikhonov. 1963. On the solution of ill-posed problems and the method of regularization. In Doklady akademii nauk, Vol. 151. Russian Academy of Sciences, 501--504.Google Scholar
- Petar Velivc ković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. In International Conference on Learning Representations (ICLR).Google Scholar
- Martin Vetterli and Jelena Kovacevic. 1995. Wavelets and subband coding. Prentice-hall.Google Scholar
- Yunhe Wang, Chang Xu, Chao Xu, and Dacheng Tao. 2018. Packing convolutional neural networks in the frequency domain. IEEE transactions on pattern analysis and machine intelligence (TPAMI), Vol. 41, 10 (2018), 2495--2510.Google Scholar
- Qitian Wu, Wentao Zhao, Zenan Li, David P Wipf, and Junchi Yan. 2022. Nodeformer: A scalable graph structure learning transformer for node classification. Neural Information Processing Systems (NeurIPS), Vol. 35 (2022), 27387--27401.Google Scholar
- Bingbing Xu, Huawei Shen, Qi Cao, Yunqi Qiu, and Xueqi Cheng. 2018b. Graph Wavelet Neural Network. In International Conference on Learning Representations (ICLR).Google Scholar
- Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. 2018a. Representation learning on graphs with jumping knowledge networks. In International Conference on Machine Learning (ICML). PMLR, 5453--5462.Google Scholar
- Nuo Xu, Pinghui Wang, Long Chen, Jing Tao, and Junzhou Zhao. 2019. MR-GNN: multi-resolution and dual graph neural network for predicting structured entity interactions. In Proceedings of the International Joint Conference on Artificial Intelligence. 3968--3974.Google ScholarCross Ref
- Gilad Yehudai, Ethan Fetaya, Eli Meirom, Gal Chechik, and Haggai Maron. 2021. From local structures to size generalization in graph neural networks. In International Conference on Machine Learning (ICML). PMLR, 11975--11986.Google Scholar
- Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD). 974--983.Google ScholarDigital Library
- Chun-Yang Zhang, Junfeng Hu, Lin Yang, CL Philip Chen, and Zhiliang Yao. 2020. Graph deconvolutional networks. Information Sciences, Vol. 518 (2020), 330--340.Google ScholarDigital Library
- Xitong Zhang, Yixuan He, Nathan Brugnone, Michael Perlmutter, and Matthew Hirn. 2021. Magnet: A neural network for directed graphs. Neural Information Processing Systems (NeurIPS), Vol. 34 (2021), 27003--27015.Google Scholar
- Wentao Zhao, Qitian Wu, Chenxiao Yang, and Junchi Yan. [n.,d.]. GLOW: Universal and Generalizable Structure Learning for Graph Neural Networks.Google Scholar
- Xuebin Zheng, Bingxin Zhou, Junbin Gao, Yu Guang Wang, Pietro Lio, Ming Li, and Guido Montufar. 2021. How Framelets Enhance Graph Neural Networks. In ICML.Google Scholar
- Xuebin Zheng, Bingxin Zhou, Yu Guang Wang, and Xiaosheng Zhuang. 2022. Decimated Framelet System on Graphs and Fast G-Framelet Transforms. Journal of Machine Learning Research (JMLR), Vol. 23, 18 (2022), 1--68.Google Scholar
- Kaixiong Zhou, Xiao Huang, Daochen Zha, Rui Chen, Li Li, Soo-Hyun Choi, and Xia Hu. 2021. Dirichlet energy constrained learning for deep graph neural networks. Neural Information Processing Systems (NeurIPS), Vol. 34 (2021).Google Scholar
- Hao Zhu and Piotr Koniusz. 2021. Simple spectral graph convolution. In International Conference on Learning Representations (ICLR).Google Scholar
- Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. 2020. Beyond homophily in graph neural networks: Current limitations and effective designs. Neural Information Processing Systems (NeurIPS), Vol. 33 (2020), 7793--7804.Google Scholar
- Meiqi Zhu, Xiao Wang, Chuan Shi, Houye Ji, and Peng Cui. 2021. Interpreting and unifying graph neural networks with an optimization framework. In Proceedings of the Web Conference 2021. 1215--1226.Google ScholarDigital Library
Index Terms
- Pyramid Graph Neural Network: A Graph Sampling and Filtering Approach for Multi-scale Disentangled Representations
Recommendations
Finding a chain graph in a bipartite permutation graph
We present a polynomial-time algorithm for solving Subgraph Isomorphism where the base graphs are bipartite permutation graphs and the pattern graphs are chain graphs. Subgraph Isomorphism is studied on graph classes.A polynomial-time algorithm is given ...
Structure of squares and efficient domination in graph classes
For any two vertices u and v in a graph G, d G ( u , v ) denote the distance between u and v in G. The square of a graph G = ( V , E ) is the graph G 2 = ( V , E 2 ) such that u v ź E 2 if and only if d G ( u , v ) ź { 1 , 2 } . An efficient dominating ...
The graph sandwich problem for 1-join composition is NP-complete
A graph is a 1-join composition if its vertex set can be partitioned into four nonempty sets AL, AR , SL and SR such that: every vertex of AL is adjacent to every vertex of AR; no vertex of SL is adjacent to vertex of AR∪SR; no vertex of SR is adjacent ...
Comments