skip to main content
10.1145/3580305.3599478acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article

Pyramid Graph Neural Network: A Graph Sampling and Filtering Approach for Multi-scale Disentangled Representations

Published: 04 August 2023 Publication History

Abstract

Spectral methods for graph neural networks (GNNs) have achieved great success. Despite their success, many works have shown that existing approaches are mainly focused on low-frequency information which may not be pertinent to the task at hand. Recent efforts have been made to design new graph filters for wider frequency profiles, but it remains an open problem how to learn multi-scale disentangled node embeddings in the graph Fourier domain. In this paper, we propose a graph (signal) sampling and filtering framework, entitled Pyramid Graph Neural Network (PyGNN), which follows the Downsampling-Filtering-Upsampling-Decoding scheme. To be specific, we develop an ω-bandlimited downsampling approach to split input graph into subgraphs for the reduction of high-frequency components, then perform spectral graph filters on subgraphs to achieve node embeddings with different frequency bands, and propose a Laplacian smoothing-based upsampling approach to extrapolate the node embedding on subgraphs to the full set of vertices on the original graph. In the end, we add frequency-aware gated units to decode node embeddings of different frequencies for downstream tasks. Results on both homophilic and heterophilic graph datasets show its superiority over state-of-the-art methods.

Supplementary Material

MP4 File (rtfp126-2min-promo.mp4)
The video presents the paper "Pyramid Graph Neural Network: A Graph Sampling and Filtering Approach for Multi-scale Disentangled Representations".

References

[1]
Aamir Anis, Akshay Gadde, and Antonio Ortega. 2016. Efficient sampling set selection for bandlimited graph signals using graph spectral proxies. IEEE Transactions on Signal Processing, Vol. 64, 14 (2016), 3775--3789.
[2]
Muhammet Balcilar, Guillaume Renton, Pierre Héroux, Benoit Gaüzère, Sébastien Adam, and Paul Honeine. 2020. Analyzing the expressive power of graph neural networks in a spectral perspective. In International Conference on Learning Representations (ICLR).
[3]
Filippo Maria Bianchi, Daniele Grattarola, Lorenzo Livi, and Cesare Alippi. 2021. Graph neural networks with convolutional arma filters. IEEE transactions on pattern analysis and machine intelligence (TPAMI) (2021).
[4]
Deyu Bo, Xiao Wang, Chuan Shi, and Huawei Shen. 2021. Beyond low-frequency information in graph convolutional networks. In Proceedings of the AAAI conference on artificial intelligence (AAAI), Vol. 35. 3950--3957.
[5]
Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann Lecun. 2014. Spectral networks and locally connected networks on graphs. In International Conference on Learning Representations (ICLR).
[6]
Peter J Burt and Edward H Adelson. 1987. The Laplacian pyramid as a compact image code. In Readings in computer vision. Elsevier, 671--679.
[7]
Lei Cai and Shuiwang Ji. 2020. A multi-scale approach for graph link prediction. In Proceedings of the AAAI conference on artificial intelligence (AAAI), Vol. 34. 3308--3315.
[8]
Chao Chen, Haoyu Geng, Gang Zeng, Zhaobing Han, Hua Chai, Xiaokang Yang, and Junchi Yan. 2023. Graph Signal Sampling for Inductive One-Bit Matrix Completion: a Closed-form Solution. In The Eleventh International Conference on Learning Representations (ICLR).
[9]
Chao Chen, Dongsheng Li, Junchi Yan, Hanchi Huang, and Xiaokang Yang. 2021. Scalable and Explainable 1-Bit Matrix Completion via Graph Signal Learning. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI).
[10]
Haochen Chen, Bryan Perozzi, Yifan Hu, and Steven Skiena. 2018b. Harp: Hierarchical representation learning for networks. In Proceedings of the AAAI conference on artificial intelligence (AAAI), Vol. 32.
[11]
Jie Chen, Tengfei Ma, and Cao Xiao. 2018a. FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling. In International Conference on Learning Representations (ICLR).
[12]
Siheng Chen, Rohan Varma, Aliaksei Sandryhaila, and Jelena Kovačević. 2015. Discrete Signal Processing on Graphs: Sampling Theory. IEEE transactions on signal processing, Vol. 63, 24 (2015), 6510--6523.
[13]
Eli Chien, Jianhao Peng, Pan Li, and Olgica Milenkovic. 2021. Adaptive Universal Generalized PageRank Graph Neural Network. In International Conference on Learning Representations (ICLR).
[14]
Fan RK Chung and Fan Chung Graham. 1997. Spectral graph theory. American Mathematical Soc.
[15]
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Neural Information Processing Systems (NIPS). 3844--3852.
[16]
C Deng, Z Zhao, Y Wang, Z Zhang, and Z Feng. 2020. GraphZoom: A Multi-level Spectral Approach for Accurate and Scalable Graph Embedding. In The International Conference on Learning Representations (ICLR).
[17]
Bin Dong. 2017. Sparse representation on graphs by tight wavelet frames and applications. Applied and Computational Harmonic Analysis, Vol. 42, 3 (2017), 452--479.
[18]
Yushun Dong, Kaize Ding, Brian Jalaian, Shuiwang Ji, and Jundong Li. 2021. AdaGNN: Graph Neural Networks with Adaptive Frequency Response Filter. In Proceedings of the ACM International Conference on Information & Knowledge Management (CIKM). 392--401.
[19]
David L Donoho, Iain M Johnstone, Gérard Kerkyacharian, and Dominique Picard. 1995. Wavelet shrinkage: asymptopia? Journal of the Royal Statistical Society: Series B (Methodological), Vol. 57, 2 (1995), 301--337.
[20]
Matthias Fey and Jan E. Lenssen. 2019. Fast Graph Representation Learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds.
[21]
Hongyang Gao and Shuiwang Ji. 2019. Graph u-nets. In International Conference on Machine Learning (ICML). PMLR, 2083--2092.
[22]
Willem H Haemers. 1995. Interlacing eigenvalues and graphs. Linear Algebra and its applications, Vol. 226 (1995), 593--616.
[23]
William L Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Neural Information Processing Systems (NIPS). 1025--1035.
[24]
David K Hammond, Pierre Vandergheynst, and Rémi Gribonval. 2011. Wavelets on graphs via spectral graph theory. Applied and Computational Harmonic Analysis, Vol. 30, 2 (2011), 129--150.
[25]
Mingguo He, Zhewei Wei, Zengfeng Huang, and Hongteng Xu. 2021. BernNet: Learning Arbitrary Graph Spectral Filters via Bernstein Approximation. In Neural Information Processing Systems (NeurIPS).
[26]
Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 639--648.
[27]
Yixuan He, Michael Perlmutter, Gesine Reinert, and Mihai Cucuringu. 2022a. Msgnn: A spectral graph neural network based on a novel magnetic signed laplacian. In Learning on Graphs Conference. PMLR, 40-1.
[28]
Yixuan He, Gesine Reinert, and Mihai Cucuringu. 2022b. DIGRAC: Digraph Clustering Based on Flow Imbalance. In Learning on Graphs Conference. PMLR, 21--1.
[29]
Yixuan He, Gesine Reinert, Songchao Wang, and Mihai Cucuringu. 2022c. SSSNET: Semi-Supervised Signed Network Clustering. In Proceedings of the 2022 SIAM International Conference on Data Mining (SDM). SIAM, 244--252.
[30]
NT Hoang, Takanori Maehara, and Tsuyoshi Murata. 2021. Revisiting Graph Neural Networks: Graph Filtering Perspective. In 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 8376--8383.
[31]
Zengfeng Huang, Shengzhong Zhang, Chong Xi, Tang Liu, and Min Zhou. 2021. Scaling up graph neural networks via graph coarsening. In Proceedings of the ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD). 675--684.
[32]
Elvin Isufi, Andreas Loukas, Andrea Simonetto, and Geert Leus. 2016. Autoregressive moving average graph filtering. IEEE Transactions on Signal Processing, Vol. 65, 2 (2016), 274--288.
[33]
Mingxuan Ju, Shifu Hou, Yujie Fan, Jianan Zhao, Yanfang Ye, and Liang Zhao. 2022. Adaptive kernel graph neural network. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 7051--7058.
[34]
Rudolph Emil Kalman. 1960. A new approach to linear filtering and prediction problems. Journal of basic Engineering, Vol. 82, 1 (1960), 35--45.
[35]
Henry Kenlay, Dorina Thanou, and Xiaowen Dong. 2021. Interpretable stability bounds for spectral graph filters. In International Conference on Machine Learning (ICML). PMLR, 5388--5397.
[36]
Elias Khalil, Hanjun Dai, Yuyu Zhang, Bistra Dilkina, and Le Song. 2017. Learning combinatorial optimization algorithms over graphs. In Neural Information Processing Systems (NeurIPS). 6351--6361.
[37]
Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR).
[38]
Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR).
[39]
Johannes Klicpera, Aleksandar Bojchevski, and Stephan Günnemann. 2018. Predict then Propagate: Graph Neural Networks meet Personalized PageRank. In International Conference on Learning Representations (ICLR).
[40]
Ron Levie, Federico Monti, Xavier Bresson, and Michael M Bronstein. 2018. Cayleynets: Graph convolutional neural networks with complex rational spectral filters. IEEE Transactions on Signal Processing, Vol. 67, 1 (2018), 97--109.
[41]
Guohao Li, Matthias Muller, Ali Thabet, and Bernard Ghanem. 2019. Deepgcns: Can gcns go as deep as cnns?. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 9267--9276.
[42]
Jia Li, Jiajin Li, Yang Liu, Jianwei Yu, Yueting Li, and Hong Cheng. 2021. Deconvolutional Networks on Graph Data. Neural Information Processing Systems (NeurIPS), Vol. 34 (2021).
[43]
Qimai Li, Zhichao Han, and Xiao-Ming Wu. 2018. Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the AAAI conference on artificial intelligence (AAAI).
[44]
Jiaqi Ma, Junwei Deng, and Qiaozhu Mei. 2021. Subgroup generalization and fairness of graph neural networks. Neural Information Processing Systems (NeurIPS), Vol. 34 (2021).
[45]
Stephane G Mallat. 1989. A theory for multiresolution signal decomposition: the wavelet representation. IEEE transactions on pattern analysis and machine intelligence (TPAMI), Vol. 11, 7 (1989), 674--693.
[46]
Antonio Ortega, Pascal Frossard, Jelena Kovačević, José MF Moura, and Pierre Vandergheynst. 2018. Graph signal processing: Overview, challenges, and applications. Proc. IEEE, Vol. 106, 5 (2018), 808--828.
[47]
Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. 1999. The PageRank citation ranking: Bringing order to the web. Technical Report. Stanford InfoLab.
[48]
Jiwoong Park, Minsik Lee, Hyung Jin Chang, Kyuewang Lee, and Jin Young Choi. 2019. Symmetric graph convolutional autoencoder for unsupervised graph representation learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 6519--6528.
[49]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Neural Information Processing Systems (NeurIPS), Vol. 32 (2019).
[50]
Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. 2019. Geom-GCN: Geometric Graph Convolutional Networks. In International Conference on Learning Representations (ICLR).
[51]
Nathanael Perraudin, Benjamin Ricaud, David I Shuman, and Pierre Vandergheynst. 2018. Global and local uncertainty principles for signals on graphs. APSIPA Transactions on Signal and Information Processing, Vol. 7 (2018).
[52]
Isaac Pesenson. 2000. A sampling theorem on homogeneous manifolds. Trans. Amer. Math. Soc., Vol. 352, 9 (2000), 4257--4269.
[53]
Isaac Pesenson. 2008. Sampling in Paley-Wiener spaces on combinatorial graphs. Trans. Amer. Math. Soc., Vol. 360 (2008), 5603--5627.
[54]
Isaac Pesenson. 2009. Variational splines and Paley-Wiener spaces on combinatorial graphs. Constructive Approximation, Vol. 29, 1 (2009), 1--21.
[55]
Jiezhong Qiu, Jian Tang, Hao Ma, Yuxiao Dong, Kuansan Wang, and Jie Tang. 2018. Deepinf: Social influence prediction with deep learning. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD). 2110--2119.
[56]
Akie Sakiyama, Yuichi Tanaka, Toshihisa Tanaka, and Antonio Ortega. 2019. Eigendecomposition-free sampling set selection for graph signals. IEEE Transactions on Signal Processing, Vol. 67, 10 (2019), 2679--2692.
[57]
David I Shuman, Mohammad Javad Faraji, and Pierre Vandergheynst. 2015. A multiscale pyramid transform for graph signals. IEEE Transactions on Signal Processing, Vol. 64, 8 (2015), 2119--2134.
[58]
David I Shuman, Sunil K Narang, Pascal Frossard, Antonio Ortega, and Pierre Vandergheynst. 2013. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Processing Magazine, Vol. 30, 3 (2013), 83--98.
[59]
David I Shuman, Benjamin Ricaud, and Pierre Vandergheynst. 2016. Vertex-frequency analysis on graphs. Applied and Computational Harmonic Analysis, Vol. 40, 2 (2016), 260--291.
[60]
Harald Steck. 2019. Markov Random Fields for Collaborative Filtering. In Neural Information Processing Systems (NeurIPS). 5474--5485.
[61]
Andrei Nikolaevich Tikhonov. 1963. On the solution of ill-posed problems and the method of regularization. In Doklady akademii nauk, Vol. 151. Russian Academy of Sciences, 501--504.
[62]
Petar Velivc ković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. In International Conference on Learning Representations (ICLR).
[63]
Martin Vetterli and Jelena Kovacevic. 1995. Wavelets and subband coding. Prentice-hall.
[64]
Yunhe Wang, Chang Xu, Chao Xu, and Dacheng Tao. 2018. Packing convolutional neural networks in the frequency domain. IEEE transactions on pattern analysis and machine intelligence (TPAMI), Vol. 41, 10 (2018), 2495--2510.
[65]
Qitian Wu, Wentao Zhao, Zenan Li, David P Wipf, and Junchi Yan. 2022. Nodeformer: A scalable graph structure learning transformer for node classification. Neural Information Processing Systems (NeurIPS), Vol. 35 (2022), 27387--27401.
[66]
Bingbing Xu, Huawei Shen, Qi Cao, Yunqi Qiu, and Xueqi Cheng. 2018b. Graph Wavelet Neural Network. In International Conference on Learning Representations (ICLR).
[67]
Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. 2018a. Representation learning on graphs with jumping knowledge networks. In International Conference on Machine Learning (ICML). PMLR, 5453--5462.
[68]
Nuo Xu, Pinghui Wang, Long Chen, Jing Tao, and Junzhou Zhao. 2019. MR-GNN: multi-resolution and dual graph neural network for predicting structured entity interactions. In Proceedings of the International Joint Conference on Artificial Intelligence. 3968--3974.
[69]
Gilad Yehudai, Ethan Fetaya, Eli Meirom, Gal Chechik, and Haggai Maron. 2021. From local structures to size generalization in graph neural networks. In International Conference on Machine Learning (ICML). PMLR, 11975--11986.
[70]
Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD). 974--983.
[71]
Chun-Yang Zhang, Junfeng Hu, Lin Yang, CL Philip Chen, and Zhiliang Yao. 2020. Graph deconvolutional networks. Information Sciences, Vol. 518 (2020), 330--340.
[72]
Xitong Zhang, Yixuan He, Nathan Brugnone, Michael Perlmutter, and Matthew Hirn. 2021. Magnet: A neural network for directed graphs. Neural Information Processing Systems (NeurIPS), Vol. 34 (2021), 27003--27015.
[73]
Wentao Zhao, Qitian Wu, Chenxiao Yang, and Junchi Yan. [n.,d.]. GLOW: Universal and Generalizable Structure Learning for Graph Neural Networks.
[74]
Xuebin Zheng, Bingxin Zhou, Junbin Gao, Yu Guang Wang, Pietro Lio, Ming Li, and Guido Montufar. 2021. How Framelets Enhance Graph Neural Networks. In ICML.
[75]
Xuebin Zheng, Bingxin Zhou, Yu Guang Wang, and Xiaosheng Zhuang. 2022. Decimated Framelet System on Graphs and Fast G-Framelet Transforms. Journal of Machine Learning Research (JMLR), Vol. 23, 18 (2022), 1--68.
[76]
Kaixiong Zhou, Xiao Huang, Daochen Zha, Rui Chen, Li Li, Soo-Hyun Choi, and Xia Hu. 2021. Dirichlet energy constrained learning for deep graph neural networks. Neural Information Processing Systems (NeurIPS), Vol. 34 (2021).
[77]
Hao Zhu and Piotr Koniusz. 2021. Simple spectral graph convolution. In International Conference on Learning Representations (ICLR).
[78]
Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. 2020. Beyond homophily in graph neural networks: Current limitations and effective designs. Neural Information Processing Systems (NeurIPS), Vol. 33 (2020), 7793--7804.
[79]
Meiqi Zhu, Xiao Wang, Chuan Shi, Houye Ji, and Peng Cui. 2021. Interpreting and unifying graph neural networks with an optimization framework. In Proceedings of the Web Conference 2021. 1215--1226.

Cited By

View all
  • (2025)Disentangled Active Learning on GraphsNeural Networks10.1016/j.neunet.2025.107130185(107130)Online publication date: May-2025
  • (2025)ReFNet: Rehearsal-based Graph Lifelong Learning with Multi-resolution Framelet Graph Neural NetworksInformation Sciences10.1016/j.ins.2024.121856(121856)Online publication date: Jan-2025
  • (2024)SLOGProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3694352(55348-55370)Online publication date: 21-Jul-2024
  • Show More Cited By

Index Terms

  1. Pyramid Graph Neural Network: A Graph Sampling and Filtering Approach for Multi-scale Disentangled Representations

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
    August 2023
    5996 pages
    ISBN:9798400701030
    DOI:10.1145/3580305
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 04 August 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. graph algorithms
    2. graph neural networks
    3. spectral graph theory

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    KDD '23
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

    Upcoming Conference

    KDD '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)508
    • Downloads (Last 6 weeks)24
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Disentangled Active Learning on GraphsNeural Networks10.1016/j.neunet.2025.107130185(107130)Online publication date: May-2025
    • (2025)ReFNet: Rehearsal-based Graph Lifelong Learning with Multi-resolution Framelet Graph Neural NetworksInformation Sciences10.1016/j.ins.2024.121856(121856)Online publication date: Jan-2025
    • (2024)SLOGProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3694352(55348-55370)Online publication date: 21-Jul-2024
    • (2023)GraphGLOW: Universal and Generalizable Structure Learning for Graph Neural NetworksProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599373(3525-3536)Online publication date: 4-Aug-2023

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media