Xupeng Miao
ABSTRACT
Mining from graph-structured data is an integral component of graph data management. A recent trending technique, graph convolutional network (GCN), has gained momentum in the graph mining field, and plays an essential part in numerous graph-related tasks. Although the emerging GCN optimization techniques bring improvements to specific scenarios, they perform diversely in different applications and introduce many trial-and-error costs for practitioners. Moreover, existing GCN models often suffer from oversmoothing problem. Besides, the entanglement of various graph patterns could lead to non-robustness and harm the final performance of GCNs. In this work, we propose a simple yet efficient graph decomposition approach to improve the performance of general graph neural networks. We first empirically study existing graph decomposition methods and propose an automatic connectivity-ware graph decomposition algorithm, DeGNN. To provide a theoretical explanation, we then characterize GCN from the information-theoretic perspective and show that under certain conditions, the mutual information between the output after l layers and the input of GCN converges to 0 exponentially with respect to l. On the other hand, we show that graph decomposition can potentially weaken the condition of such convergence rate, alleviating the information loss when GCN becomes deeper. Extensive experiments on various academic benchmarks and real-world production datasets demonstrate that graph decomposition generally boosts the performance of GNN models. Moreover, our proposed solution DeGNN achieves state-of-the-art performances on almost all these tasks.
Supplemental Material
- 2020. DropEdge openreview. https://openreview.net/forum?id=Hkx1qkrKPrGoogle Scholar
- Sami Abu-El-Haija, Amol Kapoor, Bryan Perozzi, and Joonseok Lee. 2019. N-GCN: Multi-scale Graph Convolution for Semi-supervised Node Classification. In UAI.Google Scholar
- Rianne van den Berg, Thomas N Kipf, and Max Welling. 2017. Graph convolutional matrix completion. arXiv preprint arXiv:1706.02263 (2017).Google Scholar
- Shaosheng Cao, Xinxing Yang, Cen Chen, Jun Zhou, Xiaolong Li, and Yuan Qi. 2019. TitAnt: Online Real-time Transaction Fraud Detection in Ant Financial. PVLDB, Vol. 12, 12 (2019), 2082--2093.Google Scholar
- Jie Chen, Tengfei Ma, and Cao Xiao. 2018. FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling. In ICLR.Google Scholar
- Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, and Cho-Jui Hsieh. 2019. Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks. In SIGKDD. 257--266.Google Scholar
- Yodsawalai Chodpathumwan, Amirhossein Aleyasen, Arash Termehchy, and Yizhou Sun. 2015. Universal-DB: Towards Representation Independent Graph Analytics. VLDB, Vol. 8, 12 (2015), 2016--2019.Google Scholar
- Wenfei Fan, Kun He, Qian Li, and Yue Wang. 2020. Graph algorithms: parallelization and scalability. Sci. China Inf. Sci., Vol. 63, 10 (2020), 1--21.Google ScholarCross Ref
- Hongyang Gao, Zhengyang Wang, and Shuiwang Ji. 2018. Large-scale learnable graph convolutional networks. In SIGKDD. 1416--1424.Google Scholar
- Joseph E. Gonzalez, Yucheng Low, Haijie Gu, Danny Bickson, and Carlos Guestrin. 2012. PowerGraph: Distributed Graph-Parallel Computation on Natural Graphs. In OSDI. 17--30.Google Scholar
- Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In NeurIPS. 1024--1034.Google Scholar
- Louis Jachiet, Pierre Genevè s, Nils Gesbert, and Nabil Layaïda. 2020. On the Optimization of Recursive Relational Queries: Application to Graph Queries. In SIGMOD. 681--697.Google Scholar
- Chathura Kankanamge, Siddhartha Sahu, Amine Mhedbhi, Jeremy Chen, and Semih Salihoglu. 2017. Graphflow: An Active Graph Database. In SIGMOD. 1695--1698.Google ScholarDigital Library
- George Karypis and Vipin Kumar. 1998. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM Journal on scientific Computing, Vol. 20, 1 (1998), 359--392.Google ScholarDigital Library
- Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR.Google Scholar
- Johannes Klicpera, Aleksandar Bojchevski, and Stephan Gü nnemann. 2019. Predict then Propagate: Graph Neural Networks meet Personalized PageRank. In ICLR.Google Scholar
- Boris Knyazev, Xiao Lin, Mohamed R Amer, and Graham W Taylor. 2018. Spectral Multigraph Networks for Discovering and Fusing Relationships in Molecules. arXiv preprint arXiv:1811.09595 (2018).Google Scholar
- Guohao Li, Matthias Müller, Ali Thabet, and Bernard Ghanem. 2019. Can GCNs Go as Deep as CNNs? arXiv preprint arXiv:1904.03751 (2019).Google Scholar
- Qimai Li, Zhichao Han, and Xiao-Ming Wu. 2018. Deeper insights into graph convolutional networks for semi-supervised learning. In AAAI.Google Scholar
- Renjie Liao, Marc Brockschmidt, Daniel Tarlow, Alexander L. Gaunt, Raquel Urtasun, and Richard S. Zemel. 2018. Graph Partition Neural Networks for Semi-Supervised Classification. In ICLR.Google Scholar
- Ziqi Liu, Zhengwei Wu, Zhiqiang Zhang, Jun Zhou, Shuang Yang, Le Song, and Yuan Qi. 2020. Bandit Samplers for Training Graph Neural Networks. In NeurIPS.Google Scholar
- Sitao Luan, Mingde Zhao, Xiao-Wen Chang, and Doina Precup. 2019. Break the Ceiling: Stronger Multi-scale Deep Graph Convolutional Networks. In NeurIPS.Google Scholar
- Jianxin Ma, Peng Cui, Kun Kuang, Xin Wang, and Wenwu Zhu. 2019. Disentangled Graph Convolutional Networks. In ICML, Vol. 97. 4212--4221.Google Scholar
- X. Miao, L. Ma , Z. Yang, Y. Shao, B. Cui, L. Yu, and J. Jiang. 2020. CuWide: Towards Efficient Flow-based Training for Sparse Wide Models on GPUs. TKDE (2020), 1--1. https://doi.org/10.1109/TKDE.2020.3038109Google Scholar
- Kenta Oono and Taiji Suzuki. 2019. On Asymptotic Behaviors of Graph CNNs from Dynamical Systems Perspective. arXiv preprint arXiv:1905.10947 (2019).Google Scholar
- Chanyoung Park, Carl Yang, Qi Zhu, Donghyun Kim, Hwanjo Yu, and Jiawei Han. 2020. Unsupervised Differentiable Multi-aspect Network Embedding. In SIGKDD. 1435--1445.Google Scholar
- Zhen Peng, Wenbing Huang, Minnan Luo, Qinghua Zheng, Yu Rong, Tingyang Xu, and Junzhou Huang. 2020. Graph Representation Learning via Graphical Mutual Information Maximization. In WWW. 259--270.Google Scholar
- Hannu Reittu, Ilkka Norros, Tomi R228;ty, Marianna Bolla, and Fülöp Bazsó. 2019. Regular Decomposition of Large Graphs: Foundation of a Sampling Approach to Stochastic Block Model Fitting. Data Sci. Eng., Vol. 4, 1 (2019), 44--60.Google Scholar
- Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. 2019. DropEdge: Towards Deep Graph Convolutional Networks on Node Classification. In ICLR.Google Scholar
- Andrew M Saxe, Yamini Bansal, Joel Dapello, Madhu Advani, Artemy Kolchinsky, Brendan D Tracey, and David D Cox. 2019. On the information bottleneck theory of deep learning. Journal of Statistical Mechanics: Theory and Experiment, Vol. 2019, 12 (2019), 124020.Google ScholarCross Ref
- Ohad Shamir, Sivan Sabato, and Naftali Tishby. 2010. Learning and generalization with the information bottleneck. Theoretical Computer Science, Vol. 411, 29--30 (2010).Google ScholarDigital Library
- Felipe Petroski Such, Shagan Sah, Miguel Alexander Dominguez, Suhas Pillai, Chao Zhang, Andrew Michael, Nathan D Cahill, and Raymond Ptucha. 2017. Robust spatial filtering with graph convolutional neural networks. IEEE Journal of Selected Topics in Signal Processing, Vol. 11, 6 (2017), 884--896.Google ScholarCross Ref
- Emre Telatar. 1999. Capacity of multi?antenna Gaussian channels. European transactions on telecommunications, Vol. 10 (1999), 585--595. Issue 6.Google Scholar
- Petar Velivc ković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph attention networks. ICLR (2018).Google Scholar
- Petar Velickovic, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R. Devon Hjelm. 2019. Deep Graph Infomax. In ICLR.Google Scholar
- Felix Wu, Amauri H. Souza Jr., Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Q. Weinberger. 2019 a. Simplifying Graph Convolutional Networks. In ICML.Google Scholar
- Shiwen Wu, Yuanxing Zhang, Chengliang Gao, Kaigui Bian, and Bin Cui. 2020. GARG: Anonymous Recommendation of Point-of-Interest in Mobile Networks by Graph Convolution Network. Data Sci. Eng., Vol. 5, 4 (2020), 433--447.Google Scholar
- Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S Yu. 2019 b. A comprehensive survey on graph neural networks. arXiv preprint arXiv:1901.00596 (2019).Google Scholar
- Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How Powerful are Graph Neural Networks?. In ICLR.Google Scholar
- Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. 2018. Representation Learning on Graphs with Jumping Knowledge Networks. In ICML. 5449--5458.Google Scholar
- Hanqing Zeng, Hongkuan Zhou, Ajitesh Srivastava, Rajgopal Kannan, and Viktor K. Prasanna. 2020. GraphSAINT: Graph Sampling Based Inductive Learning Method. In ICLR.Google Scholar
- Wentao Zhang, Xupeng Miao, Yingxia Shao, Jiawei Jiang, Lei Chen, Olivier Ruas, and Bin Cui. 2020. Reliable Data Distillation on Graph Convolutional Network. In SIGMOD. 1399--1414.Google Scholar
- Da Zheng, Minjie Wang, Quan Gan, Zheng Zhang, and George Karypis. 2020. Scalable Graph Neural Networks with Deep Graph Library. In SIGKDD. 3521--3522.Google Scholar
- Hao Zhong and Hong Mei. 2020. Learning a graph-based classifier for fault localization. Sci. China Inf. Sci., Vol. 63, 6 (2020).Google ScholarCross Ref
- Rong Zhu, Kun Zhao, Hongxia Yang, Wei Lin, Chang Zhou, Baole Ai, Yong Li, and Jingren Zhou. 2019. AliGraph: A Comprehensive Graph Neural Network Platform. VLDB, Vol. 12, 12 (2019), 2094--2105.Google Scholar
- Chenyi Zhuang and Qiang Ma. 2018. Dual graph convolutional networks for graph-based semi-supervised classification. In WWW. 499--508.Google Scholar
Index Terms
- DeGNN: Improving Graph Neural Networks with Graph Decomposition
Recommendations
Decompositions of Graphs into Fans and Single Edges
Given two graphs G and H, an H-decomposition of G is a partition of the edge set of G such that each part is either a single edge or forms a graph isomorphic to H. Let ï źn,H be the smallest number ï ź such that any graph G of order n admits an H-...
An extremal problem for vertex partition of complete multipartite graphs
For a graph G and a family H of graphs, a vertex partition of G is called an H -decomposition, if every part induces a graph isomorphic to one of H . For 1 a k , let A ( k , a ) denote the graph which is a join of an empty graph of order a and a ...
Minimum H-decompositions of graphs: Edge-critical case
For a given graph H let @f"H(n) be the maximum number of parts that are needed to partition the edge set of any graph on n vertices such that every member of the partition is either a single edge or it is isomorphic to H. Pikhurko and Sousa conjectured ...
Comments