short-paper

AdaptiveGCN: Efficient GCN Through Adaptively Sparsifying Graphs

Authors:

Li JiangAuthors Info & Claims

CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge Management

Pages 3206 - 3210

https://doi.org/10.1145/3459637.3482049

Published: 30 October 2021 Publication History

Abstract

Graph Convolutional Networks (GCNs) have become the prevailing approach to efficiently learn representations from graph-structured data. Current GCN models adopt a neighborhood aggregation mechanism based on two primary operations, aggregation and combination. The workload of these two processes is determined by the input graph structure, making the graph input the bottleneck of processing GCN. Meanwhile, a large amount of task-irrelevant information in the graphs would hurt the model generalization performance. This brings the opportunity of studying how to remove the redundancy in the graphs. In this paper, we aim to accelerate GCN models by removing the task-irrelevant edges in the graph. We present AdaptiveGCN, an efficient and supervised graph sparsification framework. AdaptiveGCN adopts an edge predictor module to get edge selection strategies by learning the downstream task feedback signals for each GCN layer separately and adaptively in the training stage, then only inference with the selected edges in the test stage to speed up the GCN computation. The experimental results indicate that AdaptiveGCN could yield 43% (on CPU) and 39% (on GPU) GCN model speed-up averagely with comparable model performance on public graph learning benchmarks.

References

[1]

Wenzheng Feng, Jie Zhang, Yuxiao Dong, Yu Han, Huanbo Luan, Qian Xu, Qiang Yang, Evgeny Kharlamov, and Jie Tang. 2020. Graph Random Neural Networks for Semi-Supervised Learning on Graphs. Advances in Neural Information Processing Systems (2020).

[2]

Matthias Fey and Jan Eric Lenssen. 2019. Fast graph representation learning with PyTorch Geometric. arXiv preprint arXiv:1903.02428 (2019).

[3]

Tong Geng, Ang Li, Runbin Shi, Chunshu Wu, Tianqi Wang, Yanfei Li, Pouya Haghi, Antonino Tumeo, Shuai Che, Steve Reinhardt, et al. 2020. AWB-GCN: A graph convolutional network accelerator with runtime workload rebalancing. In IEEE/ACM International Symposium on Microarchitecture.

[4]

Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems.

Digital Library

[5]

Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. 2020. Open graph benchmark: Datasets for machine learning on graphs. arXiv preprint arXiv:2005.00687 (2020).

[6]

Eric Jang, Shixiang Gu, and Ben Poole. 2016. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144 (2016).

[7]

Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).

[8]

Chris J Maddison, Andriy Mnih, and Yee Whye Teh. 2016. The concrete distribution: A continuous relaxation of discrete random variables. arXiv preprint arXiv:1611.00712 (2016).

[9]

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems.

Digital Library

[10]

Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. 2019. Dropedge: Towards deep graph convolutional networks on node classification. In International Conference on Learning Representations.

[11]

Lu Wang, Wenchao Yu, Wei Wang, Wei Cheng, Wei Zhang, Hongyuan Zha, Xiaofeng He, and Haifeng Chen. 2019. Learning Robust Representations with Graph Denoising Policy Network. In IEEE International Conference on Data Mining.

[12]

Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems (2020).

[13]

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2018. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 (2018).

[14]

Mingyu Yan, Lei Deng, Xing Hu, Ling Liang, Yujing Feng, Xiaochun Ye, Zhimin Zhang, Dongrui Fan, and Yuan Xie. 2020. Hygcn: A gcn accelerator with hybrid architecture. In IEEE International Symposium on High Performance Computer Architecture.

[15]

Hanqing Zeng, Hongkuan Zhou, Ajitesh Srivastava, Rajgopal Kannan, and Viktor Prasanna. 2019. Graphsaint: Graph sampling based inductive learning method. arXiv preprint arXiv:1907.04931 (2019).

[16]

Tong Zhao, Yozen Liu, Leonardo Neves, Oliver Woodford, Meng Jiang, and Neil Shah. 2020. Data Augmentation for Graph Neural Networks. arXiv preprint arXiv:2006.06830 (2020).

[17]

Cheng Zheng, Bo Zong, Wei Cheng, Dongjin Song, Jingchao Ni, Wenchao Yu, Haifeng Chen, and Wei Wang. 2020. Robust graph representation learning via neural sparsification. In International Conference on Machine Learning.

Cited By

Ba CInterdonato RIenco DGaito S(2025)MARA: A deep learning based framework for multilayer graph simplificationNeurocomputing10.1016/j.neucom.2024.128712612(128712)Online publication date: Jan-2025
https://doi.org/10.1016/j.neucom.2024.128712
Liu YZhang YWang QYuan HAi XYu G(2025)NeutronSketch: An in-depth exploration of redundancy in large-scale graph neural network trainingKnowledge-Based Systems10.1016/j.knosys.2024.112786309(112786)Online publication date: Jan-2025
https://doi.org/10.1016/j.knosys.2024.112786
Merkel NToussing PMayer RJacobsen H(2024)Can Graph Reordering Speed Up Graph Neural Network Training? An Experimental StudyProceedings of the VLDB Endowment10.14778/3705829.370584618:2(293-307)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.14778/3705829.3705846
Show More Cited By

Index Terms

AdaptiveGCN: Efficient GCN Through Adaptively Sparsifying Graphs
1. Computing methodologies
  1. Machine learning

Recommendations

Semi-supervised Learning with Graph Convolutional Networks Based on Hypergraph
Abstract
Graph convolutional networks (GCNs), which rely on graph structures to aggregate information of neighbors to output robust node embeddings, have been becoming a popular model for semi-supervised classification tasks. However, most existing GCNs ...
Exploiting node-feature bipartite graph in graph convolutional networks
Abstract
In recent years, Graph Convolutional Networks (GCNs), which extend convolutional neural networks to graph structure, have achieved great success on many graph learning tasks by fusing structure and feature information, such as node ...
Adaptive Graph Learning for Semi-supervised Classification of GCNs
Databases Theory and Applications
Abstract
Graph convolutional networks (GCNs) have achieved great success in social networks and other aspects. However, existing GCN methods generally require a wealth of domain knowledge to obtain the data graph, which cannot guarantee that the graph is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge Management

October 2021

4966 pages

ISBN:9781450384469

DOI:10.1145/3459637

General Chairs:
Gianluca Demartini
The University of Queensland, Australia
,
Guido Zuccon
The University of Queensland, Australia
,
Program Chairs:
J. Shane Culpepper
RMIT University, Australia
,
Zi Huang
The University of Queensland, Australia
,
Hanghang Tong
University of Illinois at Urbana-Champaign, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 October 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Conference

CIKM '21

Sponsor:

CIKM '21: The 30th ACM International Conference on Information and Knowledge Management

November 1 - 5, 2021

Queensland, Virtual Event, Australia

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
385
Total Downloads

Downloads (Last 12 months)63
Downloads (Last 6 weeks)6

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ba CInterdonato RIenco DGaito S(2025)MARA: A deep learning based framework for multilayer graph simplificationNeurocomputing10.1016/j.neucom.2024.128712612(128712)Online publication date: Jan-2025
https://doi.org/10.1016/j.neucom.2024.128712
Liu YZhang YWang QYuan HAi XYu G(2025)NeutronSketch: An in-depth exploration of redundancy in large-scale graph neural network trainingKnowledge-Based Systems10.1016/j.knosys.2024.112786309(112786)Online publication date: Jan-2025
https://doi.org/10.1016/j.knosys.2024.112786
Merkel NToussing PMayer RJacobsen H(2024)Can Graph Reordering Speed Up Graph Neural Network Training? An Experimental StudyProceedings of the VLDB Endowment10.14778/3705829.370584618:2(293-307)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.14778/3705829.3705846
Gu ZVucetic BChikkam KAliberti PHardjawana W(2024)Graph Representation Learning for Contention and Interference Management in Wireless NetworksIEEE/ACM Transactions on Networking10.1109/TNET.2024.335593532:3(2479-2494)Online publication date: Jun-2024
https://doi.org/10.1109/TNET.2024.3355935
Shi PGao J(2024)Speed-up Implicit Graph Neural Diffusion Model: A Simplified and Robust Strategy2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10651260(1-9)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10651260
Liu XXiong XYan MXue RPan SPei SDeng LYe XFan D(2024)DropNaE: Alleviating irregularity for large-scale graph representation learningNeural Networks10.1016/j.neunet.2024.106930(106930)Online publication date: Dec-2024
https://doi.org/10.1016/j.neunet.2024.106930
Wang ZZhao HLiang FShi CFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Node-dependent Semantic Search over Heterogeneous Graph Neural NetworksProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614989(2646-2655)Online publication date: 21-Oct-2023
https://doi.org/10.1145/3583780.3614989
Cai ZYuan GZhuang XWang SQiao SZhu M(2023)Adaptive self-propagation graph convolutional network for recommendationWorld Wide Web10.1007/s11280-023-01182-y26:5(3183-3206)Online publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1007/s11280-023-01182-y
Gao ZLu ZWang JYing SShi J(2022)A Convolutional Neural Network and Graph Convolutional Network Based Framework for Classification of Breast Histopathological ImagesIEEE Journal of Biomedical and Health Informatics10.1109/JBHI.2022.315367126:7(3163-3173)Online publication date: Jul-2022
https://doi.org/10.1109/JBHI.2022.3153671

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten