research-article

DCIM-GCN: Digital Computing-in-Memory to Efficiently Accelerate Graph Convolutional Networks

Authors:

Ru HuangAuthors Info & Claims

ICCAD '22: Proceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design

Article No.: 46, Pages 1 - 9

https://doi.org/10.1145/3508352.3549465

Published: 22 December 2022 Publication History

Abstract

Computing-in-memory (CIM) is emerging as a promising architecture to accelerate graph convolutional networks (GCNs) normally bounded by redundant and irregular memory transactions. Current analog based CIM requires frequent analog and digital conversions (AD/DA) that dominate the overall area and power consumption. Furthermore, the analog non-ideality degrades the accuracy and reliability of CIM. In this work, an SRAM based digital CIM system is proposed to accelerate memory intensive GCNs, namely DCIM-GCN, which covers innovations from CIM circuit level eliminating costly AD/DA converters to architecture level addressing irregularity and sparsity of graph data. DCIM-GCN achieves 2.07X, 1.76X, and 1.89× speedup and 29.98×, 1.29×, and 3.73× energy efficiency improvement on average over CIM based PIMGCN, TARe, and PIM-GCN, respectively.

References

[1]

Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR), 2017.

[2]

Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S. Yu. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1):4--24, 2021.

[3]

Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric, 2019.

[4]

Zhihao Jia, Sina Lin, Mingyu Gao, Matei Zaharia, and Alex Aiken. Improving the accuracy, scalability, and performance of graph neural networks with roc. In I. Dhillon, D. Papailiopoulos, and V. Sze, editors, Proceedings of Machine Learning and Systems, volume 2, pages 187--198, 2020.

[5]

Ziwei Zhang, Peng Cui, and Wenwu Zhu. Deep learning on graphs: A survey. IEEE Transactions on Knowledge and Data Engineering, 34(1):249--270, 2022.

Digital Library

[6]

Yann LeCun, Yoshua Bengio, and Hinton Geoffrey. Deep learning. Nature, 512:436--444, 2015.

[7]

Connor W. Coley, Wengong Jin, Luke Rogers, Timothy F. Jamison, Tommi S. Jaakkola, William H. Green, Regina Barzilay, and Klavs F. Jensen. A graph-convolutional neural network model for the prediction of chemical reactivity. Chem. Sci., 10:370--377, 2019.

[8]

Huy-Trung Nguyen, Quoc-Dung Ngo, and Van-Hoang Le. Iot botnet detection approach based on psi graph and dgcnn classifier. In 2018 IEEE International Conference on Information Communication and Signal Processing (ICICSP), pages 118--122, 2018.

[9]

Tian Xie and Jeffrey C. Grossman. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties. Phys. Rev. Lett., 120:145301, Apr 2018.

[10]

Hongxia Yang. Aligraph: A comprehensive graph neural network platform. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery amp; Data Mining, KDD '19, page 3165--3166, New York, NY, USA, 2019. Association for Computing Machinery.

Digital Library

[11]

Marinka Zitnik, Monica Agrawal, and Jure Leskovec. Modeling polypharmacy side effects with graph convolutional networks. bioinformatics. Bioinformatics, 34(13):457--466, 2018.

[12]

Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: A review of methods and applications. AI Open, 1:57--81, 2020.

[13]

Mingyu Yan, Zhaodong Chen, Lei Deng, Xiaochun Ye, Zhimin Zhang, Dongrui Fan, and Yuan Xie. Characterizing and understanding gcns on gpu. IEEE Computer Architecture Letters, 19(1):22--25, 2020.

[14]

Zhihui Zhang, Jingwen Leng, Lingxiao Ma, Youshan Miao, Chao Li, and Minyi Guo. Architectural implications of graph neural networks. IEEE Computer Architecture Letters, 19(1):59--62, 2020.

Digital Library

[15]

Milind Kulkarni, Martin Burtscher, Rajeshkar Inkulu, Keshav Pingali, and Calin Cascaval. How much parallelism is there in irregular applications? In Proceedings of the 14th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP '09, page 3--14, New York, NY, USA, 2009. Association for Computing Machinery.

Digital Library

[16]

Nagadastagiri Challapalle, Sahithi Rampalli, Linghao Song, Nandhini Chandramoorthy, Karthik Swaminathan, John Sampson, Yiran Chen, and Vijaykrishnan Narayanan. Gaas-x: Graph analytics accelerator supporting sparse data representation using crossbar architectures. In 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA), pages 433--445, 2020.

Digital Library

[17]

Tong Geng, Ang Li, Runbin Shi, Chunshu Wu, Tianqi Wang, Yanfei Li, Pouya Haghi, Antonino Tumeo, Shuai Che, Steve Reinhardt, and Martin C. Herbordt. Awb-gcn: A graph convolutional network accelerator with runtime workload rebalancing. In 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pages 922--936, 2020.

[18]

Song Han, Xingyu Liu, Huizi Mao, Jing Pu, Ardavan Pedram, Mark A. Horowitz, and William J. Dally. Eie: Efficient inference engine on compressed deep neural network. In 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), pages 243--254, 2016.

Digital Library

[19]

Dongyoung Kim, Junwhan Ahn, and Sungjoo Yoo. A novel zero weight/activation-aware hardware architecture of convolutional neural network. In Design, Automation Test in Europe Conference Exhibition (DATE), 2017, pages 1462--1467, 2017.

[20]

Shijin Zhang, Zidong Du, Lei Zhang, Huiying Lan, Shaoli Liu, Ling Li, Qi Guo, Tianshi Chen, and Yunji Chen. Cambricon-x: An accelerator for sparse neural networks. In 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pages 1--12, 2016.

[21]

A. Abou-Rjeili and G. Karypis. Multilevel algorithms for partitioning power-law graphs. In Proceedings 20th IEEE International Parallel Distributed Processing Symposium, pages 10 pp.-, 2006.

[22]

Joseph E. Gonzalez, Yucheng Low, Haijie Gu, Danny Bickson, and Carlos Guestrin. Powergraph: Distributed graph-parallel computation on natural graphs. In Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation, OSDI'12, page 17--30, USA, 2012. USENIX Association.

Digital Library

[23]

Matthieu Latapy. Main-memory triangle computations for very large (sparse (power-law)) graphs. Theoretical Computer Science, 407(1):458--473, 2008.

Digital Library

[24]

Cong Xie, Ling Yan, Wu-Jun Li, and Zhihua Zhang. Distributed power-law graph computing: Theoretical and empirical analysis. In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 1, NIPS'14, page 1673--1681, Cambridge, MA, USA, 2014. MIT Press.

[25]

Adam Auten, Matthew Tomei, and Rakesh Kumar. Hardware acceleration of graph neural networks. In 2020 57th ACM/IEEE Design Automation Conference (DAC), pages 1--6, 2020.

[26]

Yintao He, Ying Wang, Cheng Liu, Huawei Li, and Xiaowei Li. Tare: Task-adaptive in-situ reram computing for graph learning. In 2021 58th ACM/IEEE Design Automation Conference (DAC), pages 577--582, 2021.

Digital Library

[27]

Mingyu Yan, Lei Deng, Xing Hu, Ling Liang, Yujing Feng, Xiaochun Ye, Zhimin Zhang, Dongrui Fan, and Yuan Xie. Hygcn: A gcn accelerator with hybrid architecture. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA), pages 15--29, 2020.

[28]

Nagadastagiri Challapalle, Karthik Swaminathan, Nandhini Chandramoorthy, and Vijaykrishnan Narayanan. Crossbar based processing in memory accelerator architecture for graph convolutional networks. In 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD), pages 1--9, 2021.

Digital Library

[29]

Tong Geng, Chunshu Wu, Yongan Zhang, Cheng Tan, Chenhao Xie, Haoran You, Martin Herbordt, Yingyan Lin, and Ang Li. I-gcn: A graph convolutional network accelerator with runtime locality enhancement through islandization. In MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO '21, page 1051--1063, New York, NY, USA, 2021. Association for Computing Machinery.

Digital Library

[30]

Chen-Yang Tsai, Chin-Fu Nien, Tz-Ching Yu, Hung-Yu Yeh, and Hsiang-Yun Cheng. Repim: Joint exploitation of activation and weight repetitions for in-reram dnn acceleration. In 2021 58th ACM/IEEE Design Automation Conference (DAC), pages 589--594, 2021.

Digital Library

[31]

Tao Yang, Dongyue Li, Yibo Han, Yilong Zhao, Fangxin Liu, Xiaoyao Liang, Zhezhi He, and Li Jiang. Pimgcn: A reram-based pim design for graph convolutional network acceleration. In 2021 58th ACM/IEEE Design Automation Conference (DAC), pages 583--588, 2021.

Digital Library

[32]

Miao Hu, John Paul Strachan, Zhiyong Li, Emmanuelle M. Grafals, Noraica Davila, Catherine Graves, Sity Lam, Ning Ge, Jianhua Joshua Yang, and R. Stanley Williams. Dot-product engine for neuromorphic computing: Programming 1t1m crossbar to accelerate matrix-vector multiplication. In 2016 53nd ACM/EDAC/IEEE Design Automation Conference (DAC), pages 1--6, 2016.

[33]

Teyuh Chou, Wei Tang, Jacob Botimer, and Zhengya Zhang. Cascade: Connecting rrams to extend analog dataflow in an end-to-end in-memory processing paradigm. MICRO '52, page 114--125, New York, NY, USA, 2019. Association for Computing Machinery.

Digital Library

[34]

Chuan-Jia Jhang, Cheng-Xin Xue, Je-Min Hung, Fu-Chun Chang, and Meng-Fan Chang. Challenges and trends of sram-based computing-in-memory for ai edge devices. IEEE Transactions on Circuits and Systems I: Regular Papers, 68(5):1773--1786, 2021.

[35]

Fengbin Tu, Yiqi Wang, Zihan Wu, Ling Liang, Yufei Ding, Bongjin Kim, Leibo Liu, Shaojun Wei, Yuan Xie, and Shouyi Yin. A 28nm 29.2tflops/w bf16 and 36.5tops/w int8 reconfigurable digital cim processor with unified fp/int pipeline and bitwise in-memory booth multiplication for cloud deep learning acceleration. In 2022 IEEE International Solid- State Circuits Conference (ISSCC), volume 65, pages 1--3, 2022.

[36]

Hidehiro Fujiwara, Haruki Mori, Wei-Chang Zhao, Mei-Chen Chuang, Rawan Naous, Chao-Kai Chuang, Takeshi Hashizume, Dar Sun, Chia-Fu Lee, Kerem Akarvardar, Saman Adham, Tan-Li Chou, Mahmut Ersin Sinangil, Yih Wang, Yu-Der Chih, Yen-Huei Chen, Hung-Jen Liao, and Tsung-Yung Jonathan Chang. A 5-nm 254-tops/w 221-tops/mm2 fully-digital computing-in-memory macro supporting wide-range dynamic-voltage-frequency scaling and simultaneous mac and write operations. In 2022 IEEE International Solid- State Circuits Conference (ISSCC), volume 65, pages 1--3, 2022.

[37]

Baogang Zhang and Rickard Ewetz. Towards resilient deployment of in-memory neural networks with high throughput. In 2021 58th ACM/IEEE Design Automation Conference (DAC), pages 1081--1086, 2021.

Digital Library

[38]

Amr M. S. Tosson, Shimeng Yu, Mohab H. Anis, and Lan Wei. A study of the effect of rram reliability soft errors on the performance of rram-based neuromorphic systems. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 25(11):3125--3137, 2017.

[39]

Yu-Der Chih, Po-Hao Lee, Hidehiro Fujiwara, Yi-Chun Shih, Chia-Fu Lee, Rawan Naous, Yu-Lin Chen, Chieh-Pu Lo, Cheng-Han Lu, Haruki Mori, Wei-Chang Zhao, Dar Sun, Mahmut E. Sinangil, Yen-Huei Chen, Tan-Li Chou, Kerem Akarvardar, Hung-Jen Liao, Yih Wang, Meng-Fan Chang, and Tsung-Yung Jonathan Chang. 16.4 an 89tops/w and 16.3tops/mm2 all-digital sram-based full-precision compute-in memory macro in 22nm for machine-learning edge applications. In 2021 IEEE International Solid- State Circuits Conference (ISSCC), volume 64, pages 252--254, 2021.

Cited By

Sundara Raman SJohn LKulkarni J(2024)NEM-GNN: DAC/ADC-less, Scalable, Reconfigurable, Graph and Sparsity-Aware Near-Memory Accelerator for Graph Neural NetworksACM Transactions on Architecture and Code Optimization10.1145/365260721:2(1-26)Online publication date: 21-May-2024
https://dl.acm.org/doi/10.1145/3652607
Nie CChen GZhang WHe Z(2023)GIM: Versatile GNN Acceleration with Reconfigurable Processing-in-Memory2023 IEEE 41st International Conference on Computer Design (ICCD)10.1109/ICCD58817.2023.00083(499-506)Online publication date: 6-Nov-2023
https://doi.org/10.1109/ICCD58817.2023.00083

Index Terms

DCIM-GCN: Digital Computing-in-Memory to Efficiently Accelerate Graph Convolutional Networks
1. Computer systems organization
  1. Dependable and fault-tolerant systems and networks
    1. Processors and memory architectures
2. Hardware

Index terms have been assigned to the content through auto-classification.

Recommendations

AF-GCN: Completing various graph tasks efficiently via adaptive quadratic frequency response function in graph spectral domain
Abstract
Graph neural network is a breakthrough in applying deep learning to non-Euclidean space. It is widely used for tasks such as social network analysis, molecular function inference, drug repositioning and protein modeling, achieving ...
AFeCAM: An Energy Efficient Analog 1FeFET Content Addressable Memory
GLSVLSI '24: Proceedings of the Great Lakes Symposium on VLSI 2024

Content Addressable Memories (CAMs) have the ability to perform parallel searches, significantly enhancing the computational efficiency of Computing-in-Memory (CiM) architectures. CAMs can be employed in various areas, including DNA sequence analysis, IP ...
Brain-inspired GCN: Modularity-based Siamese simple graph convolutional networks
Abstract
In graph representation learning, Graph Convolutional Networks (GCNs) and their variants have received much attention. However, GCNs encounter oversmoothing as the models get deeper, limiting their ability to aggregate node representations within ...
Highlights
- Low-pass filtered features can alleviate oversmoothing.
- Nodes in the graphs have similar characteristics as brain modules.
- The nonlinearity is not necessary in graph convolutional networks.
- Preservation of modular structure ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICCAD '22: Proceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design

October 2022

1467 pages

ISBN:9781450392174

DOI:10.1145/3508352

Conference Chair:
Tulika Mitra
National University of Singapore
,
Program Chairs:
Evangeline Young
The Chinese University of Hong Kong
,
Jinjun Xiong
University at Buffalo (UB)

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGDA: ACM Special Interest Group on Design Automation

In-Cooperation

IEEE-EDS: Electronic Devices Society
IEEE CAS
IEEE CEDA

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 December 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

111 Project
National Science Foundation of China
Key R&D Program of Zhejiang Province of China

Conference

ICCAD '22

Sponsor:

SIGDA

ICCAD '22: IEEE/ACM International Conference on Computer-Aided Design

October 30 - November 3, 2022

California, San Diego

Acceptance Rates

Overall Acceptance Rate 457 of 1,762 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
339
Total Downloads

Downloads (Last 12 months)71
Downloads (Last 6 weeks)12

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Sundara Raman SJohn LKulkarni J(2024)NEM-GNN: DAC/ADC-less, Scalable, Reconfigurable, Graph and Sparsity-Aware Near-Memory Accelerator for Graph Neural NetworksACM Transactions on Architecture and Code Optimization10.1145/365260721:2(1-26)Online publication date: 21-May-2024
https://dl.acm.org/doi/10.1145/3652607
Nie CChen GZhang WHe Z(2023)GIM: Versatile GNN Acceleration with Reconfigurable Processing-in-Memory2023 IEEE 41st International Conference on Computer Design (ICCD)10.1109/ICCD58817.2023.00083(499-506)Online publication date: 6-Nov-2023
https://doi.org/10.1109/ICCD58817.2023.00083

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten