research-article

Algorithm/architecture co-design for energy-efficient acceleration of multi-task DNN

Authors:

Lee-Sup KimAuthors Info & Claims

DAC '22: Proceedings of the 59th ACM/IEEE Design Automation Conference

Pages 253 - 258

https://doi.org/10.1145/3489517.3530455

Published: 23 August 2022 Publication History

Abstract

Real-world AI applications, such as augmented reality or autonomous driving, require processing multiple CV tasks simultaneously. However, the enormous data size and the memory footprint have been a crucial hurdle for deep neural networks to be applied in resource-constrained devices. To solve the problem, we propose an algorithm/architecture co-design. The proposed algorithmic scheme, named SqueeD, reduces per-task weight and activation size by 21.9x and 2.1x, respectively, by sharing those data between tasks. Moreover, we design architecture and dataflow to minimize DRAM access by fully utilizing benefits from SqueeD. As a result, the proposed architecture reduces the DRAM access increment and energy consumption increment per task by 2.2x and 1.3x, respectively.

References

[1]

J. Deng et al., "ImageNet: A Large-Scale Hierarchical Image Database," in CVPR, 2009.

[2]

R. Mottaghi et al., "The Role of Context for Object Detection and Semantic Segmentation in the Wild" in CVPR, 2014.

[3]

P. K. N. Silberman, D. Hoiem and R. Fergus, "Indoor segmentation and support inference from RGBD images," in ECCV, 2012.

[4]

K. He, X. Zhang, S. Ren, J. Sun, "Deep residual learning for image recognition" in CVPR, 2016.

[5]

L. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, "Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation," in ECCV, 2018.

[6]

H. Kwon et al., "Heterogeneous Dataflow Accelerators for Multi-DNN Workloads," in HPCA, 2021.

[7]

L. Yang et al.,"Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Tasks" in DAC, 2020.

[8]

S. Liu, E. Johns, and A. J. Davison, "End-to-end multi-task learning with attention" in CVPR, 2019.

[9]

K. K. Maninis I. Radosavovic and I. Kokkinos "Attentive single-tasking of multiple tasks" in CVPR, 2019.

[10]

D Guo, A. M. Rush and Y. Kim, "Parameter-Efficient Transfer Learning with Diff Pruning," in ACL, 2021.

[11]

A. Kolesnikov, L. Beyer, X. Zhai, J. Puigcerver, J. Yung, S. Gelly, and N. Houlsby, "Big transfer (BiT): General visual representation learning," arXiv:1912.11370, 2019.

[12]

Mark Horowitz, "1.1 Computing's energy problem (and what we can do about it),". in ISSCC, 2014.

[13]

R. Balasubramonian et al., "CACTI 7: New Tools for Interconnect Exploration in Innovative Off-Chip Memories," ACM Transactions on Architecture and Code Optimization, 2017.

Digital Library

[14]

R. Girshick, J. Donahue, T. Darrell and J. Malik "Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation" in CVPR, 2014.

[15]

M. Nilsback and A. Zisserman, " Automated Flower Classification over a Large Number of Classes" in indian conferecne on Computer Vision, Graphics and Image Processing, 2008.

[16]

P. Welinder et al., "Caltech-UCSD Birds 200" TechnicalReport CNS-TR-2010-001, California Institute of Technol-ogy, 2010.

[17]

A. Krizhevsky and G. Hinton, "Learning multiple layers offeatures from tiny images," Technical report, 2009.

[18]

M. Ryu et al., "Compressing DMA Engine: Leveraging Activation Sparsity for Training Deep Neural Networks" in HPCA, 2018.

[19]

Y. Chen et al., "Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks," in IEEE Journal of Solid-State Circuits, Jan., 2017.

[20]

H. Kwon et al., "Understanding Reuse, Performance, and Hardware Cost of DNN Dataflow: A Data-Centric Approach" in MICRO, 2019.

Digital Library

[21]

Y. Bengio, N. Léonard and A. Courville, "Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation," in arXiv, 2013.

Cited By

Chen YFu YLee MGeorge SLiu YNarayanan VYang HLi X(2023)FAST: A Fully-Concurrent Access SRAM Topology for High Row-Wise Parallelism Applications Based on Dynamic Shift OperationsIEEE Transactions on Circuits and Systems II: Express Briefs10.1109/TCSII.2022.323158970:4(1605-1609)Online publication date: Apr-2023
https://doi.org/10.1109/TCSII.2022.3231589

Recommendations

Energy efficient task allocation for hybrid main memory architecture

Compared with the conventional dynamic random access memory (DRAM), emerging non-volatile memory technologies provide better density and energy efficiency. However, current NVM devices typically suffer from high write power, long write latency and low ...
Memory design for robust and energy-efficient computing systems
Energy-Efficient Buffer Architecture for Flash Memory
MUE '08: Proceedings of the 2008 International Conference on Multimedia and Ubiquitous Engineering

Lower energy consumption is an important consideration in the design of mobile devices. Flash memory is essential in such applications, and one solution for reducing energy consumption is to include a buffer layer in the flash memory storage system. The ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

DAC '22: Proceedings of the 59th ACM/IEEE Design Automation Conference

July 2022

1462 pages

ISBN:9781450391429

DOI:10.1145/3489517

General Chair:
Rob Oshana
NXP

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGDA: ACM Special Interest Group on Design Automation
IEEE CEDA

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 August 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

MSIT (Ministry of Science and ICT), Korea
Samsung Advanced Institute of Technology
National Research Foundation of Korea (NRF)

Conference

DAC '22

Sponsor:

SIGDA

DAC '22: 59th ACM/IEEE Design Automation Conference

July 10 - 14, 2022

California, San Francisco

Acceptance Rates

Overall Acceptance Rate 1,770 of 5,499 submissions, 32%

Upcoming Conference

DAC '25

Sponsor:
sigda

62nd ACM/IEEE Design Automation Conference

June 22 - 26, 2025

San Francisco , CA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
665
Total Downloads

Downloads (Last 12 months)117
Downloads (Last 6 weeks)5

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen YFu YLee MGeorge SLiu YNarayanan VYang HLi X(2023)FAST: A Fully-Concurrent Access SRAM Topology for High Row-Wise Parallelism Applications Based on Dynamic Shift OperationsIEEE Transactions on Circuits and Systems II: Express Briefs10.1109/TCSII.2022.323158970:4(1605-1609)Online publication date: Apr-2023
https://doi.org/10.1109/TCSII.2022.3231589

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten