Processing math: 33%
An Energy-Efficient GAN Accelerator With On-Chip Training for Domain-Specific Optimization | IEEE Journals & Magazine | IEEE Xplore

An Energy-Efficient GAN Accelerator With On-Chip Training for Domain-Specific Optimization


Abstract:

Generative adversarial networks (GANs) consist of multiple deep neural networks cooperating and competing with each other. Due to their complex architectures and large fe...Show More

Abstract:

Generative adversarial networks (GANs) consist of multiple deep neural networks cooperating and competing with each other. Due to their complex architectures and large feature map sizes, training GANs requires a huge amount of computations. Moreover, instance normalization (IN) layers in GANs dramatically increase the external memory access (EMA). However, retraining GANs with user-specific data is critical on mobile devices because the pre-trained model outputs distorted images under user-specific conditions. This article proposes a GAN training accelerator to enable energy-efficient domain-specific optimization of GAN with user’s local data. Selective layer retraining (SELRET) picks out layers that are effective in enhancing the quality of the retrained model. Without image quality degradation, the SELRET reduces the required computation by 69%. Moreover, reordering layers for instance normalization (ROLIN) is proposed to reduce the EMA of intermediate data. Through the implementation of the proposed architecture, which splits and reorders the IN layers, 38.7% and 32.2% of overall EMA reduction are achieved in the forward propagation (FP) stage and the error propagation (EP) stage, respectively. The proposed processor is fabricated in a 65-nm CMOS process, showing 0.38-TFLOPS/W energy efficiency. The chip can retrain a face modification GAN with a custom dataset of 256 \times 256 images over 100 epochs under 30 s while only consuming 274 mW. Compared to the previous FPGA implementation, this work improved the retraining performance and energy efficiency by 2 \times and 39 \times , respectively. As a result, the proposed accelerator enables GAN’s domain-specific optimization on a mobile platform.
Published in: IEEE Journal of Solid-State Circuits ( Volume: 56, Issue: 10, October 2021)
Page(s): 2968 - 2980
Date of Publication: 15 July 2021

ISSN Information:

Funding Agency:


References

References is not available for this document.