Dynamic Network Pruning Based on Local Channel-Wise Relevance

Abstract

In recent years, deep convolutional neural networks (CNNs) significantly boost the various applications, but the high computational complexity of these models hinder the further deployment on device with limited computation resources. Hence, dynamic channel pruning has been recently proposed and widely used for compressing CNN-based models. In this paper, we propose a novel plug-and-play dynamic network pruning module. With very slight extra computation burden, it can achieve the comparable performance as the original model. Specifically, our proposed module measures the importance of each convolutional channel to prune the CNNs with small decrease in accuracy. The module reduces the computation cost by global pooling and channel-wise 1-dimensional convolution that considers the channels’ locality. Comprehensive experimental results demonstrate the effectiveness of our module, which makes a better trade-off between the performance and the acquired computational resources, comparing to its competing methods. In concrete, our dynamic pruning module can reduce 51.1$\%$ FLOPs of VGG16 with only 0.18$\%$ top-1 accuracy degradation on CIFAR10.