Towards Efficient Convolutional Neural Networks Through Low-Error Filter Saliency Estimation
Filter saliency based channel pruning is a state-of-the-art method for deep convolutional neural network compression and acceleration. This channel pruning method ranks the importance of individual filter by estimating its impact of each filter’s removal on the training loss, and then remove the least important filters and fine-tune the remnant network. In this work, we propose a systematic channel pruning method that significantly reduces the estimation error of filter saliency. Different from existing approaches, our method largely reduces the magnitude of parameters in a network by introducing alternating direction method of multipliers (ADMM) into the pre-training procedure. Therefore, the estimation of filter saliency based on Taylor expansion is significantly improved. Extensive experiments with various benchmark network architectures and datasets demonstrate that the proposed method has a much improved unimportant filter selection capability and outperform state-of-the-art channel pruning method.
- Research Organization:
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC05-00OR22725
- OSTI ID:
- 1561628
- Resource Relation:
- Journal Volume: 11671; Conference: 16th Pacific Rim International Conference on Artificial Intelligence (PRICAI 2019) - cuvu, , Fiji - 8/29/2019 12:00:00 AM-8/31/2019 12:00:00 AM
- Country of Publication:
- United States
- Language:
- English
Similar Records
Wootz: a compiler-based framework for fast CNN pruning via composability
Saliency-driven system models for cell analysis with deep learning