Cited By
View all- Cong SZhou Y(2022)A review of convolutional neural network architectures and their optimizationsArtificial Intelligence Review10.1007/s10462-022-10213-556:3(1905-1969)Online publication date: 22-Jun-2022
PNFM: A Filter Level Pruning Method for CNN Compression
Author: 
Hanzhi XuAuthors Info & Claims
Hanzhi XuAuthors Info & ClaimsPages 49 - 54
Abstract
We propose a filter level pruning method, named PNFM, to compress the storage space and computational complexity of CNN(Convolution Neural Network). We judge the importance of the pruned filter by the change rate of the feature image output from the latter layer. In order to improve the efficiency of pruning, we also propose a method to make tiny data sets for pruning based on cluster algorithm. We verify the performance of our method on ILSVRC-12 benchmark. It achieves 3.21× FLOPs reduction and 16.92× storage space compression on VGG-16, with only 0.55% top-5 accuracy drop. At the same time, we also rigorously verify the feasibility of the method of making tiny data sets. It can achieve the same accuracy for pruning with at least 10 times less data.
References
[1]
R. Girshick. Fast R-CNN. In ICCV, pages 1440--1448, 2015.
[2]
X. Jia, E. Gavves, B. Fernando, and T. Tuytelaars. Guiding the long-short term memory model for image caption generation. In ICCV, pages 2407--2415, 2015.
[3]
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, pages 770--778, 2016.
[4]
H. Noh, S. Hong, and B. Han. Learning deconvolution network for semantic segmentation. In ICCV, pages 1520--1528, 2015.
[5]
K. Simonyan and A. Zisserman. Very deep convolutional networks for large scale image recognition. In ICLR, pages 1--14, 2015.
[6]
W. Wen, C. Wu, Y. Wang, Y. Chen, and H. Li. Learning structured sparsity in deep neural networks. In NIPS, pages 2074--2082, 2016.
[7]
Y. LeCun, J. S. Denker, and S. A. Solla. Optimal brain damage. In NIPS, pages 598--605, 1990.
[8]
S. Han, J. Pool, J. Tran, and W. Dally. Learning both weights and connections for efficient neural network. In NIPS, pages 1135--1143, 2015.
[9]
H. Hu, R. Peng, Y. W. Tai, and C. K. Tang. Network trimming: A data-driven neuron pruning approach towards efficient deep architectures. In arXiv preprint arXiv:1607.03250, pages 1--9, 2016.
[10]
W. Chen, J. Wilson, S. Tyree, K. Weinberger, and Y. Chen. Compressing neural networks with the hashing trick. In ICML, pages 2285--2294, 2015.
[11]
Y. Gong, L. Liu, M. Yang, and L. Bourdev. Compressing deep convolutional networks using vector quantization. In arXiv preprint arXiv:1412.6115, pages 1--10, 2014.
[12]
S. Han, H. Mao, and W. J. Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. In ICLR, pages 1--14, 2016.
[13]
J. Wu, C. Leng, Y. Wang, Q. Hu, and J. Cheng. Quantized convolutional neural networks for mobile devices. In CVPR, pages 4820--4828, 2016.
[14]
E. L. Denton, W. Zaremba, J. Bruna, Y. LeCun, and R. Fergus. Exploiting linear structure within convolutional networks for efficient evaluation. In NIPS, pages 1269--1277, 2014.
[15]
V. Sindhwani, T. Sainath, and S. Kumar. Structured transforms for small-footprint deep learning. In NIPS, pages 3088--3096, 2015.
[16]
H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf. Pruning filters for efficient ConvNets. In ICLR, pages 1--13, 2017.
[17]
P. Molchanov, S. Tyree, T. Karras, T. Aila, and J. Kautz. Pruning convolutional neural networks for resource efficient transfer learning. In ICLR, pages 1--17, 2017.
[18]
R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra. Grad-CAM: Visual explanations from deep networks via gradient-based localization. In arXiv preprint arXiv:1610.02391, pages 1--24, 2016
[19]
B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba. Learning deep features for discriminative localization. In NIPS, pages 2921--2929, 2016.
[20]
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and F.-F. Li. ImageNet large scale visual recognition challenge. IJCV, 115(3):211--252, 2015.
[21]
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The Caltech-UCSD birds-200-2011 dataset. Technical Report CNS-TR-2011-001, California Institute of Technology, 2011.
[22]
M. Lin, Q. Chen, and S. Yan. Network in network. In arXiv preprint arXiv:1312.4400, pages 1--10, 2013.
Index Terms
- PNFM: A Filter Level Pruning Method for CNN Compression
Recommendations
TBN: Convolutional Neural Network with Ternary Inputs and Binary Weights
Computer Vision – ECCV 2018AbstractDespite the remarkable success of Convolutional Neural Networks (CNNs) on generalized visual tasks, high computational and memory costs restrict their comprehensive applications on consumer electronics (e.g., portable or smart wearable devices). ...
A Novel Image Compression Algorithm Using Ridgelet Transformation with Modified EBCOT
ISECS '09: Proceedings of the 2009 Second International Symposium on Electronic Commerce and Security - Volume 02JPEG2000 is the image compression standard which includes three core techniques: wavelet lifting scheme and EBCOT and MQ arithmetic coder. The EBCOT algorithm uses a Wavelet transform to generate the subband samples, which are quantized and coded. ...
NSTBNet: Toward a nonsubsampled shearlet transform for broad convolutional neural network image denoising
AbstractDeep convolutional neural networks (CNNs) have achieved huge success in the image denoising fields. However, there still have three drawbacks to be overcome: firstly, it is a big cost to train a deeper CNN model for better denoising ...
Comments
Information & Contributors
Information
Published In
December 2020
687 pages
ISBN:9781450388665
DOI:10.1145/3452940
Copyright © 2020 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 17 May 2021
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICITEE2020
ICITEE2020: The 3rd International Conference on Information Technologies and Electrical Engineering
December 3 - 5, 2020
Hunan, Changde City, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 28Total Downloads
- Downloads (Last 12 months)2
- Downloads (Last 6 weeks)0
Reflects downloads up to 28 Feb 2025
Other Metrics
Citations
Cited By
View all- Cong SZhou Y(2022)A review of convolutional neural network architectures and their optimizationsArtificial Intelligence Review10.1007/s10462-022-10213-556:3(1905-1969)Online publication date: 22-Jun-2022
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in