Abstract
With the development of deep learning, deep learning has become more and more widely used in artificial intelligence. At this stage, the deep neural network (DNN) based on high-performance GPU and CPU devices has achieved remarkable results in the fields of object detection and recognition. The DNNs have also been applied to social media, image processing and video processing. With the improvement of neural networks, the depth and complexity of various neural networks are also increasing. On the basis of the sparsity of DNN weights, our method analyzes the influence of the weights on the feature map and obtains the relations between convolution layers. The sparsity of the network channel is deduced from the L1 norm and the L2 norm. And the weights of the DNN are pruned according to sparsity. In the vgg-16 experiment, we can accelerate the neural network by 2.7 times without affecting the accuracy of the neural network. Compared to the unstructured pruning, structured pruning based on the sparsity can effectively improve the speed of the forward and backward process, which has a certain significance for the application of DNNs.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Sun, F., Wang, C., Gong, L., et al.: A high-performance accelerator for large-scale convolutional neural networks. In: 2017 IEEE International Symposium on Parallel and Distributed Processing with Applications and 2017 IEEE International Conference on Ubiquitous Computing and Communications (ISPA/IUCC), pp. 622–629. IEEE (2017)
Han, S., Mao, H., Dally, W.J.: Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding. In: International Conference on Learning and Representation (ICLR), San Juan, pp. 233–242. IEEE (2016)
Anwar, S., Hwang, K., Sung, W.: Structured pruning of deep convolutional neural networks. ACM J. Emerg. Technol. Comput. Syst. (JETC) 13(3), 32 (2017)
Yu, N., Qiu, S., Hu, X., et al.: Accelerating convolutional neural networks by group-wise 2D-filter pruning. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 2502–2509. IEEE (2017)
He, Y., Zhang, X., Sun, J.: Channel pruning for accelerating very deep neural networks. In: International Conference on Computer Vision (ICCV), vol. 2(6) (2017)
Mao, H., et al.: Exploring the Regularity of Sparse Structure in Convolutional Neural Networks. CoRR (2017)
Han, S., Pool, J., Tran, J., et al.: Learning both weights and connections for efficient neural network. In: Advances in Neural Information Processing Systems, pp. 1135–1143 (2015)
Luo, J.H., Wu, J., Lin, W.: ThiNet: a filter level pruning method for deep neural network compression. In: 2017 IEEE International Conference on Computer Vision (ICCV), Venice, pp. 5068–5076 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
He, M., Zhao, H., Wang, G., Chen, Y., Zhu, L., Gao, Y. (2019). Deep Neural Network Acceleration Method Based on Sparsity. In: Zhai, G., Zhou, J., An, P., Yang, X. (eds) Digital TV and Multimedia Communication. IFTC 2018. Communications in Computer and Information Science, vol 1009. Springer, Singapore. https://doi.org/10.1007/978-981-13-8138-6_11
Download citation
DOI: https://doi.org/10.1007/978-981-13-8138-6_11
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-8137-9
Online ISBN: 978-981-13-8138-6
eBook Packages: Computer ScienceComputer Science (R0)