Bo Hu
ABSTRACT
Aiming at the problem that the existing computer vision detection algorithm based on deep learning consumes a lot of memory and computing resources, this paper improves the structure of convolutional neural network and proposes a lightweight algorithm for defect detection of industrial products by network pruning. The proposed algorithm uses the residual network to divide VGG-16 into different residual modules, introduces the sparse constraint of penalty factor and the attenuation constraint of weight matrix to measure the importance of each residual module, and cuts the residual modules with low importance, so as to greatly reduce the number of parameter learning in the deep residual network. Experiments show that this method can retain the accuracy, precision, recall and F1 score of the original network, and greatly improve the speed of network training to meet the real-time needs of product appearance defect detection.
- Han S, Pool J, Tran J, Learning both weights and connections for efficient neural network[J]. Advances in neural information processing systems, 2015:1135-1143.Google Scholar
- Wen W, Wu C, Wang Y, Learning structured sparsity in deep neural networks[J]. Advances in neural information processing systems, 2016:2074-2078.Google Scholar
- Lebedev V, Lempitsky V. Fast convnets using group-wise brain damage[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 2554-2564.Google Scholar
- Mao H, Han S, Pool J, Exploring the granularity of sparsity in convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2017: 13-20.Google Scholar
- Iandola F N, Han S, Moskewicz M W, SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0.5 MB model size[J]. arXiv preprint arXiv:1602.07360, 2016.Google Scholar
- Gholami A, Kwon K, Wu B, Squeezenext: Hardware-aware neural network design[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2018: 1638-1647.Google Scholar
- Howard A G, Zhu M, Chen B, Mobilenets: Efficient convolutional neural networks for mobile vision applications[J]. arXiv preprint arXiv:1704.04861, 2017.Google Scholar
- Sandler M, Howard A, Zhu M, Mobilenetv2: Inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 4510-4520.Google Scholar
- Zhang X, Zhou X, Lin M, Shufflenet: An extremely efficient convolutional neural network for mobile devices[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 6848-6856.Google Scholar
- Ma N, Zhang X, Zheng H T, Shufflenet v2: Practical guidelines for efficient cnn architecture design[C]//Proceedings of the European conference on computer vision (ECCV). 2018: 116-131.Google Scholar
- Denil M, Shakibi B, Dinh L, Predicting parameters in deep learning[J]. Advances in neural information processing systems, 2013:2148-2156.Google Scholar
- Wen W, Wu C, Wang Y, Learning structured sparsity in deep neural networks[J]. Advances in neural information processing systems, 2016, 2074-2082.Google Scholar
- Zhou H, Alvarez J M, Porikli F. Less is more: Towards compact cnns[C]//European conference on computer vision. Springer, Cham, 2016: 662-677.Google Scholar
- Liu Z, Li J, Shen Z, Learning efficient convolutional networks through network slimming[C]//Proceedings of the IEEE international conference on computer vision. 2017: 2736-2744.Google Scholar
- Huang Z, Wang N. Data-driven sparse structure selection for deep neural networks[C]//Proceedings of the European conference on computer vision (ECCV). 2018: 304-320.Google Scholar
- Lin S, Ji R, Yan C, Towards optimal structured cnn pruning via generative adversarial learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 2790-2799.Google Scholar
- Molchanov P, Tyree S, Karras T, Pruning convolutional neural networks for resource efficient inference[J]. arXiv preprint arXiv:1611.06440, 2016.Google Scholar
- Lin S, Ji R, Li Y, Accelerating Convolutional Networks via Global & Dynamic Filter Pruning[C]//IJCAI. 2018, 2(7): 2425–2432.Google Scholar
- Molchanov P, Mallya A, Tyree S, Importance estimation for neural network pruning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 11264-11272.Google Scholar
- Kim E, Ahn C, Oh S. Nestednet: Learning nested sparse structures in deep neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 8669-8678.Google Scholar
- Lee H, Lee D S, Kang H, Sparse brain network recovery under compressed sensing[J]. IEEE Transactions on Medical Imaging, 2011, 30(5): 1154-1165.Google ScholarCross Ref
- Huang Z, Wang N. Data-driven sparse structure selection for deep neural networks[C]//Proceedings of the European conference on computer vision (ECCV). 2018: 304-320.Google Scholar
- Parashar A, Rhu M, Mukkara A, SCNN: An accelerator for compressed-sparse convolutional neural networks[J]. ACM SIGARCH computer architecture news, 2017, 45(2): 27-40.Google Scholar
- Wang P, Ji Y, Hong C, SNrram: An efficient sparse neural network computation architecture based on resistive random-access memory[C]//2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC). IEEE, 2018: 1-6.Google ScholarDigital Library
- Tang A, Quan P, Niu L, A Survey for Sparse Regularization Based Compression Methods[J]. Annals of Data Science, 2022: 1-28.Google Scholar
- Nesterov Y. Efficiency of coordinate descent methods on huge-scale optimization problems[J]. SIAM Journal on Optimization, 2012, 22(2): 341-362.Google ScholarCross Ref
- Johnson R, Zhang T. Accelerating stochastic gradient descent using predictive variance reduction[J]. Advances in neural information processing systems, 2013, 26.Google Scholar
- Sutskever I, Martens J, Dahl G, On the importance of initialization and momentum in deep learning[C]//International conference on machine learning. PMLR, 2013: 1139-1147.Google Scholar
- Duchi J, Hazan E, Singer Y. Adaptive subgradient methods for online learning and stochastic optimization[J]. Journal of machine learning research, 2011, 12(7).Google Scholar
- Zeiler M D. Adadelta: an adaptive learning rate method[J]. arXiv preprint arXiv:1212.5701, 2012.Google Scholar
- Reddi S J, Kale S, Kumar S. On the convergence of adam and beyond[J]. arXiv preprint arXiv:1904.09237, 2019.Google Scholar
- Kingma D P, Ba J. Adam: A method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980, 2014.Google Scholar
- Neyshabur B, Salakhutdinov R R, Srebro N. Path-sgd: Path-normalized optimization in deep neural networks[J]. Advances in neural information processing systems, 2015:2422-2430.Google Scholar
- Bartlett P L, Foster D J, Telgarsky M J. Spectrally-normalized margin bounds for neural networks[J]. Advances in neural information processing systems, 2017:6241-6250.Google Scholar
- Keskar N S, Mudigere D, Nocedal J, On large-batch training for deep learning: Generalization gap and sharp minima[J]. arXiv preprint arXiv:1609.04836, 2016.Google Scholar
- Dziugaite G K, Roy D M. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data[J]. arXiv preprint arXiv:1703.11008, 2017.Google Scholar
- TIBSHIRANI R. Regression shrinkage and selection via the lasso[J]. Journal of the Royal Statistical Society. Series B (Methodological), 1996:267–288.Google Scholar
- ZOU H. The adaptive lasso and its oracle properties[J]. Journal of the American statistical association,2006, 101 (476):1418–288.Google Scholar
- Zhang J, Wang Y. Temporally adaptive-dynamic sparse network for modeling disease progression[C]//2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). IEEE, 2020: 1900-1904.Google Scholar
Recommendations
Improved inception-residual convolutional neural network for object recognition
AbstractMachine learning and computer vision have driven many of the greatest advances in the modeling of Deep Convolutional Neural Networks (DCNNs). Nowadays, most of the research has been focused on improving recognition accuracy with better DCNN models ...
Scene text recognition using residual convolutional recurrent neural network
Text is a significant tool for human communication, and text recognition in scene images becomes more and more important. In this paper, we propose a residual convolutional recurrent neural network for solving the task of scene text recognition. The ...
Real-time Fabric Defect Detection based on Lightweight Convolutional Neural Network
ICCPR '19: Proceedings of the 2019 8th International Conference on Computing and Pattern RecognitionFabric defect detection is an important link for quality control in a textile factory. Deep convolutional neural network (CNN) has made great progress in the field of target detection, and has proven applicable in fabric defect detection. However, the ...
Comments