Abstract
Residual convolutional neural network (R-CNN) has become a promising method for image recognition in deep learning applications. The application accuracy, as a key indicator, has a close relationship with filter weights in trained R-CNN models. In order to make filters work at full capacity, we find out that lower relevancy between filters in the same layer promotes higher accuracy for R-CNN applications. Furthermore, we propose an improved R-CNN model training method to acquire a higher accuracy and a better generalization ability. In this paper, the main focus is to control the update of filter weights during model training. The key mechanism is achieved through computing the relevancy between filters in the same layer. The relevancy is quantified by a correlation coefficient, e.g., Pearson Correlation Coefficient (PCC). The mechanism takes a larger probability to utilize the updated filter weights with a lower correlation coefficient, and vice versa. In order to validate our proposal, we construct an experiment through PCC on residual networks. The experiment demonstrates that the improved model training method is a promising mean with better generalization ability and higher recognition accuracy (0.52%-1.83%) for residual networks.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
https://pan.baidu.com/s/1kSgoF1WkkAEiJobM1PtXFQ with the extraction code: 159e
Table 1 Construction of 56-layers ResNet
References
Classification datasets results. http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html
Dehshibi MM, Bastanfard A (2010) A new algorithm for age recognition from facial images. Signal Process 90(8):2431–2444. https://doi.org/10.1016/j.sigpro.2010.02.015
Galton F (1886) Regression towards mediocrity in hereditary stature. J Roy Anthropol Inst Great Brit Ireland 15:246–263
Genovese A, Piuri V, Scotti F (2019) Towards explainable face aging with generative adversarial networks. In: 2019 IEEE international conference on image processing (ICIP). IEEE, pp 3806–3810
Han S, Liu X, Mao H, Pu J, Pedram A, Horowitz MA, Dally WJ (2016) EIE: Efficient inference engine on compressed deep neural network. ACM SIGARCH Comput Architect News 44(3):243–254
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision. Springer International Publishing, pp 630–645
Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W et al (2017) Mobilenets:, Efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
LeCun Y, Boser BE, Denker JS, Henderson D, Howard RE, Hubbard WE, Jackel LD (1990) Handwritten digit recognition with a back-propagation network. In: Advances in neural information processing systems, pp 396–404
Li X, Ding L, Wang L, Cao F (2017) FPGA accelerates deep residual learning for image recognition. In: 2017 IEEE 2nd Information technology, networking, electronic and automation control conference (ITNEC), Chengdu. https://doi.org/10.1109/ITNEC.2017.8284852, pp 837–840
Pearson K (1895) Note on regression and inheritance in the case of two parents. Proc R Soc Lond 58:240–242
Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC (2018) Mobilenetv2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4510–4520
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D et al (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9
Zhang C, Fang Z, Zhou P, Pan P, Cong J (2016) Caffeine: towards uniformed representation and acceleration for deep convolutional neural networks. In: Proceedings of the 35th international conference on computer-aided design. ACM, p 12
Zhang C, Li P, Sun G, Guan Y, Xiao B, Cong J (2015) Optimizing fpga-based accelerator design for deep convolutional neural networks. In: Proceedings of the 2015 ACM/SIGDA international symposium on field-programmable gate arrays, pp 161–170
Zhao Y, Zhang X, Fang B et al (2019) A deep residual networks accelerator on FPGA. In: 2019 eleventh international conference on advanced computational intelligence (ICACI), Guilin, China, pp 13–17
Acknowledgements
We would like to thank the anonymous reviewers for their valuable comments and professional suggestions on previous drafts of this paper. Moreover, we would like to thank my colleagues for their technical help, including Xin Zhang, Li Wang, Zhenhua Guo at Inspur, and Hongwei Wang from Kingsoft, Xiaomin Zhu from Jinan. This work is supported by Major Innovation Project of Shandong Province (Grant No. 2019TSLH0201): Research, Development and Industrialization of Artificial Intelligence Chip for Streaming Media.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Li, X., Li, R., Zhao, Y. et al. An improved model training method for residual convolutional neural networks in deep learning. Multimed Tools Appl 80, 6811–6821 (2021). https://doi.org/10.1007/s11042-020-10031-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-10031-3