Qingdong Wei
ABSTRACT
Convolution neural networks is a model of deep learning and it is popular in image recognition, object detection and speech recognition. This paper studied Convolution neural networks in detail. Firstly this paper introduced the generation and development of convolution neural networks and illustrated its advantages in image recognition tasks. Then, it summarized the classic structure of convolution neural networks. Next, this paper stated the study trends and summarized five aspects: appropriately simplified networks, reducing over-fitting, increasing gradient signal, deeper networks and randomization. Finally, this paper discussed the problems existed in convolution neural networks and looked forward to the development trends
- Hubel, D.H. & Wiesel, T.N. Receptive fields and functional architecture of monkey striate cortex. J. Physiol. (Lond.) 195, 215--243.Google Scholar
Cross Ref
- Fukushima, K. Biol. Cybernetics (1980) 36: 193.Google Scholar
- Y. LeCun and Y. Bengio: Convolutional Networks for Images, Speech, and Time-Series, in Arbib, M. A. (Eds), The Handbook of Brain Theory and Neural Networks, MIT Press, 1995. Google ScholarDigital Library
- Lécun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition{J}. Proceedings of the IEEE, 1998, 86(11): 2278--2324.Google ScholarCross Ref
- Krizhevsky A, Sutskever I, Hinton G E. ImageNet Classification with Deep Convolutional Neural Networks{J}. Advances in Neural Information Processing Systems, 2012, 25(2):2012. Google ScholarDigital Library
- Nair, Vinod, and G. E. Hinton. "Rectified Linear Units Improve Restricted Boltzmann Machines Vinod Nair." International Conference on Machine Learning 2010: 807--814. Google ScholarDigital Library
- Lin M, Chen Q, Yan S. Network In Network{J}. Computer Science, 2014.Google Scholar
- Springenberg J T, Dosovitskiy A, Brox T, et al. Striving for Simplicity: The All Convolutional Net{J}. Eprint Arxiv, 2014.Google Scholar
- Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions{J}. 2014: 1--9.Google Scholar
- He K, Zhang X, Ren S, et al. Deep Residual Learning for Image Recognition{J}. Computer Science, 2015.Google Scholar
- Srivastava R K, Greff K, Schmidhuber J. Highway Networks{J}. Computer Science, 2015.Google Scholar
- Huang G, Liu Z, Weinberger K Q. Densely Connected Convolutional Networks{J}. 2016.Google Scholar
- Huang G, Sun Y, Liu Z, et al. Deep Networks with Stochastic Depth{J}. 2016.Google Scholar
- Yann LeCun, Yoshua Bengio & Geoffrey Hinton. Deep learning.Google Scholar
- Smith L N, Topin N. Deep Convolutional Neural Network Design Patterns{J}. 2016.Google Scholar
- He K, Zhang X, Ren S, et al. Deep Residual Learning for Image Recognition{J}. Computer Science, 2015.Google Scholar
- Nagi J, Ducatelle F, Caro G A D, et al. Max-pooling convolutional neural networks for vision-based hand gesture recognition{C}// IEEE International Conference on Signal and Image Processing Applications. IEEE, 2011: 342--347.Google Scholar
- Lecun Y, Kavukcuoglu K, Farabet C. Convolutional networks and applications in vision{C}// IEEE International Symposium on Circuits and Systems. IEEE, 2010: 253--256.Google Scholar
- Jarrett K, Kavukcuoglu K, Ranzato M, et al. What is the best multi-stage architecture for object recognition?{J}. 2009, 30(2): 2146--2153.Google Scholar
- Hawkins D M. The problem of overfitting.{J}. ChemInform, 2004, 35(19): 1--12.Google ScholarCross Ref
- Hariharan B, Arbeláez P, Girshick R, et al. Hypercolumns for object segmentation and fine-grained localization{J}. 2015: 447--456.Google Scholar
- Hinton G E, Srivastava N, Krizhevsky A, et al. Improving neural networks by preventing co-adaptation of feature detectors{J}. Computer Science, 2012, 3(4):págs. 212--223.Google Scholar
- Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: a simple way to prevent neural networks from overfitting{J}. Journal of Machine Learning Research, 2014, 15(1): 1929--1958. Google ScholarDigital Library
- Wan L, Zeiler M, Zhang S, et al. Regularization of neural networks using dropconnect{C}// International Conference on Machine Learning. 2013: 1058--1066. Google ScholarDigital Library
- Hinton G E, Dayan P, Frey B J, et al. The "wake-sleep" algorithm for unsupervised neural networks.{J}. Science, 1995, 268(5214): 1158--61.Google ScholarCross Ref
- Schölkopf, B, Platt, J, Hofmann, T. Greedy Layer-Wise Training of Deep Networks{J}. Advances in Neural Information Processing Systems, 2007, 19: 153--160. Google ScholarDigital Library
- Erhan D, Bengio Y, Courville A, et al. Why Does Unsupervised Pre-training Help Deep Learning?{J}. Journal of Machine Learning Research, 2010, 11(3): 625--660. Google ScholarDigital Library
- Erhan D, Manzagol P A, Bengio Y, et al. The Difficulty of Training Deep Architectures and the Effect of Unsupervised Pre-Training.{J}. Immunology of Fungal Infections, 2009, 5: 153--160.Google Scholar
- Erhan D. Understanding Deep Architectures and the Effect of Unsupervised Pre-training{J}. 2011.Google Scholar
- Wu Y N. Data Augmentation{J}. Computer Vision, 2014: 165--166.Google Scholar
- Dyk D A V, Meng X L. The Art of Data Augmentation{J}. Journal of Computational & Graphical Statistics, 2012, 10(1): 1--50.Google Scholar
- Howard A G. Some Improvements on Deep Convolutional Neural Network Based Image Classification{J}. Computer Science, 2013.Google Scholar
- Wong S C, Gatt A, Stamatescu V, et al. Understanding data augmentation for classification: when to warp?{J}. 2016.Google Scholar
- Raiko T, Valpola H, Lecun Y. Deep learning made easier by linear transformations in perceptrons{J}. 2012, 22: 924--932.Google Scholar
- He K, Zhang X, Ren S, et al. Identity Mappings in Deep Residual Networks{J}. 2016.Google Scholar
- Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition{J}. Computer Science, 2014.Google Scholar
- Larsson G, Maire M, Shakhnarovich G. FractalNet: Ultra-Deep Neural Networks without Residuals{J}. 2016.Google Scholar
- A. L. Maas, A. Y. Hannun. And Y. Bengio. Rectifier nonlinearities improve neural network acoustic models, in ICML 2103.Google Scholar
- He K, Zhang X, Ren S, et al. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification{J}. 2015: 1026--1034. Google ScholarDigital Library
- Xu B, Wang N, Chen T, et al. Empirical Evaluation of Rectified Activations in Convolutional Network{J}. Computer Science, 2015.Google Scholar
- Agostinelli F, Hoffman M, Sadowski P, et al. Learning Activation Functions to Improve Deep Neural Networks{J}. Computer Science, 2014.Google Scholar
- Yu D, Wang H, Chen P, et al. Mixed Pooling for Convolutional Neural Networks{C}// the 9th international conference on rough sets and knowledge technology. 2014: 364--375.Google Scholar
- Zeiler M D, Fergus R. Stochastic Pooling for Regularization of Deep Convolutional Neural Networks{J}. Computer Science, 2013.Google Scholar
- Goodfellow I J, Wardefarley D, Mirza M, et al. Maxout Networks{J}. Computer Science, 2013: 1319--1327. Google ScholarDigital Library
Index Terms
- Research Summary of Convolution Neural Network in Image Recognition
Recommendations
A new type of recurrent neural network for handwritten character recognition
ICDAR '95: Proceedings of the Third International Conference on Document Analysis and Recognition (Volume 1) - Volume 1The authors propose a new type of recurrent neural network for handwritten character recognition. The proposed recurrent neural network differs from Jordan and Elman recurrent neural networks in view of functions and architectures because it was ...
A new type of recurrent neural network for handwritten character recognition
ICDAR '95: Proceedings of the Third International Conference on Document Analysis and Recognition (Volume 1) - Volume 1The authors propose a new type of recurrent neural network for handwritten character recognition. The proposed recurrent neural network differs from Jordan and Elman recurrent neural networks in view of functions and architectures because it was ...
A comparative study between convolution neural networks and multi-layer perceptron networks for hand-written digits recognition
This paper presents an investigation that aims at comparing deep learning (DL) and traditional artificial neural networks (ANNs) in the application of hand-written digits recognition (HDR). In our study, convolution neural networks (CNNs) are a ...
Comments