Abstract
Handwritten digit recognition is an important but challenging task. However, how to build an efficient artificial neural network architecture that can match human performance on the task of recognition of handwritten digit is still a difficult problem. In this paper, we proposed a new very deep neural network architecture for handwritten digit recognition. What is remarkable is that we did not depart from the classical convolutional neural networks architecture, but pushed it to the limit by substantially increasing the depth. By a carefully crafted design, we proposed two different basic building block and increase the depth of the network while keeping the computational budget constant. On the very competitive MNIST handwriting benchmark, our method achieve the best error rate ever reported on the original dataset (\(0.47\,\% \pm 0.05\,\%\)), without data distortion or model combination, demonstrating the superiority of our work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2323 (1998)
Simard, P.Y., Steinkraus, D., Platt, J.C.: Best practices for convolutional neural networks applied to visual document analysis. In: 7th IEEE International Conference on Document Analysis and Recognition, pp. 958–963 (2003)
Ciresan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3642–3649 (2012)
Srivastava, N.: Improving neural networks with dropout. University of Toronto (2013)
Lin, M., Chen, Q., Yan, S.: Network in network. ArXiv preprint (2014). arXiv:1312.4400
Ciresan, D.C., Meier, U., Gambardella, L.M., Schmidhuber, J., Cire, D.C., Meier, U., Gambardella, L.M.: Handwritten digit recognition with a committee of deep neural nets on gpus. ArXiv preprint (2011). arXiv:1103.4487
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (ICLR), pp. 1–14 (2015)
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015)
Vedaldi, A., Lenc, K.: MatConvNet. In: 23th ACM International Conference on Multimedia, pp. 689–692 (2015)
Salakhutdinov, R., Hinton, G.E.: Deep Boltzmann machines. In: 12th International Conference on Artificial Intelligence and Statistics, pp. 448–455 (2009)
Ranzato, M.A., Poultney, C., Chopra, S., Lecun, Y.: Efficient learning of sparse representations with an energy-based model. In: Advances in Neural Information Processing Systems (NIPS), pp. 1137–1134 (2006)
Goodfellow, I.J., Warde-Farley, D., Mirza, M., Courville, A., Bengio, Y.: Maxout networks. ArXiv preprint (2013). arXiv:1302.4389
Goodfellow, I., Courville, A., Bengio, Y.: Joint training of deep Boltzmann machines for classification. In: International Conference on Learning Representations Workshops (ICLRW) (2013)
Deng, L., Yu, D.: Deep convex net: a scalable architecture for speech pattern classification. In: Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), pp. 2285–2288 (2011)
Rifai, S., Dauphin, Y.: The manifold tangent classifier. In: Advances in Neural Information Processing Systems (NIPS), pp. 2294–2302 (2011)
Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R.: Improving neural networks by preventing co-adaptation of feature detectors. ArXiv preprint (2012). arXiv:1207.0580
Wan, L., Zeiler, M., Zhang, S., LeCun, Y., Fergus, R.: Regularization of neural networks using dropconnect. In: Proceedings of the International Conference on Machine Learning (ICML), pp. 1058–1066 (2013)
Jarrett, K., Kavukcuoglu, K., Ranzato, M., LeCun, Y.: What is the best multi-stage architecture for object recognition? In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 2146–2153 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Li, Y., Li, H., Xu, Y., Wang, J., Zhang, Y. (2016). Very Deep Neural Network for Handwritten Digit Recognition. In: Yin, H., et al. Intelligent Data Engineering and Automated Learning – IDEAL 2016. IDEAL 2016. Lecture Notes in Computer Science(), vol 9937. Springer, Cham. https://doi.org/10.1007/978-3-319-46257-8_19
Download citation
DOI: https://doi.org/10.1007/978-3-319-46257-8_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-46256-1
Online ISBN: 978-3-319-46257-8
eBook Packages: Computer ScienceComputer Science (R0)