Abstract
This paper presents a comparative analysis of several popular and freely available deep learning frameworks. We compare functionality and usability of the frameworks trying to solve popular computer vision problems like hand-written digit recognition. Four libraries have been chosen for the detailed study: Caffe, Pylearn2, Torch, and Theano. We give a brief description of these libraries, consider key features and capabilities, and provide case studies. We also investigate the performance of the libraries. This study allows making a decision which deep learning framework suites us best and will be used for our future research.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Kustikova, V.D., Druzhkov, P.N.: A survey of deep learning methods and software for image classification and object detection. In: Proceedings of the 9th Open German-Russian Workshop on Pattern Recognition and Image Understanding (2014)
Torch. http://www.torch.ch
Hinton, G.E.: Learning multiple layers of representation. Trends Cogn. Sci. 11, 428–434 (2007)
LeCun, Y., Kavukcuoglu, K., Farabet, C.: Convolutional networks and applications in vision. In: Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS), pp. 253–256 (2010)
Hayat, M., Bennamoun, M., An, S.: Learning non-linear reconstruction models for image set classification. In: Proceedings of the IEEE Conference on CVPR (2014)
Restricted Boltzmann Machines (RBMs). http://www.deeplearning.net/tutorial/rbm.html
DeapLearnToolbox. https://github.com/rasmusbergpalm/DeepLearnToolbox
Deepnet Library. https://github.com/nitishsrivastava/deepnet
DeepMat Library. https://github.com/kyunghyuncho/deepmat
Package Darch. http://cran.r-project.org/web/packages/darch/index.html
nnForge Library. http://milakov.github.io/nnForge
Cuda-convnet - high-performance C++/CUDA implementation of convolutional neural networks. http://code.google.com/p/cuda-convnet
Cuda CNN Library. http://www.mathworks.com/matlabcentral/fileexchange/24291-cnn-convolutional-neural-network-class
Caffes repository. https://github.com/BVLC/caffe
Unofficial version of library Caffe to the Windows. https://github.com/niuzhiheng/caffe
NVIDIA(R) cuDNN - GPU Accelerated Machine Learning. https://developer.nvidia.com/cuDNN
Bottou, L.: Stochastic gradient descent tricks. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) NN: Tricks of the Trade. LNCS, vol. 7700, 2nd edn, pp. 421–436. Springer, Heidelberg (2012). http://research.microsoft.com/pubs/192769/tricks-2012.pdf
Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. ML Res. 12, 2121–2159 (2011)
Sutskever, I., Martens, J., Dahl, G., Hinton, G.: On the importance of initialization and momentum in deep learning. In: Proceedings of the 30th International Conference on ML (2013)
The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist
Pylearn2s repository. https://github.com/lisa-lab/pylearn2
Torch cheatsheet. https://github.com/torch/torch7/wiki/Cheatsheet
Example of logistic regression (Theano). https://github.com/ITLab-Vision/DNN-develop/blob/master/theano/src/mnist/logistic_sgd.py
Ciresan, D., Meier, U., Schmidhuber J.: Multi-column Deep Neural Networks for Image Classification. http://arxiv.org/pdf/1202.2745v1.pdf
Acknowledgments
The work has been performed in Information Technologies Laboratory at Computational Mathematics and Cybernetics Department, Lobachevsky State University of Nizhni Novgorod under support by Itseez Co. and Argus Center for Computer Vision Co. Ltd.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Kruchinin, D., Dolotov, E., Kornyakov, K., Kustikova, V., Druzhkov, P. (2015). Comparison of Deep Learning Libraries on the Problem of Handwritten Digit Classification. In: Khachay, M., Konstantinova, N., Panchenko, A., Ignatov, D., Labunets, V. (eds) Analysis of Images, Social Networks and Texts. AIST 2015. Communications in Computer and Information Science, vol 542. Springer, Cham. https://doi.org/10.1007/978-3-319-26123-2_38
Download citation
DOI: https://doi.org/10.1007/978-3-319-26123-2_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26122-5
Online ISBN: 978-3-319-26123-2
eBook Packages: Computer ScienceComputer Science (R0)