Alexander Woodward
ABSTRACT
The Image Receptive Fields Neural Networks (IRFNN) algorithm is a recent approach for image classification that is as accurate and an order of magnitude faster than using a traditional feed-forward neural network (multi-layer perceptron), with a linear input layer, non-linear hidden layer and linear output layer. This paper investigates the algorithm's optimal parameter configuration along with a GPU implementation, further extending the performance of the algorithm. Optimization of classification is achieved through a deep search of potential configurations with respect to the number of neurons in the hidden layer and receptive field placement within the image plane. Second stage refinement is achieved through a search for optimal Gaussian receptive field size and shape in 2D. These processes guarantee an optimal network configuration. Secondly, a GPU acceleration of the feed-forward processing of images into the network is implemented. Receptive fields are uploaded to the GPU and all computations take place on the GPU resulting in a large performance increase. Analysis of both improvements are described in the paper.
- AT&T Laboratories Cambridge. Our Database of Faces (formerly 'The ORL Database of Faces'), 2011. http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html, visited on October 9, 2011.Google Scholar
- P. Daum, J.-L. Buessler, and J.-P. Urban. Image receptive fields neural networks for object recognition. In T. Honkela, W. Duch, M. A. Girolami, and S. Kaski, editors, ICANN (2), volume 6792 of Lecture Notes in Computer Science, pages 95--102. Springer, 2011. Google ScholarDigital Library
- Eric Antonelo, the site administrator. Reservoir Computing: Shaping Dynamics into Information, 2011. http://reservoir-computing.org, visited on October 12, 2011.Google Scholar
- A. Ghani, T. M. McGinnity, L. Maguire, L. McDaid, and A. Belatreche. Neuro-Inspired Speech Recognition Based on Reservoir Computing, Advances in Speech Recognition, Noam Shabtai (Ed.). InTech, 2010.Google Scholar
- B. J. Grzyb, E. Chinellato, G. M. Wojcik, and W. A. Kaminski. Facial expression recognition based on liquid state machines built of alternative neuron models. IEEE International Joint Conference on Neural Networks, (1): 1011--1017, 2009. Google ScholarDigital Library
- H. Jaeger. The "echo state" approach to analysing and training recurrent neural networks. Technical Report 148, GMD - German National Research Institute for Computer Science, 2001.Google Scholar
- H. Jaeger. Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the "echo state network" approach. Technical report, Fraunhofer Institute AIS, St. Augustin-Germany, 2002.Google Scholar
- Y. LeCun and Y. Bengio. Convolutional networks for images, speech, and time-series. In M. A. Arbib, editor, The Handbook of Brain Theory and Neural Networks. MIT Press, 1995. Google ScholarDigital Library
- W. Maass, T. Natschlaeger, and H. Markram. Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation, 14(11): 2531--2560, 2002. Google ScholarDigital Library
- S. R. Madane, W. Banu, P. S., and S. C. R. Madane. BImplementation of High Speed Face Recognition Based on Karhunen Loeve Transform and FisherÕs Discriminant, Radial Basis Function of Echo State Neural Network. International Journal of Soft Computing, 3(3): 248--253, 2008.Google Scholar
- E. Niv and S. J. Weddell. A comparative study of random hidden node networks for pattern recognition. In Proceedings of the Image and Vision Computing New Zealand Conference (IVCNZ), pages 311--314, New Zealand, 2011.Google Scholar
- S. Scherer, M. Oubbati, F. Schwenker, and G. Palm. Real-time emotion recognition from speech using echo state networks. Artificial Neural Networks in Pattern Recognition, 5064: 205--216, 2008. Google ScholarDigital Library
- C. Y. Tsai, X. Dutoit, K. T. Song, H. V. Brussel, and M. Nuttin. Robust face tracking control of a mobile robot using self-tuning Kalman filter and echo state network. Asian Journal of Control, 12(4): 488--509, 2010.Google Scholar
- A. Woodward and T. Ikegami. A reservoir computing approach to image classification using coupled echo state and back-propagation neural networks. In Proceedings of the Image and Vision Computing New Zealand Conference (IVCNZ), pages 453--458, New Zealand, 2011.Google Scholar
Index Terms
- An optimal parameter analysis and GPU acceleration of the image receptive fields neural network approach
Recommendations
Multifold Acceleration of Neural Network Computations Using GPU
ICANN '09: Proceedings of the 19th International Conference on Artificial Neural Networks: Part IWith emergence of graphics processing units (GPU) of the latest generation, it became possible to undertake neural network based computations using GPU on serially produced video display adapters. In this study, NVIDIA CUDA technology has been used to ...
Neural acceleration for GPU throughput processors
MICRO-48: Proceedings of the 48th International Symposium on MicroarchitectureGraphics Processing Units (GPUs) can accelerate diverse classes of applications, such as recognition, gaming, data analytics, weather prediction, and multimedia. Many of these applications are amenable to approximate execution. This application ...
Brain-inspired hierarchical spiking neural network using unsupervised STDP rule for image classification
ICMLC '18: Proceedings of the 2018 10th International Conference on Machine Learning and ComputingThe human brain has powerful unsupervised learning ability without needs of large amounts of labeled samples, so brain-inspired neural networks have gained great attention recently. In this paper, a hierarchical spiking neural network (HSNN) is proposed ...
Comments