Skip to main content
SpringerLink
Log in

GPU-enabled back-propagation artificial neural network for digit recognition in parallel

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

In this paper, we show that the GPU (graphics processing unit) can be used not only for processing graphics, but also for high speed computing. We provide a comparison between the times taken on the CPU and GPU to perform the training and testing of a back-propagation artificial neural network. We implemented two neural networks for recognizing handwritten digits; one consists of serial code executed on the CPU, while the other is a GPU-based version of the same system which executes in parallel. As an experiment for performance evaluation, a system for neural network training on the GPU is developed to reduce training time. The programming environment that the system is based on is CUDA which stands for compute unified device architecture, which allows a programmer to write code that will run on an NVIDIA GPU card. Our results over an experiment of digital image recognition using neural network confirm the speed-up advantages by tapping on the resources of GPU. Our proposed model has an advantage of simplicity, while it shows on par performance with the state-of-the-arts algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Park SI, Ponce SP, Huang J, Cao Y, Quek F (2008) Low-cost, high-speed computer vision using NVIDIA’s CUDA architecture. In: 37th IEEE applied imagery pattern recognition workshop, pp 1–7

  2. NVidia CUDA Zone. http://www.nvidia.com/object/cuda_home.html

  3. Steinkrau D, Simard PY, Buck I (2013) Using GPUs for machine learning algorithms. In: 12th International conference on document analysis and recognition, pp 1115–1119

  4. Lopez-Fandino J, Heras DB, Arguello F (2014) Efficient classification of hyperspectral images on commodity GPUs using ELM-based techniques. In: Conference PDPTA’14, CSREA Press, July 21–24, pp 1–13

  5. Catanzaro B, Sundaram N, Keutzer K (2008) Fast support vector machine training and classification on graphics processors. In: Proceedings of the 25th international conference on machine learning (ICML 2008), Helsinki, Finland, pp 104–111

  6. van Heeswijk M, Miche Y, Lindh-Knuutila T, Hilbers P, Honkela T, Oja E, Lendasse A (2009) Adaptive ensemble models of extreme learning machines for time series prediction. In: 19th International conference on artificial neural networks, Limassol, Cyprus, 9

  7. Neural Networks on the GPU. http://leenissen.dk/fann/html_latest/files2/gpu-txt.html

  8. Neural Networks with Parallel and GPU Computing. http://www.mathworks.com/help/nnet/ug/neural-networks-with-parallel-and-gpu-computing.html

  9. A Neural Network on GPU. http://www.codeproject.com/Articles/24361/A-Neural-Network-on-GPU

  10. Huang G-B, Chen L, Siew C-K (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE TNN 17(4):879–892

    Google Scholar 

  11. Hayashi A, Ishizaki K, Koblents G, Sarkar V (2015) Machine-learning-based performance heuristics for runtime CPU/GPU selection. In: Proceedings of the principles and practices of programming on the Java platform, pp 27–36

  12. Ribeiro B, Goncalves J (2012) Restricted Boltzmann machines and deep belief networks on multi-core processors. In: The 2012 international joint conference on neural networks (IJCNN), 10–15 June 2012, pp 1–7

  13. Huqqani AA, Schikuta E, Ye S, Chen P (2013) Multicore and GPU parallelization of neural networks for face recognition. In: International conference on computational science, ICCS 2013, Procedia Computer Science, vol 18, pp 349–358

  14. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  15. Cambria et al (2013) Extreme learning machines. IEEE Trans Cybern 28(6):30–59

Download references

Acknowledgments

The authors are thankful for the financial support from the research grant “Peer-production approaches to e-Learning (PPAeL),” Grant No. FDCT 019/2011/A1, offered by Macau Fundo para o Desenvolvimento das Ciências e da Tecnologia.

Author information

Authors and Affiliations

  1. Department of Computer and Information Science, University of Macau, Macau SAR, China

    Ricardo Brito & Simon Fong

  2. Department of Computer and Multimedia Engineering, Dongguk University, Seoul, Korea

    Kyungeun Cho

  3. College of Information Engineering, North China University of Technology, Beijing, China

    Wei Song

  4. School of Computer Science and Engineering, University of New South Wales, Sydney, Australia

    Raymond Wong

  5. Department of Computer Science, Lakehead University, Thunder Bay, Canada

    Sabah Mohammed & Jinan Fiaidhi

Authors
  1. Ricardo Brito
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simon Fong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kyungeun Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Raymond Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sabah Mohammed
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jinan Fiaidhi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simon Fong.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Brito, R., Fong, S., Cho, K. et al. GPU-enabled back-propagation artificial neural network for digit recognition in parallel. J Supercomput 72, 3868–3886 (2016). https://doi.org/10.1007/s11227-016-1633-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-016-1633-y

Keywords

Search

Navigation