Skip to main content
SpringerLink
Log in

FPGA acceleration on a multi-layer perceptron neural network for digit recognition

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

This paper proposes field-programmable gate array (FPGA) acceleration on a scalable multi-layer perceptron (MLP) neural network for classifying handwritten digits. First, an investigation to the network architectures is conducted to find the optimal FPGA design corresponding to different classification rates. As a case study, then a specific single-hidden-layer MLP network is implemented with an eight-stage pipelined structure on Xilinx Ultrascale FPGA. It mainly contains a timing controller designed by Verilog Hardware Description Language (HDL) and sigmoid neurons integrated by Xilinx IPs. Finally, experimental results show a greater than \(\times 10\) speedup compared with prior implementations. The proposed FPGA architecture is expandable to other specifications on different accuracy (up to 95.82%) and hardware cost.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Benidis K, Rangapuram S, Flunkert V (2020) Neural forecasting: introduction and literature overview. arXiv:2004.10240

  2. Ismayilov G, Topcuoglu HR (2020) Neural network based multi-objective evolutionary algorithm for dynamic workflow scheduling in cloud computing. Future Gen Comput Syst 102:307–322

    Article  Google Scholar 

  3. Molchanov P, Mallya A, Tyree S, Frosio I, Kautz J (2019) Importance estimation for neural network pruning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 11264–11272

  4. Son C, Park S, Lee J, Paik J (2019) Deep learning-based number detection and recognition for gas meter reading. IEIE Trans Smart Process Comput 8(5):367–372

    Article  Google Scholar 

  5. Shawahna A, Sait SM, El-Maleh A (2019) FPGA-based accelerators of deep learning networks for learning and classification: a review. IEEE Access 7:7823–7859

    Article  Google Scholar 

  6. Nurvitadhi E, Kwon D, Jafari A et al (2019) Evaluating and enhancing Intel Stratix 10 FPGAs for persistent real-time AI. In: Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, p 119

  7. Guo K, Zeng S, Yu J, Wang Y, Yang H (2017) A survey of FPGA based neural network accelerator. arXiv:1712.08934

  8. Albert R et al (2019) Survey and benchmarking of machine learning accelerators. arXiv:1908.11348v1

  9. Gschwend D (2020) ZynqNet: an FPGA-accelerated embedded convolutional neural network. arXiv:2005.06892

  10. Gao C, Braun S, Kiselev I, Anumula J, Delbruck T, Liu S (2019) Real-time speech recognition for IoT purpose using a delta recurrent neural network accelerator. In: 2019 IEEE International Symposium on Circuits and Systems (ISCAS), pp 1–5

  11. Vaca K, Gajjar A, Yang X (2019) Real-time automatic music transcription (AMT) with Zync FPGA. In: IEEE Computer Society Annual Symposium on VLSI (ISVLSI), pp 378–384

  12. Li Q, Zhang X, Xiong J, Hwu W, Chen D (2019) Implementing neural machine translation with bi-directional GRU and attention mechanism on FPGAs using HLS. In: Proceedings of the 24th Asia and South Pacific Design Automation Conference, pp 693–698

  13. Rongshi D, Yongming T (2019) Accelerator implementation of Lenet-5 convolution neural network based on FPGA with HLS. 2019 3rd International Conference on Circuits, System and Simulation (ICCSS). Nanjing, China, pp 64–67

    Chapter  Google Scholar 

  14. Zhang Q, Cao J, Zhang Y, Zhang S, Zhang Q, Yu D (2019) FPGA implementation of quantized convolutional neural networks. In: 2019 IEEE 19th International Conference on Communication Technology (ICCT), pp 1605–1610

  15. Cong J, Liu B, Neuendorffer S et al (2011) High-level synthesis for FPGAs: from prototyping to deployment. IEEE Trans Comput Aided Des Integr Circuits Syst 30(4):473–491

    Article  Google Scholar 

  16. Akgun OC, Mei J (2020) An energy efficient time-mode digit classification neural network implementation. Philos Trans R Soc A 37820190163

  17. Xiang Y et al (2019) Hardware implementation of energy efficient deep learning neural network based on nanoscale flash computing array. Adv Mater Technol 4(5):1800720

    Article  Google Scholar 

  18. Ma Y, Guo J, Wei W (2019) An exceedingly fast model for low resolution handwritten digit string recognition. In: IEEE 7th International Conference on Computer Science and Network Technology (ICCSNT), pp 282–288

  19. Abdulrazzaq MB, Saeed JN (2019) A comparison of three classification algorithms for handwritten digit recognition. In: International Conference on Advanced Science and Engineering (ICOASE), pp 58–63

  20. Ahlawat S, Choudhary A, Nayyar A, Singh S, Yoon B (2020) Improved handwritten digit recognition using convolutional neural networks (CNN). Sensors 20:3344

    Article  Google Scholar 

  21. Ali S, Shaukat Z, Azeem M et al (2019) An efficient and improved scheme for handwritten digit recognition based on convolutional neural network. SN Appl Sci 1:1125

    Article  Google Scholar 

  22. Cho M, Kim Y (2020) Implementation of data-optimized FPGA-based accelerator for convolutional neural network. In: International Conference on Electronics, Information, and Communication (ICEIC), pp 1–2

  23. Madadum H, Becerikli Y (2019) FPGA-based optimized convolutional neural network framework for handwritten digit recognition. In: 1st International Informatics and Software Engineering Conference (UBMYK), pp 1–6

  24. Tsai T-H, Ho Y-C, Sheu M-H (2019) Implementation of FPGA-based accelerator for deep neural networks. In: 2019 IEEE 22nd International Symposium on Design and Diagnostics of Electronic Circuits and Systems (DDECS)

  25. Xiao R, Shi J, Zhang C (2020) FPGA implementation of CNN for handwritten digit recognition. In: IEEE Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), pp 1128–1133

  26. Si J, Yfantis E, Harris SL (2019) A SS-CNN on an FPGA for handwritten digit recognition. In: 2019 IEEE 10th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON), pp 88–93

  27. Si J, Harris SL (2018) Handwritten digit recognition system on an FPGA. In: 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC), pp 402–407

  28. LeCun Y, Cortes C, Burges C (2010) MNIST handwritten digit database

  29. Nielsen M (2019) Neural Networks and Deep Learning. Determination Press, neuralnetworksanddeeplearning.com, Neural Networks and Deep Learning

  30. Zynq-7000 SoC Data Sheet: Overview. V1.11.1, Xilinx, July 2 (2018)

  31. Zynq-7000 SoC Technical Reference Manual. V1.12.2, Xilinx, July (2018)

Download references

Author information

Authors and Affiliations

  1. University of Houston Clear Lake, 2700 Bay Area Blvd, Houston, TX, 77058, USA

    Isaac Westby & Xiaokun Yang

  2. Lawrence Technological University, 21000 West Ten Mile Road, Southfield, MI, 48075, USA

    Tao Liu

  3. California State University Long Beach, 1250 Bellflower Blvd., Long Beach, CA, 90840, USA

    Hailu Xu

Authors
  1. Isaac Westby
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaokun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hailu Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaokun Yang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Westby, I., Yang, X., Liu, T. et al. FPGA acceleration on a multi-layer perceptron neural network for digit recognition. J Supercomput 77, 14356–14373 (2021). https://doi.org/10.1007/s11227-021-03849-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-021-03849-7

Keywords

Search

Navigation