Skip to main content

Advertisement

SpringerLink
Log in

POLYBiNN: Binary Inference Engine for Neural Networks using Decision Trees

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

Convolutional Neural Networks (CNNs) and Deep Neural Networks (DNNs) have gained significant popularity in several classification and regression applications. The massive computation and memory requirements of DNN and CNN architectures pose particular challenges for their FPGA implementation. Moreover, programming FPGAs requires hardware-specific knowledge that many machine-learning researchers do not possess. To make the power and versatility of FPGAs available to a wider deep learning user community and to improve DNN design efficiency, we introduce POLYBiNN, an efficient FPGA-based inference engine for DNNs and CNNs. POLYBiNN is composed of a stack of decision trees, which are binary classifiers in nature, and it utilizes AND-OR gates instead of multipliers and accumulators. POLYBiNN is a memory-free inference engine that drastically cuts hardware costs. We also propose a tool for the automatic generation of a low-level hardware description of the trained POLYBiNN for a given application. We evaluate POLYBiNN and the tool for several datasets that are normally solved using fully connected layers. On the MNIST dataset, when implemented in a ZYNQ-7000 ZC706 FPGA, the system achieves a throughput of up to 100 million image classifications per second with 90 ns latency and 97.26% accuracy. Moreover, POLYBiNN consumes 8× less power than the best previously published implementations, and it does not require any memory access. We also show how POLYBiNN can be used instead of the fully connected layers of a CNN and apply this approach to the CIFAR-10 dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9

Similar content being viewed by others

References

  1. LeCun, Y., Bengio, Y., Hinton, G. (2015). Deep learning. Nature.

  2. Hunter, D., Yu, H., Pukish, M.S., Kolbusz, J, Wilamowski, B.M. (2012). Selection of proper neural network sizes and architectures—a comparative study. IEEE Transactions on Industrial Informatics.

  3. Nurvitadhi, E., Venkatesh, G., Sim, J., Marr, D., Huang, R., Hock, J., Liew, Y.T., Srivatsan, K., Moss, D., Subhaschandra, S., Boudoukh, G. (2017). Can FPGAs beat GPUs in accelerating next-generation deep neural networks? ACM/SIGDA International Symposium on Field-Programmable Gate Arrays.

  4. Misra, J., & Saha, I. (2010). Artificial neural networks in hardware: a survey of two decades of progress. Neurocomputing.

  5. Sze, V., Chen, Y.H., Yang, T.J., Emer, J. (2017). Efficient processing of deep neural networks: a tutorial and survey. Proceedings of the IEEE.

  6. Courbariaux, M., Hubara, I., Soudry, D., Yaniv, R.E., Bengio, Y. (2016). Binarized neural networks: training deep neural networks with weights and activations constrained to + 1 or -1. Computer Research Repository. arXiv:1602.02830.

  7. Rastegari, M., Ordonez, V., Redmon, J., Farhadi, A. (2016). Xnor-net: imagenet classification using binary convolutional neural networks. Springer European Conference on Computer Vision.

  8. Deng, L., Jiao, P., Pei, J., Wu, Z., Li, G. (2017). Gated XNOR networks: deep neural networks with ternary weights and activations under a unified discretization framework. arXiv:1705.09283.

  9. Hastie, T., Tibshirani, R., Friedman, J. (2008). The elements of statistical learning, 2nd edn. New York: Springer.

    MATH  Google Scholar 

  10. Lloyd, S. (1982). Least squares quantization in PCM. IEEE Transactions on Information Theory, 28, 129–137.

    Article  MathSciNet  Google Scholar 

  11. Cristianini, N., & Shawe-Taylor, J. (2000). An introduction to support vector machines and other Kernel-Based learning methods. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  12. Breiman, L., Friedman, J., Olshen, R., Stone, C. (1984). Classification and regression trees. Boca Raton: CRC Press.

    MATH  Google Scholar 

  13. Akers, S.B. (1978). Binary decision diagrams. IEEE Transactions on Computers.

  14. Tang, P.T. (1991). Table-lookup algorithms for elementary functions and their error analysis. IEEE Symposium on Computer Arithmetic.

  15. Abdelsalam, A.M., Elsheikh, A., David, J.P., Langlois, J.M.P. (2018). POLYBiNN: a scalable and efficient combinatorial inference engine for neural networks on FPGA. IEEE Design and Architectures for Signal and Image Processing.

  16. Cheng, Y., Wang, D., Zhou, P., Zhang, T. (2017). A survey of model compression and acceleration for deep neural networks. arXiv:1710.09282.

  17. Courbariaux, M., Bengio, Y., David, J.P. (2015). Binaryconnect: training deep neural networks with binary weights during propagations. In Advances in neural information processing systems.

  18. Umuroglu, Y., Fraser, N.J., Gambardella, G., Blott, M., Leong, P., Jahre, M., Vissers, K. (2017). FINN: A framework for fast, scalable binarized neural network inference. ACM/SIGDA International Symposium on Field-Programmable Gate Arrays.

  19. Nakahara, H., Fujii, T., Sato, S. (2017). A fully connected layer elimination for a binarized convolutional neural network on an FPGA. IEEE International Conference on Field Programmable Logic and Applications (FPL).

  20. Zhao, R., Song, W., Zhang, W., Xing, T., Lin, J.H., Srivastava, M., Gupta, R., Zhang, Z. (2017). Accelerating binarized convolutional neural networks with software-programmable FPGAs. ACM/SIGDA International Symposium on Field-Programmable Gate Arrays.

  21. Alemdar, H., Leroy, V., Prost-Boucle, A., Petrot, F. (2017). Ternary neural networks for resource-efficient AI applications. IEEE International Joint Conference on Neural Networks.

  22. Liang, S., Yin, S., Liu, L., Luk, W., Wei, S. (2018). FP-BNN binarized neural network on FPGA. Neurocomputing, 275, 1072–1086.

    Article  Google Scholar 

  23. Han, S., Mao, H., Dally, W.J. (2016). Deep compression. Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding. ICLR.

  24. Rokach, L., & Maimon, O.Z. (2008). Data mining with decision trees: theory and applications. World Scientific.

  25. Robert, C. (2014). Machine learning, A probabilistic perspective.

  26. Lorena, A.C., De Carvalho, A.C., Gama, J.M. (2008). A review on the combination of binary classifiers in multiclass problems artificial intelligence review.

  27. Struharik, J.R. (2011). Implementing decision trees in hardware. IEEE International Symposium on Intelligent Systems and Informatics.

  28. Furnkranz, J., Gamberger, D., Lavrac, N. (2012). Foundations of rule learning. Berlin: Springer.

    Book  Google Scholar 

  29. Duda, R.O., Hart, P., Stork, D.G. (2012). Pattern classification. New York: Wiley.

    MATH  Google Scholar 

  30. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P. (1998). Gradient-based learning applied to document recognition. In Proceedings of the IEEE.

  31. UCI Machine Learning Repository. http://archive.ics.uci.edu/ml/datasets/ISOLET.

  32. UCI Machine Learning Repository. https://archive.ics.uci.edu/ml/datasets/UJIIndoorLoc.

  33. UCI Machine Learning Repository. https://archive.ics.uci.edu/ml/datasets/Daily+and+Spor/discretionary-ts+Activities https://archive.ics.uci.edu/ml/datasets/Daily+and+Spor/discretionary-ts+Activities.

  34. Krizhevsky, A., & Hinton, G. (2009). Learning multiple layers of features from tiny images technical report.

  35. Abdelsalam, A.M., Boulet, F., Demers, G., Langlois, J.M.P., Cheriet, F. (2018). An efficient FPGA-based overlay inference architecture for fully connected DNNs. IEEE International Conference on Reconfigurable Computing.

  36. Ba, J., & Caruana, R. (2014). Do deep nets really need to be deep? Advances in Neural Information Processing Systems.

Download references

Acknowledgments

The authors would like to thank Safa Berrima, Imad Benacer, Jeferson Santiago da Silva, Thibaut Stimpfling and Thomas Luinaud for their insightful comments.

Author information

Authors and Affiliations

  1. Department of Computer and Software Engineering, Polytechnique Montréal, Montreal, Canada

    Ahmed M. Abdelsalam & J. M. Pierre Langlois

  2. Department of Mathematics and Industrial Engineering, Polytechnique Montréal, Montreal, Canada

    Ahmed Elsheikh

  3. Department of Electrical Engineering, Polytechnique Montréal, Montreal, Canada

    Sivakumar Chidambaram & Jean-Pierre David

Authors
  1. Ahmed M. Abdelsalam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ahmed Elsheikh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sivakumar Chidambaram
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jean-Pierre David
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. J. M. Pierre Langlois
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmed M. Abdelsalam.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abdelsalam, A.M., Elsheikh, A., Chidambaram, S. et al. POLYBiNN: Binary Inference Engine for Neural Networks using Decision Trees. J Sign Process Syst 92, 95–107 (2020). https://doi.org/10.1007/s11265-019-01453-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-019-01453-w

Keywords

Search

Navigation