Skip to main content
SpringerLink
Log in

Low-error, High-speed Approximation of the Sigmoid Function for Large FPGA Implementations

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

There has been much study of ASIC neurocomputers but, in comparison, relatively little for FPGA neurocomputers. Nevertheless, with current (and future) dense, high-speed FPGAs, the latter are now viable and will be more successful than the former. In this paper, we discuss a technique for low-error, high-speed implementations of the sigmoid function in such FPGAs. This function is commonly used as an activation function in artificial neural networks, but it also has applications in many other areas. Our results compare very favourably with others that have been reported in the published literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. A. R. Omondi and J. Rajapakse, “Neural Networks in FPGAs”, in Proc. of the 9th International Conference on Neural Information Processing (ICONIP), Singapore, 18–22 November, 2002, pp. 954–959.

  2. A. R. Omondi, “Neurocomputers: A Dead End?”, Int. J. Neural. Syst., vol. 10, 2000, pp. 475–481.

    Google Scholar 

  3. M. Zhang, S. Vassiliadis, and J. G. Delgado-Frias, “Sigmoid Generators for Neural Computing Using Piecewise Approximations,” IEEE Trans. Comput., vol. 45, 1996, pp. 1045–1049.

    Article  MATH  Google Scholar 

  4. L. Fang and N. Nagarajan, “A back-up onboard orbit propagator using neural networks for small satellite missions”, in Proc. of the 23rd International Symposium on Space Technology and Science, Matsue, Japan, May 26–June 2, 2002.

  5. J. Wawrzynek et al., “SPERT II: A Vector Microprocessor System”, IEEE Computer, vol. 29, 1996, pp. 79–86.

    Google Scholar 

  6. J. L. Ayala, A. G. Lomeña, M. López-Vallejo, and A. Fernández, “Design of a pipelined hardware architecture for real-time neural network applications”, in Proc. of the IEEE Midwest Symposium on Circuits and Systems, Tulsa, USA, 2002.

  7. J. M. Muller, “Elementary Functions: Algorithms and Implementation”, Birkhauser, Boston, USA, 1997.

    MATH  Google Scholar 

  8. A. R. Omondi, “Computer Arithmetic Systems: Algorithms, Architecture, and Implementations”, Prentice-Hall, UK, 1994.

    MATH  Google Scholar 

  9. H. Amin, M. K. Curtis, and R. B. Hayes-Gill, “Piecewise Linear Approximation Applied to Non-linear Activation Functions of a Neural Network”, IEE Proc. Circ. Devices Syst., vol. 144, 1997, pp. 313–317.

    Article  Google Scholar 

  10. C. Alippi and G. Storti-Gajani, “Simple approximations of sigmoidal functions: realistic design of digital neural networks capable of learning”, in Proceedings IEEE Int. Symp. on Circuits and Systems, Singapore, 11–14 June, 1991, pp. 1505–1508.

  11. K. M. Sammut and S. R. Jones, “Implementing Non-linear Activation Functions in Neural Network Emulators”, Electron. Lett., vol. 27, 1991, pp. 1037–1038.

    Article  Google Scholar 

  12. K. Basterretxea, J. M. Tarela, and I. del Campo, “Approximation of Sigmoid Function for Hardware Implementation of Artificial Neurons”, IEE Proc. Circ. Devices Syst., vol. 151, 2004, pp. 18–24.

    Article  Google Scholar 

  13. D. J. Myers and R. A. Hutchison, “Efficient Implementation of Piecewise Linear Activation for Digital VLSI Neural Networks”, Electron. Lett., vol. 25, 1989, pp. 1662–1663.

    Article  Google Scholar 

  14. M. Al-Nsour and H. S. Abdel-Aty-Zahdy, “Implementation of programmable digital sigmoid function circuit for neuro-computing”, in Proc. of the IEEE Midwest Symposium on Circuits and Systems, 1998, pp. 571–574.

  15. Virtex-4 User Guide, Xilinx, 2004.

  16. O. Mencer and W. Luk, “Parameterized High Throughput Function Evaluation for FPGAs”, J. VLSI Signal Process., vol. 36, 2004, pp. 17–25.

    Article  Google Scholar 

  17. S. M. Pizer and V. L. Wallace, “To Compute Numerically: Concepts and Strategies”, Little, Brown, Boston, USA, 1983.

    Google Scholar 

  18. S. Vassiliadis, M. Zhang and J. G. Delgado-Frias, “Elementary Function Generators for Neural-Network Emulators”, IEEE Trans. Neural Netw., vol. 11, 2000, pp. 1438–1449.

    Article  Google Scholar 

  19. M. T. Tommiska, “Efficient Digital Implementation of Sigmoid Function for Reprogrammable Logic”, IEE Proc. Comput. Digit. Tech., vol. 150, 2003, pp. 403–411.

    Article  Google Scholar 

  20. H. K. Kwan, “Simple Sigmoid-Like Activation Function Suitable for Digital Hardware Implementation”, Electron. Lett., vol. 29, 1992, pp. 1379–1380.

    Article  Google Scholar 

  21. XtremeDSP Design Considerations: User Guide, Xilinx, 2004.

  22. M. Bajger and A.R. Omondi, “Implementations of the square-root and exponential functions for large FPGAs”, in Proc. of the 11th Asia-Pacific Computer Systems Architecture Conference, Shanghai, 6–8 September, 2006, LNCS 4186, pp. 6–23, Springer-Verlag.

  23. Maple 8 Programming Guide, Waterloo Maple, 2002.

  24. R. M. Corless, G. H. Gonnet, D. E. G. Hare, D. J. Jeffrey, and D. E. Knuth, “On the Lambert W Function”, Adv. Comput. Math., vol. 12, 1996, pp. 329–359.

    Article  MathSciNet  Google Scholar 

  25. M.J. Flynn and S.F. Oberman. “Advanced Computer Arithmetic Design”, Wiley, New York, 2001.

    Google Scholar 

  26. A. Peng and A.R. Omondi. “FPGA implementations of CORDIC algorithms”, in 11th International Conference on Signal Processing Applications and Technology, Dallas, Texas, 2000.

Download references

Author information

Authors and Affiliations

  1. School of Informatics and Engineering, Flinders University, Adelaide, Australia

    Mariusz Bajger

  2. School of Electrical and Electronic Engineering, Yonsei University, Seoul, South Korea

    Amos Omondi

Authors
  1. Mariusz Bajger
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amos Omondi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amos Omondi.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bajger, M., Omondi, A. Low-error, High-speed Approximation of the Sigmoid Function for Large FPGA Implementations. J Sign Process Syst Sign Image Video Technol 52, 137–151 (2008). https://doi.org/10.1007/s11265-007-0140-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-007-0140-z

Keywords

Search

Navigation