Abstract
This paper proposes an efficient hardware architecture for a function generator suitable for an artificial neural network (ANN). A spline-based approximation function is designed that provides a good trade-off between accuracy and silicon area, whilst also being inherently scalable and adaptable for numerous activation functions. This has been achieved by using a minimax polynomial and through optimal placement of the approximating polynomials based on the results of a genetic algorithm. The approximation error of the proposed method compares favourably to all related research in this field. Efficient hardware multiplication circuitry is used in the implementation, which reduces the area overhead and increases the throughput.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Widrow, B., Rumelhart, D.E., Lehr, M.A.: Neural Networks: Applications in Industry, Business and Science. Communications of the ACM 37, 93–105 (1994)
Cauwenberghs, G., Bayoumi, M.: Learning on Silicon - Adaptive VLSI Neural Systems. Kluwer Academic, Dordrecht (1999)
Zhang, D., Pal, S.K.: Neural Networks and Systolic Array Design. World Scientific, New Jersey (2002)
Ruckert, U.: ULSI Architectures for Artificial Neural Aetworks. IEEE Micro 22, 10–19 (2002)
Draghici, S.: On the Capabilities of Neural Networks using Limited Precision Weights. Neural Networks 15, 395–414 (2002)
Holt, J., Hwang, J.: Finite Precision Error Analysis of Neural Network Hardware Implementations. IEEE Transactions on Computers 42, 280–291 (1993)
Haykins, S.: Neural Networks - A Comprehensive Foundation. Prentice-Hall, Englewood Cliffs (1999)
Myers, D.J., Hutchinson, R.: Efficient Implementation of Piecewise Linear Activation Function for Digital VLSI Neural Networks. Electronics Letters 25, 1662–1663 (1989)
Alippi, C., Storti-Gajani, G.: Simple Approximation of Sigmoid Functions: Realistic Design of Digital VLSI Neural Networks. In: Proceedings of the IEEE Int’l Symp. Circuits and Systems, pp. 1505–1508 (1991)
Amin, H., Curtis, K., Hayes Gill, B.: Piecewise Linear Approximation applied to Non-Linear Function of a Neural Network. IEE Proceedings Circuits, Devices and Systems 144, 313–317 (1997)
Vassiliadis, S., Zhang, M., Delgado-Frias, J.: Elementary Function Generators for Neural-Network Emulators. IEEE Transactions on Neural Networks 11, 1438–1449 (2000)
Faiedh, H., Gafsi, Z., Besbes, K.: Digital Hardware Implementation of Sigmoid Function and its Derivative for Artificial Neural Networks. In: Proceedings of the 13th International Conference on Microelectronics, pp. 189–192. Rabat, Morocco (2001)
Basterretxea, K., Tarela, J., del Campo, I.: Approximation of Sigmoid Function and the Derivative for Hardware Implementation of Artificial Neurons. IEE Proceedings of Circuits, Devices and Systems 151, 18–24 (2004)
Tommiska, M.: Efficient Digital Implementation of the Sigmoid Function for Reprogrammable Logic. IEE Proceedings Computers and Digital Techniques 150, 403–411 (2003)
Pineiro, J., Oberman, S., Muller, J., Bruguera, J.: High-Speed Function Approximation Using a Minimax Quadratic Interpolator. IEEE Transactions on Computers 54, 304–318 (2005)
The MathWorks: MATLAB and Simulink for Technical Computing (2006), http://www.mathworks.com/
Koren, I.: Computer Arithmetic Algorithms, 2nd edn. A K Peters Ltd, Wellesley (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Larkin, D., Kinane, A., Muresan, V., O’Connor, N. (2006). An Efficient Hardware Architecture for a Neural Network Activation Function Generator. In: Wang, J., Yi, Z., Zurada, J.M., Lu, BL., Yin, H. (eds) Advances in Neural Networks - ISNN 2006. ISNN 2006. Lecture Notes in Computer Science, vol 3973. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11760191_192
Download citation
DOI: https://doi.org/10.1007/11760191_192
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-34482-7
Online ISBN: 978-3-540-34483-4
eBook Packages: Computer ScienceComputer Science (R0)