Abstract
The most difficult part of an artificial neural network to implement in hardware is the nonlinear activation function. For most implementations, the function used is the hyperbolic tangent. This function has received much attention in relation to hardware implementation. Nevertheless, there is no consensus regarding the best solution. In this paper, we propose a new approach by implementing the hyperbolic tangent in hardware with a polynomial modeling of the fractional exponential part. The results in the paper then demonstrate, through the use of an example, that this solution is faster than the CORDIC algorithm, but slower than the piecewise linear solution with the same error. The advantage over the piecewise linear approach is that it uses less memory.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Stieglitz T, Meyer J (2006) Biomedical microdevices for neural implants, vol 16. BIOMEMS Microsystems, Springer, pp 71–137
Frounchi J, Karimian G, Keshtkar A (2009) An artificial neural network hardware for bladder cancer. Eur J Sci Res 27(1):46–55
Lindsey C, Lindblad T (1995) Survey of neural network hardware. In: Rogers SK, Ruck DW (eds) Applications and science of artificial neural networks, SPIE 2492, pp 1194–1205
Liao Y (2012) Neural networks in hardware: a survey. Available in the internet at http://wwwcsif.cs.ucdavis.edu/~liaoy/research/NNhardware.pdf. Accessed Feb 2012
Hassibi K (2000) Detecting payment card fraud with neural networks. In: Lisboa P, Edisbury B, Vellido A (eds) Business applications of neural networks, progress in neural processing 13. World Scientific, Singapore
Dias FM, Antunes A, Mota AM (2004) Artificial neural networks: a review of commercial hardware. Eng Appl Artif Intell 17:945–952
Misra J, Saha I (2010) Artificial neural networks in hardware: a survey of two decades of progress. Neurocomputing 74:239–255
Ferreira P, Ribeiro P, Antunes A, Dias FM (2007) A high bit resolution FPGA implementation of a FNN with a new algorithm for the activation function. Neurocomputing 71(1–3):71–77
Ferreira P, Ribeiro P, Antunes A, Dias FM (2004) Artificial neural networks processor—a hardware implementation using a FPGA. Field-programmable logic and its applications, Belgium, LNCS-3203
Leon M, Castro A, Ascenccio R (1999) An artificial neural network on a field programmable gate array as a virtual sensor. In: Proceedings of the third international workshop on design of mixed-mode integrated circuits and applications, Puerto Vallarta, Mexico, pp 114–117
Ayala JL, Lomena AG, López-Vallejo M, Fernández A (2002) Design of a pipelined hardware architecture for real-time neural network computations. IEEE midwest symposium on circuits and systems, USA
Soares AM, Pinto JOP, Bose BK, Leite LC, da Silva LEB, Romero ME (2006) Field programmable gate array (FPGA) based neural network implementation of stator flux oriented vector control of induction motor drive. IEEE international conference on industrial technology
Chen X, Wang G, Zhou W, Chang S, Sun S (2006) Efficient sigmoid function for neural networks based FPGA design. ICIC 2006, LNCS 4113. Springer, Berlin, Heidelberg, pp 672–677
Ghariani M, Kharrat MW, Masmoudin N, Kamoun L (2004) Electronic implementation of a neural observer in FPGA technology: application to the control of electric vehicle. 16th international conference on microelectronics
Qian M (2006) Application of CORDIC algorithm to neural networks VLSI design. IMACS multiconference on computational engineering in systems applications
Duren RW, Marks RJ II, Reynolds PD, Trumbo ML (2007) Real-time neural network inversion on the SRC-6e reconfigurable computer. IEEE Trans Neural Networks 18(3):889–901
Andraka R (1998) A survey of CORDIC algorithms for FPGA based computers. In: Proceedings of the ACM/SIGDA 6th international symposium on FPGA, Monterey, CA, USA, pp 191–200
Soria-Olivas E, Martín-Guerrero JD, Camps-Valls G, Serrano-López AJ, Calpe-Maravilla J, Gómez-Chova L (2003) A low-complexity fuzzy activation function for artificial neural networks. IEEE Trans Neural Networks 14(6):1576–1579
Rosado-Muñoz A, Soria-Olivas E, Gomez-Chova L, Vila Francés J (2008) An IP core and GUI for implementing multilayer perceptron with a fuzzy activation function on configurable logic devices. J Univers Comput Sci 14(10):1678–1694
Kwan HK (1992) Simple sigmoid-like activation function suitable for digital hardware implementation. Electron Lett 28(15):1379–1380
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Nascimento, I., Jardim, R. & Morgado-Dias, F. A new solution to the hyperbolic tangent implementation in hardware: polynomial modeling of the fractional exponential part. Neural Comput & Applic 23, 363–369 (2013). https://doi.org/10.1007/s00521-012-0919-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-012-0919-0