Abstract
The hyperbolic tangent (tanh) is a traditional choice for the activation function of the neurons of an artificial neural network. Here, we go through a simple calculation that shows that this modeling choice is linked to Bayesian decision theory. Our brief, tutorial-like discussion is intended as a reference to an observation rarely mentioned in standard textbooks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1996)
Haykin, S.: Neural Networks and Learning Machines: A Comprehensive Foundation. Prentice Hall, New York (2008)
MacKay, D.J.C.: Information Theory, Inference, and Learning Algorithms. Cambridge University Press, Cambridge (2003)
Ramamurthy, R., Stenzel, R., Sifa, R., Ladi, A., Bauckhage, C.: Echo state networks for named entity recognition. In: International Conference on Artificial Neural Networks. Springer (2019)
Rojas, R.: Neural Networks: A Systematic Introduction. Springer, Heidelberg (1996)
Russell, S.J., Norvig, P.: Artificial Intelligence: A Modern Approach. Prentice Hall, Upper Saddle River (1995)
Sifa, R., Bauckhage, C.: Archetypical motion: supervised game behavior learning with archetypal analysis. In: Proceedings of the IEEE CIG (2013)
Acknowledgments
In parts, the authors of this work were supported by the Fraunhofer Research Center for Machine Learning (RCML) within the Fraunhofer Cluster of Excellence Cognitive Internet Technologies (CCIT) and by the Competence Center for Machine Learning Rhine Ruhr (ML2R) which is funded by the Federal Ministry of Education and Research of Germany (grant no. 01|S18038A). We gratefully acknowledges this support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Bauckhage, C., Sifa, R., Hecker, D. (2021). tanh Neurons Are Bayesian Decision Makers. In: Arai, K., Kapoor, S., Bhatia, R. (eds) Intelligent Systems and Applications. IntelliSys 2020. Advances in Intelligent Systems and Computing, vol 1252. Springer, Cham. https://doi.org/10.1007/978-3-030-55190-2_56
Download citation
DOI: https://doi.org/10.1007/978-3-030-55190-2_56
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-55189-6
Online ISBN: 978-3-030-55190-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)