Abstract
The main idea behind artificial neural networks is to simulate how data is processed in the data processing devoice that has been optimized by million-years natural selection–our brain. Such networks are indeed very successful, but interestingly, the most recent successes came when researchers replaces the original biology-motivated sigmoid activation function with a completely different one–known as rectified linear function. In this paper, we explain that this somewhat unexpected function actually naturally appears in physics-based data processing.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
C.M. Bishop, Pattern Recognition and Machine Learning (Springer, New York, 2006)
R. Feynman, R. Leighton, M. Sands, The Feynman Lectures on Physics (Addison Wesley, Boston, Massachusetts, 2005)
I. Goodfellow, Y. Bengio, A. Courville, Deep Learning (MIT Press, Cambridge, Massachusetts, 2016)
K.S. Thorne, R.D. Blandford, Modern Classical Physics: Optics, Fluids, Plasmas, Elasticity, Relativity, and Statistical Physics (Princeton University Press, Princeton, New Jersey, 2017)
Acknowledgements
This work was supported in part by the National Science Foundation grants 1623190 (A Model of Change for Preparing a New Generation for Professional Practice in Computer Science), and HRD-1834620 and HRD-2034030 (CAHSI Includes), and by the AT&T Fellowship in Information Technology.
It was also supported by the program of the development of the Scientific-Educational Mathematical Center of Volga Federal District No. 075-02-2020-1478, and by a grant from the Hungarian National Research, Development and Innovation Office (NRDI).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Contreras, J., Ceberio, M., Kreinovich, V. (2023). One More Physics-Based Explanation for Rectified Linear Neurons. In: Ceberio, M., Kreinovich, V. (eds) Uncertainty, Constraints, and Decision Making. Studies in Systems, Decision and Control, vol 484. Springer, Cham. https://doi.org/10.1007/978-3-031-36394-8_32
Download citation
DOI: https://doi.org/10.1007/978-3-031-36394-8_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36393-1
Online ISBN: 978-3-031-36394-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)