Skip to main content

One More Physics-Based Explanation for Rectified Linear Neurons

  • Chapter
  • First Online:
Uncertainty, Constraints, and Decision Making

Part of the book series: Studies in Systems, Decision and Control ((SSDC,volume 484))

  • 256 Accesses

Abstract

The main idea behind artificial neural networks is to simulate how data is processed in the data processing devoice that has been optimized by million-years natural selection–our brain. Such networks are indeed very successful, but interestingly, the most recent successes came when researchers replaces the original biology-motivated sigmoid activation function with a completely different one–known as rectified linear function. In this paper, we explain that this somewhat unexpected function actually naturally appears in physics-based data processing.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. C.M. Bishop, Pattern Recognition and Machine Learning (Springer, New York, 2006)

    MATH  Google Scholar 

  2. R. Feynman, R. Leighton, M. Sands, The Feynman Lectures on Physics (Addison Wesley, Boston, Massachusetts, 2005)

    MATH  Google Scholar 

  3. I. Goodfellow, Y. Bengio, A. Courville, Deep Learning (MIT Press, Cambridge, Massachusetts, 2016)

    MATH  Google Scholar 

  4. K.S. Thorne, R.D. Blandford, Modern Classical Physics: Optics, Fluids, Plasmas, Elasticity, Relativity, and Statistical Physics (Princeton University Press, Princeton, New Jersey, 2017)

    MATH  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Science Foundation grants 1623190 (A Model of Change for Preparing a New Generation for Professional Practice in Computer Science), and HRD-1834620 and HRD-2034030 (CAHSI Includes), and by the AT&T Fellowship in Information Technology.

It was also supported by the program of the development of the Scientific-Educational Mathematical Center of Volga Federal District No. 075-02-2020-1478, and by a grant from the Hungarian National Research, Development and Innovation Office (NRDI).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Vladik Kreinovich .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Contreras, J., Ceberio, M., Kreinovich, V. (2023). One More Physics-Based Explanation for Rectified Linear Neurons. In: Ceberio, M., Kreinovich, V. (eds) Uncertainty, Constraints, and Decision Making. Studies in Systems, Decision and Control, vol 484. Springer, Cham. https://doi.org/10.1007/978-3-031-36394-8_32

Download citation

Publish with us

Policies and ethics