Skip to main content
Log in

On the statistical physics of radial basis function networks

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Techniques from statistical physics have been applied successfully in recent years to the analysis of the generalization performance of neural networks. However, most of the analysis to date has been for perceptron-like networks or simple generalizations thereof such as committee machines, and none of the networks studied are used to any significant extent in practice. This letter presents results obtained in applying techniques from statistical physics to a popular class of neural networks that has been used successfully in many practical applications: the Gaussian radial basis function networks. We obtain expressions for the learning curves exhibited by these networks in the high-temperature limit for both realizable and unrealizable target rules.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. H. S. Seung, H. Sompolinsky, N. Tishby. Statistical mechanics of learning from examples,Physical Review A, vol. 45, no. 8, pp. 6056–6091, April 1992.

    Google Scholar 

  2. T. L. H. Watkin, A. Rau, M. Biehl. The statistical mechanics of learning a rule,Rev. Mod. Phys., vol. 65, no. 2, pp. 499–556, April 1993.

    Google Scholar 

  3. H. Schwarze, M. Opper, W. Kinzel. Generalization in a two-layer neural network,Physical Review A, vol. 46, no. 10, pp. R6185-R6188, 1992.

    Google Scholar 

  4. E. Barkai, D. Hansel, H. Sompolinsky. Broken symmetries in multilayered perceptrons,Physical Review A, vol. 45, no. 6, pp. 4146–4161, March 1992.

    Google Scholar 

  5. D. S. Broomhead, David Lowe. Multivariable functional interpolation and adaptive networks,Complex Systems, vol. 2, pp. 321–355, 1988.

    Google Scholar 

  6. M. J. D. Powell. The theory of radial basis function approximation in 1990,Technical report #DAMTP 1990/NA11, Department of Applied Mathematics and Theoretical Physics, University of Cambridge, December 1990.

  7. T. Poggio, F. Girosi. Networks for approximation and learning,Proceedings of the IEEE, vol. 78, no. 9, pp. 1481–1497, September 1990.

    Google Scholar 

  8. S. Renals, R. Rohwer. Phoneme classification experiments using radial basis functions,Proceedings of the International Joint Conference on Neural Networks, pp. I-461–I-467, June 1989.

  9. M. Niranjan, F. Fallside. Neural networks and radial basis functions in classifying static speech patterns,Computer Speech and Language, vol. 4, no. 3, pp. 275–289, July 1990.

    Google Scholar 

  10. S. B. Holden, P. J. W. Rayner, Generalization and PAC learning: Some new results for the class of generalized single layer networks,IEEE Transactions on Neural Networks, vol. 6, no. 2, pp. 368–380, March 1995.

    Google Scholar 

  11. M. Anthony, S. B. Holden. Quantifying generalization in linearly weighted neural networks,Complex Systems, vol. 8, pp. 91–114, 1994.

    Google Scholar 

  12. E. Levin, N. Tishby, S. A. Solla. A statistical approach to learning and generalization in layered neural networks,Proceedings of the IEEE, vol. 78, no. 10, pp. 1568–1574, October 1990.

    Google Scholar 

  13. S. B. Holden, M. Niranjan. Average-case learning curves for radial basis function networks,Technical Report CUED/F-INFENG/TR.212, Cambridge University Engineering Department, Trumpington Street, Cambridge CB2 1PZ, 1995.

    Google Scholar 

  14. S. Godsill.The restoration of degraded audio signals, Ph.D. thesis, Cambridge University Engineering Department, Trumpington Street, Cambridge CB2 1PZ, U.K., 1993.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Holden, S.B., Niranjan, M. On the statistical physics of radial basis function networks. Neural Process Lett 2, 16–19 (1995). https://doi.org/10.1007/BF02279933

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02279933

Keywords

Navigation