Abstract
We prove that an artificial neural network with multiple hidden layers and akth-order sigmoidal response function can be used to approximate any continuous function on any compact subset of a Euclidean space so as to achieve the Jackson rate of approximation. Moreover, if the function to be approximated has an analytic extension, then a nearly geometric rate of approximation can be achieved. We also discuss the problem of approximation of a compact subset of a Euclidean space with such networks with a classical sigmoidal response function.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
A.R. Barron, Universal approximation bounds for superposition of a sigmoidal function, Preprint (November 1990).
S.M. Caroll and S.M. Dickinson, Construction of neural nets using the radon transform, Preprint (1990).
T.P. Chen, H. Chen and R.W. Liu, A constructive proof of approximation by superposition of sigmoidal functions for neural networks, Preprint (1990).
C.K. Chui and X. Li, Approximation by ridge functions and neural networks with one hidden layer, CAT Report No. 222, Texas A&M University (1990).
C.K. Chui and X. Li, Realization of neural networks with one hidden layer, CAT Report No. 244, Texas A&M University (March 1991).
G. Cybenko, Approximation by superposition of sigmoidal functions, Math. Control, Signals and Systems 2(1989)303–314.
W. Dahmen and C.A. Micchelli, Some remarks on ridge functions, Approx. Theory Appl. 3(1987)139–143.
R. DeVore, R. Howard and C.A. Micchelli, Optimal nonlinear approximation, Manuscripta Mathematica 63(1989)469–478.
G. Freud and V.A. Popov, On approximation by spline functions,Proc. Conf. on Constructive Theory of Functions (Akadémiai Kiadó, 1972) pp. 163–173.
K.I. Funahashi, On the approximate realization of continuous mappings by neural networks, Neural Networks 2(1989)183–192.
R. Hecht-Nielsen,Neurocomputing (Addison-Wesley, New York, 1989).
K. Hornik, M. Stinchcombe and H. White, Multilayer feedforward networks are universal approximators, Neural Networks 2(1989)359–366.
B. Irie and S. Miyake, Capabilities of three layered perceptrons,IEEE Int. Conf. on Neural Networks, Vol. 1 (1988) pp. 641–648.
Leshno, Lin, Pinkus and Schocken, manuscript.
H.N. Mhaskar and C.A. Micchelli, Approximation by superposition of a sigmoidal function, Adv. Appl. Math. 13(1992)350–373.
T. Poggio and F. Girosi, Regularization algorithms for learning that are equivalent to multilayer networks, Science 247(1990)978–982.
V.A. Popov, Direct and converse theorems for spline approximation with free knots, Bulg. Math. Publ. 1(1975)218–224.
L.L. Schumaker,Spline Functions: Basic Theory (Wiley, New York, 1981).
J. Siciak, On some extremeal functions and their applications in the theory of analytic functions of several complex variables, Trans. Amer. Math. Soc. 105(1962)322–357.
M. Stinchcombe and H. White, Universal approximation using feedforward network with non-sigmoidal hidden layer activation functions, in:Proc. Int. Joint Conf. on Neural Networks, San Diego (SOS printing, 1989) pp. 613–618.
M. Stinchcombe and H. White, Approximating and learning unknown mappings using multilayer feedforward networks with bounded weights,IEEE Int. Conf. on Neural Networks, Vol. 3 (1990) pp. III-7–III-16.
A.F. Timan,Theory of Approximation of Functions of a Real Variable (Macmillan, New York, 1963).
Author information
Authors and Affiliations
Additional information
Dedicated to Dr. C.A. Micchelli on the occasion of his fiftieth birthday, December 1992
Research supported in part by AFOSR Grant No. 226 113 and by the AvH Foundation.
Rights and permissions
About this article
Cite this article
Mhaskar, H.N. Approximation properties of a multilayered feedforward artificial neural network. Adv Comput Math 1, 61–80 (1993). https://doi.org/10.1007/BF02070821
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02070821