Function learning from interpolation (extended abstract)

Abstract

In the problem of learning a real-valued function from examples, in an extension of the ‘PAC’ model, a learner sees a sequence of values of an unknown function at a number of randomly chosen points. On the basis of these examples, the learner chooses a function—called a hypothesis—from some class H of hypotheses, with the aim that the learner's hypothesis is close to the target function on future random examples. In this paper we require that, for most training samples, with high probability the absolute difference between the values of the learner's hypothesis and the target function on a random point is small. A natural learning algorithm to consider is one that chooses a function in H that is close to the target function on the training examples. This, together with the success criterion described above, leads to the definition of a statistical property which we would wish a class of functions to possess.

We derive a characterization of function classes that have this property, in terms of their ‘fat-shattering function’, a notion that has proven useful in other problems in computational learning theory, such as the learnability of probabilistic concepts and the learnability of functions in the presence of random noise. This work has applications to the learning of functions in the presence of malicious noise.

This research was carried out while Martin Anthony was visiting the Department of Systems Engineering, ANU. This research is supported in part by the Australian Telecommunications and Electronics Research Board. The work of Martin Anthony is supported in part by the European Union through the “Neurocolt” ESPRIT Working Group.