Notes on the PAC learning of geometric concepts with additional information

Abstract

In the PAC learning model, the learner must output good approximate results with only the information that tells the learner whether the example is “positive” or “negative”. This restriction results in narrowing the area of the learnable concepts or in increasing the size of examples required for successful learning. However, if the learner receives some additional information about the example besides being positive or negative, e.g., real values corresponding to the degree of the positiveness (or negativeness), a larger class may become learnable or the number of necessary examples may be reduced.

In the case of learning geometric concepts, such additional information may be given, for instance, how close a given positive or negative example is to the boundary of a target geometric concept. Using this type of additional information for geometric concepts consisting of complex boundaries, the learner may identify the geometric concept more rigorously or with sampling the smaller number of examples for the required accuracy. In the case of neural networks of threshold functions, some of values of weighted sum of inputs at nodes, say a value at the output node, may be output instead of simply producing outputs of either 0 or 1 after comparing the values with the threshold values at nodes. Such values represent the closeness of an input to the threshold boundary.

This note investigates the effect of such additional information in computational learning theory. Two types of networks of threshold-like functions are considered, and it is demonstrated how some additional information is profitable to learn these networks more efficiently.