Learnability with restricted focus of attention guarantees noise-tolerance

Abstract

We consider the question of learning in the presence of classification noise. More specifically, we address the issue of identifying conditions that, once a learning algorithm meets them, it can be transformed into a noise-tolerant algorithm.

While the question of whether every PAC learning algorithm can be made noise-tolerant is still open, the bottom line of this work, loosely stated, is that any restriction on the amount of data an algorithm is allowed to retrieve from its input samples, suffices to render the existence of a noise-tolerant variant that is efficient whenever the original algorithm is. The result is obtained by proving that such a restricted learning is equivalent to learning from statistical queries, and by applying Kearns transformation from statistical learning into noise-tolerant learning.