Oracles and Queries That Are Sufficient for Exact Learning

Abstract

We show that the class of all circuits is exactly learnable in randomized expected polynomial time using weak subset and weak superset queries. This is a consequence of the following result which we consider to be of independent interest: circuits are exactly learnable in randomized expected polynomial time with equivalence queries and the aid of anNP-oracle. We also show that circuits are exactly learnable in deterministic polynomial time with equivalence queries and a ∑^P₃-oracle. The hypothesis class for the above learning algorithms is the class of circuits of larger—but polynomially related—size. Also, the algorithms can be adapted to learn the class of DNF formulas with hypothesis class consisting of depth-3 ∧-∨-∧ formulas (by the work of Angluin this is optimal in the sense that the hypothesis class cannot be reduced to DNF formulas, i.e., depth-2 ∨-∧ formulas). We also investigate the power of anNP-oracle in the context of learning withmembershipqueries. We show that there are deterministic learning algorithms that use membership queries and anNP-oracle to learn: monotone boolean functions in time polynomial in the DNF size and CNF size of the target formula; and the class ofO(log n)-DNF∩O(log n)-CNF formulas in time polynomial inn. We also show that, with anNP-oracle and membership queries, there is a randomized expected polynomial time algorithm that learns any class that is learnable from membership queries with unlimited computational power. Using similar techniques, we show the following both for membership and for equivalence queries (when the hypotheses allowed are precisely the concepts in the class); any class learnable with unbounded ?computational-power is learnable in deterministic polynomial time with a ∑:^p₅-oracle. Furthermore, we identify the combinatorial properties that completely determine learnability in this information-theoretic sense. Finally we point out a consequence of our result in structural complexity theory showing that if everyNPset has polynomial-size circuits then the polynomial hierarchy collapses to ZPP^NP.