Learning $AC^0$ Under k-Dependent Distributions

Abstract

It is well known that $AC^0$ circuits can be learned by the Low Degree Algorithm in quasi-polynomial-time under the uniform distribution due to Linial, Mansour and Nisan. Furst et al. and Blais et al. Then showed that this learnability also holds when the input variables are mutually independent or conform to some product distributions. However, a long-standing question is whether we can learn $AC^0$ beyond these distributions, e.g. under some non-product distributions.

In this paper we show $AC^0$ can be non-trivially learned under a sort of distributions, which we call k-dependent distributions. Informally, a k-dependent distribution is one satisfying that for a randomly sampled string (as input to a circuit being learned), some bits of it are mutually independent, of which each other bit is dependent on at most k ones. We note that this sort of distributions contains some natural non-product distributions. We show that with respect to any such distribution, if the dependence relations of all bits of sampled strings are known, $AC^0$ can be learned in quasi-polynomial-time in the case that k is poly-logarithmic, and otherwise, the learning costs exponential-time but still uses similarly many examples as the former case. We note that in the latter case although the time complexity is exponential, it is significantly smaller than that of the brute-force method (when the size of the circuit being learned is sufficiently large).