Abstract
In this paper, we consider Kearns’ [4] Statistical Query Model of learning. It is well known [3] that the number of statistical queries, needed for “weakly learning” an unknown target concept (i.e. for gaining significant advantage over random guessing) is polynomially related to the so-called Statistical Query dimension of the concept class. In this paper, we provide a similar characterization for “strong learning” where the learners final hypothesis is required to approximate the unknown target concept up to a small rate of misclassification. The quantity that characterizes strong learnability in the Statistical Query model is a surprisingly close relative of (though not identical to) the Statistical Query dimension. For the purpose of proving the main result, we provide other characterizations of strong learnability which are given in terms of covering numbers and related notions. These results might find some interest in their own right. All characterizations are purely information-theoretical and ignore computational issues.
This work was supported in part by the IST Programme of the European Community, under the PASCAL Network of Excellence, IST-2002-506778. This publication only reflects the authors’ views. Part of this work was done during a visit of the Helsinki Institute of Information Technology.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Balcázar, J.L., Castro, J., Guijarro, D.: A new abstract combinatorial dimension for exact learning via queries. Journal of Computer and System Sciences 64(1), 2–21 (2002)
Ben-David, S., Itai, A., Kushilevitz, E.: Learning by distances. Information and Computation 117(2), 240–250 (1995)
Blum, A., et al.: Weakly learning DNF and characterizing statistical query learning using Fourier analysis. In: Proceedings of the 26th Annual Symposium on Theory of Computing, pp. 253–263 (1994)
Kearns, M.: Efficient noise-tolerant learning from statistical queries. Journal of the Association on Computing Machinery 45(6), 983–1006 (1998)
Köbler, J., Lindner, W.: A general dimension for approxmately learning boolean functions. In: Cesa-Bianchi, N., Numao, M., Reischuk, R. (eds.) ALT 2002. LNCS (LNAI), vol. 2533, pp. 139–148. Springer, Heidelberg (2002)
Simon, H.U.: Spectral norm in learning theory: some selected topics (Invited Talk) (Invited Talk). In: Balcázar, J.L., Long, P.M., Stephan, F. (eds.) ALT 2006. LNCS (LNAI), vol. 4264, pp. 13–27. Springer, Heidelberg (2006)
Valiant, L.G.: A theory of the learnable. Communications of the ACM 27(11), 1134–1142 (1984)
Yang, K.: New lower bounds for statistical query learning. Journal of Computer and System Sciences 70(4), 485–509 (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Simon, H.U. (2007). A Characterization of Strong Learnability in the Statistical Query Model. In: Thomas, W., Weil, P. (eds) STACS 2007. STACS 2007. Lecture Notes in Computer Science, vol 4393. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-70918-3_34
Download citation
DOI: https://doi.org/10.1007/978-3-540-70918-3_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-70917-6
Online ISBN: 978-3-540-70918-3
eBook Packages: Computer ScienceComputer Science (R0)