Abstract
PAC learning from examples is factored so that (i) the membership queries are used to evaluate empirically “statistical queries” — certain expectations of functionals involving the unknown target. (ii) approximate value of these statistical queries are used to compute an output — an approximation of the target.
Kearns' original formulation of statistical queries [we use the abbreviation SQ], is extended here to include as SQ functionals of arbitrary range and order higher than one — second order being the most useful addition. This enables us to capture more ground for casting efficient PAC learning algorithms in SQ form: The important Kushilevitz-Mansour Fourier - based algorithm has an SQ rendition, as well as its derivatives, e.g. Jackson's recent DNF learning.
Efficient evaluation of extended SQ by membership queries, if possible at all, becomes quite intricate. We show, however, that it is usually robust under classification noise.
Partially supported by the U.S-Israeli Binational Science Foundation, grant 90-00189
This is a preview of subscription content, log in via an institution.
Preview
Unable to display preview. Download preview PDF.
References
Javed A. Aslam and Scott E. Decatur, Simulating statistical queries in the presence of classification noise, unpublished manuscript, 1993.
Avrim Blum, Merrick Furst, Jeffrey Jackson, Michael Kearns, Yishay Mansour, Steven Rudich, Weakly learning DNF and characterizing statistical query learning using Fourier analysis. Proceedings of the 26th Annual ACM Symposium on Theory of Computation, 253–262, 1994.
Jeffrey Jackson, An Efficient Membership-Query Algorithm for Learning DNF with Respect to the Uniform Distribution, Proceedings of the 35th Annual IEEE Symposium on the Foundation of Computer Science, 42–53, 1994.
Michael J. Kearns, Efficient noise-tolerant learning form statistical queries. In Proceedings of the 25th Annual ACM Symposium on Theory of Computing pages 392–401, 1993.
Eyal Kushilevitz and Yishay Mansour, Learning decision trees using the Fourier transform. SIAM Journal con Computing, 22(6) 1331–1348, 1993.
Thimothy L. H. Watkins, Albrecht Rau and Michael Biehl, The Statistical Mechanics of Learning a Rule, Reviews of Modern Physics 65, p.499, 1993.
Dana Angluin, Queries and Concept Learning, Machine Learning 2, 219–342, 1988.
Leonid Khachiyan, Complexity of Polytope Volume Computation, In: New Trends in Discrete and Computation Geometry, Janos Pach (Ed.), Springer Verlag, 1993.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1995 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Shamir, E., Shwartzman, C. (1995). Learning by extended statistical queries and its relation to PAC learning. In: Vitányi, P. (eds) Computational Learning Theory. EuroCOLT 1995. Lecture Notes in Computer Science, vol 904. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-59119-2_191
Download citation
DOI: https://doi.org/10.1007/3-540-59119-2_191
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-59119-1
Online ISBN: 978-3-540-49195-8
eBook Packages: Springer Book Archive