Abstract
The Kushilevitz-Mansour (KM)algorithm is an algorithm that finds all the “heavy” Fourier coefficients of a boolean function. It is the main tool for learning decision trees and DNF expressions in the PAC model with respect to the uniform distribution. The algorithm requires an access to the membership query (MQ)oracle.
We weaken this requirement by producing an analogue of the KM algorithm that uses extended statistical queries (SQ)(SQs in which the expectation is taken with respect to a distribution given by a learning algorithm). We restrict a set of distributions that a learning algorithm may use for its SQs to be a set of specific constant bounded product distributions. Our analogue finds all the “heavy” Fourier coefficients of degree lower than c log n (we call it BS). We use BS to learn decision trees and by adapting Freund’s boosting technique we give algorithm that learns DNF in this model. Learning in this model implies learning with persistent classification noise and in some cases can be extended to learning with product attribute noise.
We develop a characterization for learnability with these extended SQs and apply it to get several negative results about the model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Javed Aslam, Scott Decatur. General bounds on statistical query learning and PAC learning with noise via hypothesis boosting. In Proceedings of 34-th Annual Symposium on Foundations of Computer Science, pp. 282–291, 1993.
Dana Angluin, Philip Laird. Learning from noisy examples. In Machine Learning, 2(4) pp.343–370, 1988.
Dana Angluin, Michael Kharitonov. When won’t membership queries help? In Proceedings of the 23-rd Annual ACM Symposium on Theory of Computing, 1991, pp. 454–454.
Avrim Blum, Merrick Furst, Jeffrey Jackson, Michael Kearns, Yishay Mansour, Steven Rudich. Weakly learning DNF and characterizing statistical query learning using Fourier analysis. In Proceedings of the 26-th Annual ACM Symposium on the Theory of Computing, pp. 253–262, 1994.
Nader Bshouty, Jeffrey Jackson, Christino Tamon. Uniform-distribution attribute noise learnability. In Proceedings of the 12-th Annual Conference on COLT, pp. 75–80, 1999.
Avrim Blum, Adam Kalai, Hal Wasserman. Noise-tolerant learning, the parity problem and the Statistical Query model. In Proceedings of the 32-th Annual ACM Symposium on Theory of Computing, pp. 435–440, 2000.
Yoav Freund. Boosting a weak learning algorithm by majority. In Proceedings of the Third Annual Workshop on COLT, pp. 202–216, 1990.
Sally Goldman, Robert Sloan. Can PAC learning algorithms tolerate random attribute noise? In Algorithmica, 14(1) pp. 70–84, 1995.
Sally Goldman, Michael Kearns, Robert Shapire. Exact identification of circuits using fixed points of amplification functions, In SIAM Journal on Computing, 22 (1993), pp. 705–726.
Jeffrey Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. In Proceedings of the 35th Annual Symposion on Foundations of Computer Science, pp. 42–53, 1993.
Jeffrey Jackson, Eli Shamir, Clara Shwartzman. Learning with queries corrupted by classification noise. In Proceedings of Fifth Israel Symposium on Theory of Computing and Systems, pp. 45–53, 1997.
Michael Kearns. Efficient noise-tolerant learning from statistical queries. In Proceedings of the Forth Annual Workshop on COLT, pp. 392–401, 1993.
Eyal Kushilevitz, Yishay Mansour. Learning decision trees using the Fourier spectrum. In Proceedings of the 23-rd Annual Symposium on Theory of Computing, pages 455–464.
Michael Kearns, Robert Shapire, Linda Sellie. Toward efficient agnostic learning. 2In Proceedings of the Fifth Annual Workshop on COLT, pp. 341–352, 1992.
Nathan Linial, Yishay Mansour, Noam Nisan. Constant depth circuits, Fourier transform, and learnability. In Proceedings of the 31-st Symposium on the Foundations of Computer Science, pp. 574–579, 1989.
Yishay Mansour. Learning Boolean Functions via the Fourier Transform. In Theoretical Advances in Neural Computation and Learning, (V.P. Roychodhury and K-Y. Siu and A. Orlitsky, ed.), 391–424, 1994.
Eli Shamir, Clara Shwartzman. Learning by extended statistical queries and its relation to PAC learning. In Proceedings of Second European Conference, EuroCOLT’ 95, pp. 357–366, 1995.
George Shakelford, Dennis Volper. Learning k-DNF with noise in the attributes. In Proceedings of the 1988Workshop on COLT, pp. 97–103, 1988.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bshouty, N.H., Feldman, V. (2001). On Using Extended Statistical Queries to Avoid Membership Queries. In: Helmbold, D., Williamson, B. (eds) Computational Learning Theory. COLT 2001. Lecture Notes in Computer Science(), vol 2111. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44581-1_35
Download citation
DOI: https://doi.org/10.1007/3-540-44581-1_35
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42343-0
Online ISBN: 978-3-540-44581-4
eBook Packages: Springer Book Archive