Skip to main content
Springer Nature Link
Log in

Statistical Query Learning

  • Reference work entry
  • First Online:
Encyclopedia of Algorithms

Years and Authors of Summarized Original Work

  • 1998; Kearns

Problem Definition

The problem deals with learning to classify from random labeled examples in Valiant’s PAC model [30]. In the random classification noise model of Angluin and Laird [1], the label of each example given to the learning algorithm is flipped randomly and independently with some fixed probability η called the noise rate. Robustness to such benign form of noise is an important goal in the design of learning algorithms. Kearns defined a powerful and convenient framework for constructing noise-tolerant algorithms based on statistical queries. Statistical query (SQ) learning is a natural restriction of PAC learning that models algorithms that use statistical properties of a data set, rather than individual examples. Kearns demonstrated that any learning algorithm that is based on statistical queries can be automatically converted to a learning algorithm in the presence of random classification noise of arbitrary rate...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Recommended Reading

  1. Angluin D, Laird P (1988) Learning from noisy examples. Mach Learn 2:343–370

    Google Scholar 

  2. Aslam J, Decatur S (1998) General bounds on statistical query learning and pac learning with noise via hypothesis boosting. Inf Comput 141(2):85–118

    Article  MathSciNet  MATH  Google Scholar 

  3. Balcan M-F, Feldman V (2013) Statistical active learning algorithms. In: NIPS, Lake Tahoe, pp 1295–1303

    Google Scholar 

  4. Ben-David S, Itai A, Kushilevitz E (1990) Learning by distances. In: Proceedings of COLT, Rochester, pp 232–245

    MATH  Google Scholar 

  5. Blum A, Dwork C, McSherry F, Nissim K (2005) Practical privacy: the SuLQ framework. In: Proceedings of PODS, Baltimore, pp 128–138

    Google Scholar 

  6. Blum A, Frieze A, Kannan R, Vempala S (1997) A polynomial time algorithm for learning noisy linear threshold functions. Algorithmica 22(1/2):35–52

    MathSciNet  MATH  Google Scholar 

  7. Blum A, Furst M, Jackson J, Kearns M, Mansour Y, Rudich S (1994) Weakly learning DNF and characterizing statistical query learning using Fourier analysis. In: Proceedings of STOC, Montréal, pp 253–262

    Google Scholar 

  8. Blum A, Kalai A, Wasserman H (2003) Noise-tolerant learning, the parity problem, and the statistical query model. J ACM 50(4):506–519

    Article  MathSciNet  MATH  Google Scholar 

  9. Bshouty N, Feldman V (2002) On using extended statistical queries to avoid membership queries. J Mach Learn Res 2:359–395

    MathSciNet  MATH  Google Scholar 

  10. Chu C, Kim S, Lin Y, Yu Y, Bradski G, Ng A, Olukotun K (2006) Map-reduce for machine learning on multicore. In: Proceedings of NIPS, Vancouver, pp 281–288

    Google Scholar 

  11. Dachman-Soled D, Feldman V, Tan L-Y, Wan A, Wimmer K (2014) Approximate resilience, monotonicity, and the complexity of agnostic learning. arXiv, CoRR, abs/1405.5268

    Google Scholar 

  12. Dunagan J, Vempala S (2004) A simple polynomial-time rescaling algorithm for solving linear programs. In: Proceedings of STOC, Chicago, pp 315–320

    MATH  Google Scholar 

  13. Dwork C, McSherry F, Nissim K, Smith A (2006) Calibrating noise to sensitivity in private data analysis. In: TCC, New York, pp 265–284

    MATH  Google Scholar 

  14. Feldman V (2008) Evolvability from learning algorithms. In: Proceedings of STOC, Victoria, pp 619–628

    MATH  Google Scholar 

  15. Feldman V (2012) A complete characterization of statistical query learning with applications to evolvability. J Comput Syst Sci 78(5):1444–1459

    Article  MathSciNet  MATH  Google Scholar 

  16. Feldman V (2014) Open problem: the statistical query complexity of learning sparse halfspaces. In: COLT, Barcelona, pp 1283–1289

    Google Scholar 

  17. Feldman V, Grigorescu E, Reyzin L, Vempala S, Xiao Y (2013) Statistical algorithms and a lower bound for planted clique. In: STOC, Palo Alto. ACM, pp 655–664

    MATH  Google Scholar 

  18. Feldman V, Kanade V (2012) Computational bounds on statistical query learning. In: COLT, Edinburgh, pp 16.1–16.22

    Google Scholar 

  19. Feldman V, Lee H, Servedio R (2011) Lower bounds and hardness amplification for learning shallow monotone formulas. In: COLT, Budapest, vol 19, pp 273–292

    Google Scholar 

  20. Feldman V, Perkins W, Vempala S (2013) On the complexity of random satisfiability problems with planted solutions. In: CoRR, abs/1311.4821

    Google Scholar 

  21. Jackson J, Shamir E, Shwartzman C (1997) Learning with queries corrupted by classification noise. In: Proceedings of the fifth Israel symposium on the theory of computing systems, Ramat-Gan, pp 45–53

    MATH  Google Scholar 

  22. Kallweit M, Simon H (2011) A close look to margin complexity and related parameters. In: COLT, Budapest, pp 437–456

    Google Scholar 

  23. Kasiviswanathan SP, Lee HK, Nissim K, Raskhodnikova S, Smith A (2011) What can we learn privately? SIAM J Comput 40(3):793–826

    Article  MathSciNet  MATH  Google Scholar 

  24. Kearns M (1998) Efficient noise-tolerant learning from statistical queries. J ACM 45(6): 983–1006

    Article  MathSciNet  MATH  Google Scholar 

  25. Klivans A, Sherstov A (2007) Unconditional lower bounds for learning intersections of halfspaces. Mach Learn 69(2–3):97–114

    Article  Google Scholar 

  26. Laird P (1988) Learning from good and bad data. Kluwer Academic, Boston

    Book  MATH  Google Scholar 

  27. Sherstov AA (2008) Halfspace matrices. Comput Complex 17(2):149–178

    Article  MathSciNet  MATH  Google Scholar 

  28. Simon H (2007) A characterization of strong learnability in the statistical query model. In: Proceedings of symposium on theoretical aspects of computer science, Aachen, pp 393–404

    MATH  Google Scholar 

  29. Szörényi B (2009) Characterizing statistical query learning: simplified notions and proofs. In: Proceedings of ALT, Porto, pp 186–200

    MATH  Google Scholar 

  30. Valiant LG (1984) A theory of the learnable. Commun ACM 27(11):1134–1142

    Article  MATH  Google Scholar 

  31. Valiant LG (2009) Evolvability. J ACM 56(1):3.1–3.21. Earlier version in ECCC, 2006

    Google Scholar 

  32. Yang K (2005) New lower bounds for statistical query learning. J Comput Syst Sci 70(4):485–509

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IBM Research – Almaden, San Jose, CA, USA

    Vitaly Feldman

Authors
  1. Vitaly Feldman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vitaly Feldman .

Editor information

Editors and Affiliations

  1. Department of Electrical Engineering and Computer Science, Northwestern University, Evanston, IL, USA

    Ming-Yang Kao

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer Science+Business Media New York

About this entry

Cite this entry

Feldman, V. (2016). Statistical Query Learning. In: Kao, MY. (eds) Encyclopedia of Algorithms. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-2864-4_401

Download citation

Publish with us

Policies and ethics