Years and Authors of Summarized Original Work
1988; Pitt, Valiant
Problem Definition
The work of Pitt and Valiant [18] deals with learning Boolean functions in the Probably Approximately Correct (PAC) learning model introduced by Valiant [19]. A learning algorithm in Valiant’s original model is given random examples of a function f : {0, 1}n → {0, 1} from a representation class \(\mathcal{F}\) and produces a hypothesis \(h \in \mathcal{F}\) that closely approximates f. Here, a representation class is a set of functions and a language for describing the functions in the set. The authors give examples of natural representation classes that are NP-hard to learn in this model, whereas they can be learned if the learning algorithm is allowed to produce hypotheses from a richer representation class \(\mathcal{H}\). Such an algorithm is said to learn \(\mathcal{F}\) by \(\mathcal{H}\); learning \(\mathcal{F}\) by \(\mathcal{F}\) is called proper learning.
The results of Pitt and Valiant were...
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Recommended Reading
Alekhnovich M, Braverman M, Feldman V, Klivans A, Pitassi T (2008) The complexity of properly learning simple classes. J Comput Syst Sci 74(1):16–34
Ben-David S, Eiron N, Long PM (2003) On the difficulty of approximately maximizing agreements. J Comput Syst Sci 66(3):496–514
Blum AL, Rivest RL (1992) Training a 3-node neural network is NP-complete. Neural Netw 5(1):117–127
Blumer A, Ehrenfeucht A, Haussler D, Warmuth M (1989) Learnability and the Vapnik-Chervonenkis dimension. J ACM 36(4):929–965
Bshouty N (1995) Exact learning via the monotone theory. Inform Comput 123(1):146–153
Feldman V, Gopalan P, Khot S, Ponuswami A (2009) On agnostic learning of parities, monomials and halfspaces. SIAM J Comput 39(2):606–645
Feldman V (2009) Hardness of approximate two-level logic minimization and pac learning with membership queries. J Comput Syst Sci 75(1):13–26
Feldman V, Guruswami V, Raghavendra P, Wu Y (2012) Agnostic learning of monomials by halfspaces is hard. SIAM J Comput 41(6):1558–1590
Garey M, Johnson DS (1979) Computers and intractability. W.H. Freeman, San Francisco
Guruswami V, Raghavendra P (2009) Hardness of learning halfspaces with noise. SIAM J Comput 39(2):742–765
Hancock T, Jiang T, Li M, Tromp J (1995) Lower bounds on learning decision lists and trees. In: Mayr EW, Puech C (eds) STACS 95: 12th Annual Symposium on Theoretical Aspects of Computer Science, Munich, pp 527–538. Springer, Berlin/Heidelberg
Haussler D (1992) Decision theoretic generalizations of the PAC model for neural net and other learning applications. Inform Comput 100(1):78–150
Jackson J (1997) An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. J Comput Syst Sci 55:414–440
Kearns M, Schapire R, Sellie L (1994) Toward efficient agnostic learning. Mach Learn 17(2–3):115–141
Kearns M, Valiant L (1994) Cryptographic limitations on learning boolean formulae and finite automata. J ACM 41(1):67–95
Kearns M, Vazirani U (1994) An introduction to computational learning theory. MIT, Cambridge
Khot S, Saket R (2011) On the hardness of learning intersections of two halfspaces. J Comput Syst Sci 77(1):129–141
Pitt L, Valiant L (1988) Computational limitations on learning from examples. J ACM 35(4):965–984
Valiant LG (1984) A theory of the learnable. Commun ACM 27(11):1134–1142
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer Science+Business Media New York
About this entry
Cite this entry
Feldman, V. (2016). Hardness of Proper Learning. In: Kao, MY. (eds) Encyclopedia of Algorithms. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-2864-4_177
Download citation
DOI: https://doi.org/10.1007/978-1-4939-2864-4_177
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4939-2863-7
Online ISBN: 978-1-4939-2864-4
eBook Packages: Computer ScienceReference Module Computer Science and Engineering