Abstract
Many studies have been done in the literature on minimum disagreement problems and their connection to Agnostic learning and learning with Malicious errors. We further study these problems and some extensions of them. The classes that are studied in the literature are monomials, monotone monomials, antimonotone monomials, decision lists, halfspaces, neural networks and balls. For some of these classes we improve on the best previously known factors for approximating the minimum disagreement. We also find new bounds for exclusive-or, k-term DNF, k-DNF and multivariate polynomials (Xor of monomials).
We then apply the above and some other results from the literature to Agnostic learning and give negative and positive results for Agnostic learning and PAC learning with malicious errors of the above classes.
This research was supported by the fund for promotion of research at the Technion. Research no. 120-025. Part of this research was done at the University of Calgary, Calgary, Alberta, Canada.
This research was supported by an NSERC PGS-B Scholarship, an Izaak Walton Killam Memorial Scholarship, and an Alberta Informatics Circle of Research Excellence (iCORE) Fellowship.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
D. Angluin and P. D. Laird, Learning from noisy examples, Machine Learning, 2:343–370, 1988.
S. Arora, L. Babai, J. Stern and Z. Sweedyk, Hardness of approximate optima in lattices, codes, and systems of linear equations, JCSS 54(2): 317–331, 1997.
P. L. Barlett, S. Ben-David. Hardness results for neural network approximation problems. 4th Euro-COLT, 50–62, 1999.
S. Ben-David, N. Eiron, P. M. Long. On the difficulty of approximately maximizing agreement. 13th COLT, 266–274, 2000.
P. Berman and M. Karpinski. Approximating minimal unsatisfiability of linear equationsx, Manuscript. From ECCC reports 2001, TR01–025.
N. H. Bshouty and L. Burroughs. Maximizing Agreements, CoAgnostic Learning and Weak Agnostic Learning. Manuscript, 2002.
U. Feige, A threshold of log n for approximating set cover, 28th STOC, 314–318, 1996.
M. R. Garey, D.S. Johnson, Computers and Intractability: A Guide to the theory of NP-Completeness. W. H. Freeman, 1979.
J. Håstad, Some optimal inapproximability results, 29th STOC, 1–10, 1997.
D. Haussler, Decision theoretic generalization of the PAC model for neural net and other learning applications. Inform. Comput., 100(1):78–150, Sept. 1992.
Klaus-U Höffgen, Hans-U. Simon and Kevin S. Van Horn, Robust train-ability of single neurons, J. of Comput. Syst. Sci., 50(1): 114–125, 1995.
D. Karger, R. Motwani, and M. Sudan, Approximate graph coloring by semidefinite programming. 35th FOCS, 2–13, 1994.
M. Kearns and M. Li, Learning in the presence of malicious errors, SIAM Journal on Computing, 22(4): 807–837, 1993.
M. Kearns, M. Li, L. Pitt and L. Valiant. On the learnability of Boolean formulae, 19th STOC, 285–295, 1987.
M. Kearns, R. E. Schapire and L. M. Sellie. Toward efficient agnostic learning. 5th COLT, 341–352, 1992.
S. Khanna, N. Linial and S. Safra. On the hardness of approximating the chromatic number, Proc. 2nd Israel Symp. on Theory of Computing and Systems, 250–260, 1993.
C. Kuhlmann. Harness Results for General Two-Layer Neural Networks. 13th COLT, 275–285, 2000.
Y. Mansour, Learning Boolean Functions via the Fourier Transform. In Theoretical Advances in Neural Computation and Learning, (V. P. Roy chodhury, K-Y. Siu and A. Orlitsky, ed), 391–424 (1994).
L. Pitt and L. G. Valiant, Computational Limitation on Learning from Examples. JACM, 35,(4): 965–984, 1988.
R. Raz, A parallel repetition theorem. 27th STOC, 447–456, 1995.
H. U. Simon, private communication, 2001.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bshouty, N.H., Burroughs, L. (2002). Bounds for the Minimum Disagreement Problem with Applications to Learning Theory. In: Kivinen, J., Sloan, R.H. (eds) Computational Learning Theory. COLT 2002. Lecture Notes in Computer Science(), vol 2375. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45435-7_19
Download citation
DOI: https://doi.org/10.1007/3-540-45435-7_19
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43836-6
Online ISBN: 978-3-540-45435-9
eBook Packages: Springer Book Archive