Abstract
We investigate the learnability of the class κ-ZOTFn of disjunctions of (at most) κk zero-one threshold functions with queries. We describe a poly(n)-time algorithm that identifies any concept from 2-ZOTFn with one proper equivalence query and O(n 2) membership queries, propose some techniques that work for larger κ in special cases and via a case analysis obtain an algorithm for learning 3-ZOTFn with O(n 5) membership and proper equivalence queries. Then we prove non-learnability results via exhibiting bounds κ(n) for which polynomial time learnability of κ(n)-ZOTFn with proper equivalence and membership queries becomes presumably intractable. Finally, we provide results on learning a single zero-one threshold function with queries: an efficient membership query algorithm for the case when the target has few relevant attributes, and a parallel algorithm that identifies a zero-one threshold function in constant time with O(n) membership queries.
On leave from Comenius University, Bratislava, Slovakia. Partially supported by the Academy of Finland under grant 22586, and by the Slovak Grant Agency VEGA under grant 14315.
Preview
Unable to display preview. Download preview PDF.
References
H. Aizenstein, L. Hellerstein and L. Pitt, “Read-Thrice DNF is Hard to Learn with Membership and Equivalence Queries”, in: Proceedings of the 33rd Annual IEEE Symposium on Foundations of Computer Science (FOCS'92), IEEE Computer Society Press, Los Alamitos, CA, 1992, pp. 523–532.
H. Aizenstein, T. Hegedűs, L. Hellerstein and L. Pitt, “Complexity Theoretic Hardness Results for Query Learning”, to appear in Computational Complexity.
D. Angluin, “Learning k-term DNF Formulas Using Queries and Counterexamples”, Technical Report YALEU/DCS/TR-559, Yale University, Department of Computer Science, 1987.
D. Angluin, “Queries and Concept Learning”, Machine Learning 2 (1988) 319–342.
D. Angluin, L. Hellerstein and M. Karpinski, “Learning Read-Once Formulas with Queries”, Journal of the ACM 40(1) (1993) 185–210.
M. Anthony and N. Biggs, Computational Learning Theory, Cambridge University Press, Cambridge, 1992.
P. T. Baffes and R. J. Mooney, “Extending Theory Refinement to M-of-N Rules”, Informatica 17 (1993) 387–397.
A. Blum, P. Chalasani and J. Jackson, “On Learning Embedded Symmetric Concepts”, in: Proceedings of the 6th Annual ACM Conference on Computational Learning Theory (COLT'93), ACM Press, New York, NY, 1993, pp. 337–346.
A. Blum, L. Hellerstein and N. Littlestone, “Learning in the Presence of Finitely or Infinitely Many Irrelevant Attributes”, Journal of Computer and System Sciences 50 (1995) 32–40.
N. H. Bshouty, “Exact Learning of Formulas in Parallel”, Machine Learning 26(1) (1997) 25–41.
N. H. Bshouty, T. R. Hancock, L. Hellerstein and M. Karpinski, “An Algorithm to Learn Read-Once Threshold Formulas, and Transformations between Learning Models”, Computational Complexity 4 (1994) 37–61.
M. D. Frazier, “Matterns Horn and Other Features in the Computational Learning Theory Landscape: The Notion of Membership”, Technical Report UIUCDCS-R-94-1858, University of Illinois at Urbana-Champaign, Department of Computer Science, 1994.
M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness, Freeman, San Francisco, CA, 1979.
Q. P. Gu and A. Maruoka, “Learning Monotone Boolean Functions by Uniformly Distributed Examples”, SIAM Journal on Computing 21(3) (1992) 587–599.
T. Hegedfis, “On Training Simple Neural Networks and Small-Weight Neurons”, in: Computational Learning Theory: EuroCOLT'93, Oxford University Press, Oxford, 1994, pp. 69–82.
K.-U. Höffgen, H.-U. Simon and K. S. Van Horn, “Robust Trainability of Single Neurons”, Journal of Computer and System Sciences 50(1) (1995) 114–125.
L. T. Kou, L. J. Stockmeyer and C. K. Wong, “Covering Edges by Cliques with Regard to Keyword Conflicts and Intersection Graphs”, Communications of the ACM 21(2) (1978) 135–139.
P. Langley, Elements of Machine Learning, Morgan Kaufmann, San Francisco, CA, 1995.
M. Li and P. Vitányi, “Learning Simple Concepts under Simple Distributions”, SIAM Journal on Computing 20 (1991) 911–935.
N. Littlestone, “Learning Quickly when Irrelevant Attributes Abound: a New Linear Threshold Algorithm”, Machine Learning 2(4) (1988) 285–318.
D. W. Loveland, “Finding Critical Sets”, Journal of Algorithms 8 (1987) 362–371.
W. Maass and Gy. Turán, “Lower Bound Methods and Separation Results for On-Line Learning Models”, Machine Learning 9 (1992) 107–145.
P. M. Murphy and M. J. Pazzani, “ID2-of-3: Constructive Induction of M-of-N Concepts for Discriminators in Decision Trees”, in: Proceedings of the 8th International Workshop on Machine Learning, Morgan Kaufmann, San Mateo, CA, 1991, pp. 183–187.
J. Orlin, “Contentment in Graph Theory: Covering Graphs with Cliques”, Indagationes Mathematicae 39(5) (1977) 406–424.
K. Pillaipakkamnatt and V. Raghavan, “On the Limits of Proper Learnability of Subclasses of DNF Formulas”, Machine Learning 25(2/3) (1996) 237–263.
L. Pitt and L. Valiant, “Computational Limitations on Learning from Examples”, Journal of the ACM 35 (1988) 965–984.
C. Schaffer, “Learning M-of-N and Related Concepts: An Empirical Comparison Study”, unpublished manuscript, 1994 (presented at the COLT'94 “State of the art in learning DNF rules” tutorial).
M. Schmitt, “On the Complexity of Consistency Problems for Neurons with Binary Weights”, Ulmer Informatik-Bericht Nr. 94–01, Fakultät für Informatik der Universität Ulm, 1994.
R. H. Sloan and Gy. Turán, “Learning from Incomplete Boundary Queries Using Split Graphs and Hypergraphs”, in: Proceedings of the 3rd European Conference on Computational Learning Theory (EuroCOLT'97), Springer-Verlag, LNCS 1208 (subseries LNAI), Berlin, 1997, pp. 38–50.
K. A. Spackman, “Learning Categorical Decision Criteria in Biomedical Domains”, in: Proceedings of the 5th International Workshop on Machine Learning, Morgan Kaufmann, San Mateo, CA, 1988, pp. 36–46.
K. M. Ting, “An M-of-N Rule Induction Algorithm and its Application to DNA Domain”, in: Proceedings of the 27th Annual Hawaii International Conference on System Sciences, Vol. V. Biotechnology Computing, IEEE Computer Society Press, Los Alamitos, CA, 1994, pp. 33–140.
R. Uehara, K. Tsuchida and I. Wegener, “Optimal Attribute-Efficient Learning of Disjunction, Parity, and Threshold Functions”, ECCC Technical Report TR96-061, 1996.
L. G. Valiant, “A Theory of the Learnable”, Communications of the ACM 27(11) (1984) 1134–1142.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1997 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hegedüs, T., Indyk, P. (1997). On learning disjunctions of zero-one threshold functions with queries. In: Li, M., Maruoka, A. (eds) Algorithmic Learning Theory. ALT 1997. Lecture Notes in Computer Science, vol 1316. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-63577-7_60
Download citation
DOI: https://doi.org/10.1007/3-540-63577-7_60
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-63577-2
Online ISBN: 978-3-540-69602-5
eBook Packages: Springer Book Archive