Abstract
We prove the following results. Any Boolean function of O(log n) relevant variables can be exactly learned with a set of non-adaptive membership queries alone and a minimum sized decision tree representation of the function constructed, in polynomial time. In contrast, such a function cannot be exactly learned with equivalence queries alone using general decision trees and other representation classes as hypotheses.
Our results imply others which may be of independent interest. We show that truth-table minimization of decision trees can be done in polynomial time, complementing the well-known result of Masek that truth-table minimization of DNF formulas is NP-hard. The proofs of our negative results show that general decision trees and related representations are not learnable in polynomial time using equivalence queries alone, confirming a folklore theorem.
Supported by the Esprit EU BRA program under project 20244 (ALCOM-IT) and the EC Working Group NeuroCOLT2 - EP27150.
Supported by the Esprit EU BRA program under project 20244 (ALCOM-IT) and the EC Working Group NeuroCOLT2 - EP27150.
This work was supported by NSF grant CCR-9510392.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
D. Angluin. Queries and Concept Learning. Machine Learning, 2, (1988), 319–342.
D. Angluin. Negative Results for Equivalence Queries. Machine Learning, 5, (1990), 121–150.
J.C. Bioch and T. Ibaraki. Complexity of Identification and Dualization of Positive Boolean Functions. Information and Computation, 123:50–63, 1995.
A. Blum, L. Hellerstein, N. Littlestone. Learning in the Presence of Finitely or Infinitely Many Attributes. Journal of Computer and System Science, pages 50:32–40, 1995.
N. Bshouty. Exact Learning Boolean Functions via the Monotone Theory. Information and Computation, 123(1):146–153, Nov 1995.
N. Bshouty, R. Cleve, S. Kannan, and C. Tamon. Oracles and Queries that are Sufficient for Exact Learning. Proceedings of the Seventh Annual ACM Conference on Computational Learning Theory, pages 130–139, 1994.
N.H. Bshouty, L. Hellerstein. Attribute-efficient Learning in Query and Mistake Bound Models. Proceedings of the Ninth Conference on Computational Learning Theory, pages 235–243, 1996.
P. Damaschke. Adaptive versus Non-Adaptive Attribute-Efficient Learning. Proceedings of the 35th ACM Symposium on Theory of Computing, pages 590–596, 1998.
P. Damaschke. Computational Aspects of Parallel Attribute-Efficient Learning. To appear in: Proceedings of the 9th International Workshop on Al gorithmic Learning Theory, 1998.
A. Dhagat, L. Hellerstein. PAC Learning with Irrelevant Attributes. Proceedings of the 35th IEEE Foundations of Computer Science, pages 64–75, 1994.
R. Gavaldá. On the Power of Equivalence Queries. Proceedings of the 2nd European Conference on Computational Learning Theory, (1993), 193–203.
M.R. Garey and D.S. Johnson. Computers and Intractability: A Guide to the Theory of NP-Completeness. W.H. Freeman and Company, New York, 1979.
S. Goldman, M. Kearns. On the Complexity of Teaching. Proceedings of the 4rd Workshop on Computational Learning Theory, (1991), 303–314.
T. Hancock. Identifying µ-Formula Decision Trees with Queries. Proceedings of the Third Annual Workshop on Computational Learning Theory, pages 23–37, 1990.
L. Hellerstein, K. Pillaipakkamnatt, V. Raghavan, D. Wilkins. How Many Queries are Needed to Learn? Proceedings of the 27th ACM Symposium on the Theory of Computing, (1995), 190–199.
E. Kushilevitz and Y. Mansour. Learning Decision Trees Using the Fourier Spectrum. SIAM Journal of Computing, 22(6):1331–1348, 1993.
N. Littlestone. Learning Quickly When Irrelevant Variables Abound. Machine Learning, 2:285–318, 1988.
J. Naor and M. Naor. Small-bias probability spaces: Efficient constructions and applications. Proceedings of the 22nd ACM Symposium on the Theory of Computing, (1990), 312–323.
D. Wilkins. Learning Restricted-Read Branching Programs with Queries. Ph.D. Thesis, Vanderbilt University, 1995.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Guijarro, D., Lavín, V., Raghavan, V. (1999). Exact Learning when Irrelevant Variables Abound. In: Fischer, P., Simon, H.U. (eds) Computational Learning Theory. EuroCOLT 1999. Lecture Notes in Computer Science(), vol 1572. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-49097-3_8
Download citation
DOI: https://doi.org/10.1007/3-540-49097-3_8
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-65701-9
Online ISBN: 978-3-540-49097-5
eBook Packages: Springer Book Archive