Skip to main content
SpringerLink
Log in

On the Proper Learning of Axis Parallel Concepts

  • Conference paper
  • First Online:
Computational Learning Theory (COLT 2002)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 2375))

Included in the following conference series:

  • 1147 Accesses

Abstract

We study the proper learnability of axis parallel concept classes in the PAC learning model and in the exact learning model with membership and equivalence queries. These classes include union of boxes, DNF, decision trees and multivariate polynomials. For the constant dimensional axis parallel concepts C we show that the following problems have the same time complexity

  1. 1.

    C is α-properly exactly learnable (with hypotheses of size at most α times the target size) from membership and equivalence queries.

  2. 2.

    C is α-properly PAC learnable (without membership queries) under any product distribution.

  3. 3.

    There is an α-approximation algorithm for the MinEqui C problem. (given a g ∈ C find a minimal size f ∈ C that is equivalent to g).

In particular, C is α-properly learnable in poly time from membership and equivalence queries if and only if C is α-properly PAC learnable in poly time under the product distribution if and only if MinEqui C has a poly time α-approximation algorithm. Using this result we give the first proper learning algorithm of decision trees over the constant dimensional domain and the first negative results in proper learning from membership and equivalence queries for many classes.

For the non-constant dimensional axis parallel concepts we show that with the equivalence oracle (1) ⇒ (3). We use this to show that (binary) decision trees are not properly learnable in polynomial time (assuming P≠NP) and DNF is not sε-properly learnable (∈ < 1) in polynomial time even with an NP-oracle (assuming Σ 2 p ≠ P NP).

This research was supported by the fund for promotion of research at the Technion. Research no. 120-025. Part of this research was done at the University of Calgary, Calgary, Alberta, Canada and supported by NSERC of Canada.

This research was supported by an NSERC PGS-B Scholarship, an Izaak Walton Killam Memorial Scholarship, and an Alberta Informatics Circle of Research Excellence (iCORE) Fellowship.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. D. Angluin. Queries and concept learning. Machine Learning, 319–342, 1988.

    Google Scholar 

  2. S. Arora, L. Babai, J. Stern, Z. Sweedyk. The hardness of approximate optima in lattices, codes, and systems of linear equations. JCSS, 43, 317–331, 1997.

    MathSciNet  Google Scholar 

  3. A. Beimel, F. Bergadano, N. H. Bshouty, E. Kushilevitz, S. Varricchio. Learning functions represented as multiplicity automata. JACM 47(3): 506–530, 2000.

    Article  MATH  MathSciNet  Google Scholar 

  4. A. Beimel and E. Kushilevitz. Learning boxes in high dimension. Algorithmica, 22(1/2):76–90, 1998.

    Article  MATH  MathSciNet  Google Scholar 

  5. S. Ben-David, N. H. Bshouty, E. Kushilevitz. A composition theorem for learning algorithms with applications to geometric concept classes. 29th STOC, 1997.

    Google Scholar 

  6. F. Bergadano, D. Catalano, S. Varricchio. Learning sat-k-DNF formulas from membership queries. In 28th STOC, pp 126–130, 1996.

    Google Scholar 

  7. P. Berman, B. DasGupta. Approximating the rectilinear polygon cover problems. In Proceedings of the 4th Canadian Conference on Computational Geometry, pages 229–235, 1992.

    Google Scholar 

  8. N. H. Bshouty, Exact learning of boolean functions via the monotone theory. Information and Computation, 123, pp. 146–153, 1995.

    Article  MATH  MathSciNet  Google Scholar 

  9. N. H. Bshouty. Simple learning algorithms using divide and conquer. In Proceedings of the Annual ACM Workshop on Computational Learning Theory, 1995.

    Google Scholar 

  10. N. H. Bshouty. A new composition theorem for learning algorithms. Proceedings of the 30th annual ACM Symposium on Theory of Computing (STOC), May 1998.

    Google Scholar 

  11. N. H. Bshouty, R. Cleve, R. Gavalda, S. Kannan, C. Tamon. Oracles and queries that are sufficient for exact learning. JCSS 52(3): pp. 421–433, 1996.

    MATH  MathSciNet  Google Scholar 

  12. N. H. Bshouty, P. W. Goldberg, S. A. Goldman, D. H. Mathias. Exact learning of discretized geometric concepts. SIAM J. of Comput. 28(2), 678–699, 1999.

    MathSciNet  Google Scholar 

  13. Z. Chen and W. Homer. The bounded injury priority method and the learnability of unions of rectangles. Annals of Pure and Applied Logic 77(2):143–168, 1996.

    Article  MATH  MathSciNet  Google Scholar 

  14. Z. Chen, W. Maass. On-line learning of rectangles and unions of rectangles, Machine Learning, 17(1/2), pages 201–223, 1994.

    MATH  Google Scholar 

  15. V. J. Dielissen, A. Kaldewaij. Rectangular partition is polynomial in two dimensions but NP-complete in three. In IPL, pages 1–6, 1991.

    Google Scholar 

  16. D. Franzblau. Performance guarantees on a sweep-line heuristic for covering rectilinear polygons with rectangles. SIAM J. on Discrete Math 2, 307–321, 1989.

    Article  MATH  MathSciNet  Google Scholar 

  17. L. Hellerstein, K. Pillaipakkamnatt, V. Raghavan, D. Wilkins. How many queries are needed to learn? JACM, 43(5), pages 840–862, 1996.

    Article  MATH  MathSciNet  Google Scholar 

  18. L. Hellerstein and V. Raghavan. Exact learning of DNF formulas using DNF hypotheses. In 34rd Annual Symposium on Theory of Computing (STOC), 2002.

    Google Scholar 

  19. A. Klivans and R. Servedio. Learning DNF in Time 2O(n1/3). In 33rd Annual Symposium on Theory of Computing (STOC), 2001, pp. 258–265.

    Google Scholar 

  20. W. Lipski, Jr., E. Lodi, F. Luccio, C. Mugnai and L. Pagli. On two-dimensional data organization II. In Fund. Inform. 2, pages 245–260, 1979.

    MathSciNet  MATH  Google Scholar 

  21. W. Maass and M. K. Warmuth Efficient learning with virtual threshold gates Information and Computation, 141(1): 66–83, 1998.

    Article  MATH  MathSciNet  Google Scholar 

  22. K. Pillaipakkamntt, V. Raghavan, On the limits of proper learnability of subclasses of DNF formulas, Machine Learning, 25(2/3), pages 237–263, 1996.

    Article  Google Scholar 

  23. L. Pitt and L. G. Valiant. Computational Limitations on Learning from Examples. Journal of the ACM, 35(4):965–984, October 1988.

    Google Scholar 

  24. R. E. Schapire, L. M. Sellie. Learning sparse multivariate polynomial over a field with queries and counterexamples. In Proceedings of the Sixth Annual ACM Workshop on Computational Learning Theory. July, 1993.

    Google Scholar 

  25. C. Umans. Hardness of approximating Σ2 p minimization problems. In Proceedings of the 40th Symposium on Foundations of Computer Science, 1999.

    Google Scholar 

  26. L. Valiant. A theory of the learnable. Communications of the ACM, 27(11):1134–1142, November 1984.

    Google Scholar 

  27. H. Zantema, H. Bodlaender. Finding small equivalent decision trees is hard. In International Journal of Foundations of Computer Science, 11(2): 343–354, 2000.

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Technion, Haifa, Israel

    Nader H. Bshouty

  2. Department of Computer Science, University of Calgary, Calgary, Alberta, Canada

    Lynn Burroughs

Authors
  1. Nader H. Bshouty
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lynn Burroughs
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Research School of Information Sciences and Engineering, Australian National University, Canberra, ACT, 0200, Australia

    Jyrki Kivinen

  2. Computer Science Department, University of Illinois at Chicago, 851 S. Morgan St., Chicago, IL, 60607, USA

    Robert H. Sloan

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Bshouty, N.H., Burroughs, L. (2002). On the Proper Learning of Axis Parallel Concepts. In: Kivinen, J., Sloan, R.H. (eds) Computational Learning Theory. COLT 2002. Lecture Notes in Computer Science(), vol 2375. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45435-7_20

Download citation

  • DOI: https://doi.org/10.1007/3-540-45435-7_20

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-43836-6

  • Online ISBN: 978-3-540-45435-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation