Abstract
We present some problems with geometric characterizations that arise naturally in practical applications of machine learning. Our motivation comes from a well known machine learning problem, the problem of computing decision trees. Typically one is given a dataset of positive and negative points, and has to compute a decision tree that fits it. The points are in a low dimensional space, and the data are collected experimentally. Most practical solutions use heuristic algorithms.
To compute decision trees quickly, one has to solve optimization problems in one or more dimensions efficiently. In this paper we give geometric characterizations for these problems. We present a selection of algorithms for some of them. These algorithms are motivated from practice, and have been in many cases implemented and used as well. In addition, they are theoretically interesting, and typically employ sophisticated geometric techniques. Finally we present future research directions.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
E. Arkin, H. Meijer, J. Mitchell, D. Rappaport, and S. Skiena, Decision Trees for Geometric Models. Proc. Comput. Geom. Conf. (1993), 369–378.
T. Asano, D. Chen, N. Katoh, and T. Tokuyama, Polynomial-Time solutions to Image Segmentation. Proc. 7th ACM-SIAM Symp. on Disc. Algorithms (1996), 104–113.
P. Auer, R. Holte and W. Maass, Theory and Applications of Agnostic PAC-Learning with Small Decision Trees. Proc. 12th Int. Conf. Machine Learning (1995).
C. Boutilier, R. Dearden and M. Goldszmidt. Exploiting Structure in Policy Construction Proceedings of the International Joint Conference on Artificial Intelligence, Montreal, Michigan, 1995.
L. Breiman, J.H. Friedman, R.A. Olshen, and C.J. Stone. Classification and Regression Trees, Belmont, CA: Wadsworth International Group, 1984.
W. Buntine and T. Niblett, A further comparison of splitting rules for decision-tree induction. Machine Learning, 8 (1992), 75–82.
G. Das and M. Goodrich, On the complexity of Optimization Problems for 3-Dimensional Convex Polyhedra and Decision Trees. WADS 1995.
D. Dobkin, D. Eppstein and D. Mitchell, Computing the Discrepancy with Applications to Supersampling Patterns. ACM Transactions on Graphics, to appear.
D. Dobkin and D. Gunopulos, Concept Learning with Geometric Hypotheses, 8th ACM Conference on Learning Theory (1995).
D. Dobkin, D. Gunopulos and S. Kasif, Computing optimum shallow decision trees. 4th AI and Math. Symposium (1996).
D. Dobkin, D. Gunopulos, S. Kasif, J. Fulton and S. Salzberg, Induction of Shallow Decision Trees. submitted to IEEE PAMI.
D. Dobkin, D. Gunopulos and W. Maass, Computing the maximum Bichromatic Discrepancy, with applications in Computer Graphics and Machine Learning. J. Comp. Syst. Sciences, to appear.
P. Fischer, More or less efficient agnostic learning of convex polygons. 8th ACM Conference on Computational Learning Theory (1995).
T. Fulton, S. Kasif and S. Salzberg, An Efficient Algorithm for Finding Multi-way Splits in Decision Trees. Proc. Machine Learning 1995.
D. Haussler, Decision theoretic generations of the PAC-model for neural nets and other applications. Inf. and Comp., 100 (1992), 78–150.
R.C. Holte, Very simple classification rules perform well on most commonly used datasets. Machine Learning, 11 (1993), 63–91.
M. Kearns, R.E. Schapire and L.M. Sellie, Toward efficient agnostic learning. 5th ACM Workshop on Computational Learning Theory (1992), 341–352.
S. Kwek, Minimizing disagreements for geometric regions using dynamic programming, with applications to machine learning and computer graphics. Manuscript, 1995.
D. Lubinsky, Bivariate splits and consistent split criteria in dichotomous classification trees. Ph.D. Thesis, Rutgers University, Department of Computer Science, 1994.
W. Maass, Efficient Agnostic PAC-Learning with Simple Hypotheses. 7th Ann. ACM Conference on Computational Learning Theory (1994), 67–75.
J. Mingers, An empirical comparison of pruning methods for decision tree induction. Machine Learning, 4 (1989), 227–243.
B. M.E. Moret, Decision Trees and diagrams. Computing surveys, 14(4) (1982), 593–623.
S. Murthy, S. Kasif and S. Salzberg, A system for induction of oblique decision trees. Journal of Artificial Intelligence Research, 1 (1994), 257–275.
S. Murthy, S. Kasif, S. Salzberg and R. Beigel, OC1: Randomized induction of oblique decision trees. AAAI 93 [2], 322–327.
R. W. Payne and D. A. Preece, Identification trees and diagnostic tables: A review. Journal of the Royal Statistical Society: series A, 143 (1980), 253.
J.R. Quinlan. Induction of Decision Trees. Machine Learning, 1 (1986), 81–106.
J.R. Quinlan. C4.5: Programs for Machine Learning, Morgan Kaufmann, Los Altos, CA, 1993.
J. R. Quinlan. Oversearching and Layered Search in Empirical Learning. Proceedings of the International Joint Conference on Artificial Intelligence, Montreal, Michigan, 1995.
S. R. Safarin and D. Landgrebe, A survey of decision tree classifier methodology. IEEE Transactions on Systems, Man and Cybernetics, 21(3) (1994), 309–318.
S. Salzberg, R. Chandar, H. Ford, S. Murthy and R. White, Decision trees for automated identification of cosmic-ray hits in humble space telescope images. Publications of the Astronomical Society of the Pacific, 107, 1–10 (March 1995).
L.G. Valiant, A theory of the learnable. Comm. of the ACM 27 (1984), 1134–1142.
S.M. Weiss, R. Galen and P.V. Tadepalli, Maximizing the predictive value of production rules. Art. Int. 45 (1990), 47–71.
S.M. Weiss and I. Kapouleas, An empirical comparison of pattern recognition, neural nets, and machine learning classification methods. 11th Int. Joint Conf. on Art. Int. (1990), Morgan Kauffmann, 781–787.
S.M. Weiss and C.A. Kulikowski, Computer Systems that Learn, Morgan Kauffmann Publishers, Palo Alto, CA, 1991.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1996 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dobkin, D., Gunopulos, D. (1996). Geometric problems in machine learning. In: Lin, M.C., Manocha, D. (eds) Applied Computational Geometry Towards Geometric Engineering. WACG 1996. Lecture Notes in Computer Science, vol 1148. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0014490
Download citation
DOI: https://doi.org/10.1007/BFb0014490
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-61785-3
Online ISBN: 978-3-540-70680-9
eBook Packages: Springer Book Archive