Abstract
In 1990, E. Baum gave an elegant polynomial-time algorithm for learning the intersection of two origin-centered halfspaces with respect to any symmetric distribution (i.e., any \({\mathcal D}\) such that \({\mathcal D}(E) = {\mathcal D}(-E)\)) [3]. Here we prove that his algorithm also succeeds with respect to any mean zero distribution \({\mathcal D}\) with a log-concave density (a broad class of distributions that need not be symmetric). As far as we are aware, prior to this work, it was not known how to efficiently learn any class of intersections of halfspaces with respect to log-concave distributions.
The key to our proof is a “Brunn-Minkowski” inequality for log-concave densities that may be of independent interest.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Achlioptas, D., McSherry, F.: On spectral learning with mixtures of distributions. In: Auer, P., Meir, R. (eds.) COLT 2005. LNCS, vol. 3559, pp. 458–469. Springer, Heidelberg (2005)
Arriaga, R., Vempala, S.: An algorithmic theory of learning: Robust concepts and random projection. In: Proceedings of the 40th Annual Symposium on Foundations of Computer Science (FOCS), pp. 616–623 (1999)
Baum, E.: A polynomial time algorithm that learns two hidden unit nets. Neural Computation 2(4), 510–522 (1990)
Blum, A., Kannan, R.: Learning an intersection of a constant number of halfspaces under a uniform distribution. Journal of Computer and System Sciences 54(2), 371–380 (1997)
Blum, E.K., Lototsky, S.V.: Mathematics of Physics and Engineering. World Scientific, Singapore (2006)
Blumer, A., Ehrenfeucht, A., Haussler, D., Warmuth, M.K.: Learnability and the Vapnik-Chervonenkis dimension. JACM 36(4), 929–965 (1989)
Caramanis, C., Mannor, S.: An inequality for nearly log-concave distributions with applications to learning. IEEE Transactions on Information Theory 53(3), 1043–1057 (2007)
Dunagan, J.D.: A geometric theory of outliers and perturbation. PhD thesis, MIT (2002)
Kalai, A., Klivans, A., Mansour, Y., Servedio, R.: Agnostically learning halfspaces. In: Proceedings of the 46th IEEE Symposium on Foundations of Computer Science (FOCS), pp. 11–20 (2005)
Kannan, R., Salmasian, H., Vempala, S.: The spectral method for general mixture models. In: Proceedings of the Eighteenth Annual Conference on Learning Theory (COLT), pp. 444–457 (2005)
Kearns, M., Vazirani, U.: An introduction to computational learning theory. MIT Press, Cambridge (1994)
Klivans, A., O’Donnell, R., Servedio, R.: Learning geometric concepts via Gaussian surface area. In: Proc. 49th IEEE Symposium on Foundations of Computer Science (FOCS), pp. 541–550 (2008)
Klivans, A., Servedio, R.: Learning intersections of halfspaces with a margin. In: Proceedings of the 17th Annual Conference on Learning Theory, pp. 348–362 (2004)
Lovász, L., Vempala, S.: The geometry of logconcave functions and sampling algorithms. Random Structures and Algorithms 30(3), 307–358 (2007)
Vapnik, V.: Estimations of dependences based on statistical data. Springer, Heidelberg (1982)
Vempala, S.: A random sampling based algorithm for learning the intersection of halfspaces. In: Proc. 38th IEEE Symposium on Foundations of Computer Science (FOCS), pp. 508–513 (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Klivans, A.R., Long, P.M., Tang, A.K. (2009). Baum’s Algorithm Learns Intersections of Halfspaces with Respect to Log-Concave Distributions. In: Dinur, I., Jansen, K., Naor, J., Rolim, J. (eds) Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques. APPROX RANDOM 2009 2009. Lecture Notes in Computer Science, vol 5687. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03685-9_44
Download citation
DOI: https://doi.org/10.1007/978-3-642-03685-9_44
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03684-2
Online ISBN: 978-3-642-03685-9
eBook Packages: Computer ScienceComputer Science (R0)