Agnostic Learning in Permutation-Invariant Domains
Author: 
Karl WimmerAuthors Info & Claims
Karl WimmerAuthors Info & ClaimsArticle No.: 46, Pages 1 - 22
Abstract
We generalize algorithms from computational learning theory that are successful under the uniform distribution on the Boolean hypercube {0, 1}n to algorithms successful on permutation-invariant distributions, distributions that stay invariant constant on permutating the coordinates in the instances. While the tools in our generalization mimic those used for the Boolean hypercube, the fact that permutation-invariant distributions are not product distributions presents a significant obstacle.
We prove analogous results for permutation-invariant distributions; more generally, we work in the domain of the symmetric group. We define noise sensitivity in this setting and show that noise sensitivity has a nice combinatorial interpretation in terms of Young tableaux. The main technical innovations involve techniques from the representation theory of the symmetric group, especially the combinatorics of Young tableaux. We show that low noise sensitivity implies concentration on “simple” components of the Fourier spectrum and that this fact will allow us to agnostically learn halfspaces under permutation-invariant distributions to constant accuracy in roughly the same time as in the uniform distribution over the Boolean hypercube case.
References
[1]
Eric Blais, Ryan O’Donnell, and Karl Wimmer. 2010. Polynomial regression under arbitrary product distributions. Mach. Learn. 80, 2 (2010), 273--294. http://www.springerlink.com/index/988211hq3v2q5171.pdf.
[2]
Dan Boneh. 1995. Learning using group representations. In Proceedings of the Eighth Annual Conference on Computational Learning Theory. ACM, 418--426. http://dl.acm.org/citation.cfm?id=225349
[3]
Persi Diaconis. 1988. Group representations in probability and statistics. Lecture Notes-Monograph Series 11 (1988), i--192. http://www.jstor.org/stable/10.2307/4355560.
[4]
David Ellis, Ehud Friedgut, and Haran Pilpel. 2011. Intersecting families of permutations. J. Am. Math. Soc. 24, 3 (2011), 649--682.
[5]
Vitaly Feldman and Pravesh Kothari. 2015. Agnostic learning of disjunctions on symmetric distributions. J. Mach. Learn. Res. 16 (2015), 3455--3467.
[6]
William Fulton. 1997. Young Tableaux: With Applications to Representation Theory and Geometry. Vol. 35. Cambridge University Press.
[7]
Jonathan Huang, Carlos Guestrin, and Leonidas Guibas. 2009. Fourier theoretic probabilistic inference over permutations. J. Mach. Learn. Res. 10 (2009), 997--1070. http://dl.acm.org/citation.cfm?id=1577106
[8]
G. D. James and Adalbert Kerber. 1981. The Representation Theory of the Symmetric Group. Addison-Wesley, Reading, MA. pp. xxviii, 510.
[9]
Adam Tauman Kalai, Adam R. Klivans, Yishay Mansour, and Rocco A. Servedio. 2008. Agnostically learning halfspaces. SIAM J. Comput. 37, 6 (2008), 1777--1805. http://epubs.siam.org/doi/abs/10.1137/060649057.
[10]
Adam Tauman Kalai, Alex Samorodnitsky, and Shang-Hua Teng. 2009. Learning and smoothed analysis. In Foundations of Computer Science, 2009. FOCS’09. 50th Annual IEEE Symposium on. IEEE, 395--404. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5438615.
[11]
Daniel M. Kane, Adam Klivans, and Raghu Meka. 2013. Learning halfspaces under log-concave densities: Polynomial approximations and moment matching. In COLT. 522--545.
[12]
Jovan Karamata. 1932. Sur une inégalité rélative aux fonctions convexes. Publ. Math. Univ. Belgrade 1 (1932), 145--148.
[13]
Michael J. Kearns, Robert E. Schapire, and Linda M. Sellie. 1994. Toward efficient agnostic learning. Mach. Learn. 17, 2--3 (1994), 115--141. http://link.springer.com/article/10.1007/BF00993468.
[14]
Adam R. Klivans, Ryan O’Donnell, and Rocco A. Servedio. 2004. Learning intersections and thresholds of halfspaces. J. Comput. Syst. Sci. 68, 4 (2004), 808--840.
[15]
Risi Kondor and Karsten M. Borgwardt. 2008. The skew spectrum of graphs. In Proceedings of the 25th International Conference on Machine Learning. ACM, 496--503. http://dl.acm.org/citation.cfm?id=1390219
[16]
Risi Kondor, Andrew Howard, and Tony Jebara. 2007. Multi-object tracking with representations of the symmetric group. In International Conference on Artificial Intelligence and Statistics. 211--218.
[17]
Nathan Linial, Yishay Mansour, and Noam Nisan. 1993. Constant depth circuits, Fourier transform, and learnability. J. ACM 40, 3 (1993), 607--620. http://dl.acm.org/citation.cfm?id=174138
[18]
Ryan O’Donnell. 2007. Lecture notes on analysis of boolean functions. (2007).
[19]
Ryan O’Donnell and Karl Wimmer. 2009. KKL, Kruskal-Katona, and monotone nets. In Foundations of Computer Science, 2009. FOCS’09. 50th Annual IEEE Symposium on. IEEE, 725--734. http://ieeexplore. ieee.org/xpls/abs_all.jsp?arnumber=5438582.
[20]
Andrei Okounkov and Grigori Olshanski. 1998. Shifted Schur functions. St. Petersburg Math. J. 9, 2 (1998), 239--300.
[21]
Yuval Peres. 2004. Noise stability of weighted majority. arXiv preprint math/0412377 (2004). http://arxiv.org/ abs/math/0412377.
[22]
Pannagadatta K. Shivaswamy and Tony Jebara. 2006. Permutation invariant SVMs. In Proceedings of the 23rd International Conference on Machine Learning. ACM, 817--824. http://dl.acm.org/citation. cfm?id=1143947
Index Terms
- Agnostic Learning in Permutation-Invariant Domains
Recommendations
Agnostically Learning under Permutation Invariant Distributions
FOCS '10: Proceedings of the 2010 IEEE 51st Annual Symposium on Foundations of Computer ScienceWe generalize algorithms from computational learning theory that are successful under the uniform distribution on the Boolean hypercube $\{0,1\}^n$ to algorithms successful on permutation invariant distributions. A permutation invariant distribution is ...
Permutation tableaux and permutation patterns
In this paper we introduce and study a class of tableaux which we call permutation tableaux; these tableaux are naturally in bijection with permutations, and they are a distinguished subset of the -diagrams of Alex Postnikov [A. Postnikov, Webs in ...
On the sign-imbalance of permutation tableaux
Permutation tableaux were introduced by Steingrmsson and Williams. Corteel and Kim defined the sign of a permutation tableau in terms of the number of unrestricted columns. The sign-imbalance of permutation tableaux of length n is the sum of signs over ...
Comments
Information & Contributors
Information
Published In
September 2016
310 pages
Copyright © 2016 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 03 August 2016
Accepted: 01 November 2015
Received: 01 January 2015
Published in TALG Volume 12, Issue 4
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed
Funding Sources
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 330Total Downloads
- Downloads (Last 12 months)40
- Downloads (Last 6 weeks)4
Reflects downloads up to 03 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in