Abstract
We present a polynomial time algorithm for learning several models of algebraic computation. We show that any arithmetic circuit whose partial derivatives induce a “low”-dimensional vector space is exactly learnable from membership and equivalence queries. As a consequence we obtain the first polynomial time algorithm for learning depth three multilinear arithmetic circuits. In addition, we give the first polynomial time algorithms for learning restricted algebraic branching programs and noncommutative arithmetic formulae. Previously only restricted versions of depth-2 arithmetic circuits were known to be learnable in polynomial time. Our learning algorithms can be viewed as solving a generalization of the well known polynomial interpolation problem where the unknown polynomial has a succinct representation. We can learn representations of polynomials encoding exponentially many monomials. Our techniques combine a careful algebraic analysis of arithmetic circuits’ partial derivatives with the “multiplicity automata” techniques due to Beimel et al [BBB+00].
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Aspnes, J., Beigel, R., Furst, M., Rudich, S.: The expressive power of voting polynomials. In: Awerbuch, B. (ed.) Proceedings of the 23rd Annual ACM Symposium on the Theory of Computing, pp. 402–409. ACM Press, New York (1991)
Arora, S., Lund, C., Motwani, R., Sudan, M., Szegedy, M.: Proof verification and the hardness of approximation problems. Journal of the ACM 45(3), 501–555 (1998)
Angluin, D.: Queries and concept learning. Machine Learning 2, 319 (1987)
Bshouty, D., Bshouty, N.H.: On interpolating arithmetic readonce formulas with exponentiation. Journal of Computer and System Sciences 56(1), 112–124 (1998)
Beimel, A., Bergadano, F., Bshouty, N.H., Kushilevitz, E., Varricchio, S.: Learning functions represented as multiplicity automata. J. ACM 47(3), 506–530 (2000)
Bergadano, F., Bshouty, N.H., Tamon, C., Varricchio, S.: On learning branching programs and small depth circuits. In: Ben-David, S. (ed.) EuroCOLT 1997. LNCS (LNAI), vol. 1208, pp. 150–161. Springer, Heidelberg (1997)
Bshouty, N., Mansour, Y.: Simple learning algorithms for decision trees and multivariate polynomials. SICOMP: SIAM Journal on Computing 31 (2002)
Bshouty, N.H., Tamon, C., Wilson, D.K.: On learning width two branching programs. In: Proc. 9th Annu. Conf. on Comput. Learning Theory, pp. 224–227. ACM Press, New York (1996)
Carlyle, J.W., Paz, A.: Realizations by stochastic finite automata. Journal of Computer and System Sciences 5(1), 26–40 (1971)
Fliess, M.: Matrices de Hankel. J. Math. Pures et Appl. 53, 197–224 (1974)
Grigoriev, D., Karpinski, M., Singer, M.F.: Computational complexity of sparse rational interpolation. SIAM Journal on Computing 23(1), 1–11 (1994)
Grigoriev, D., Razborov, A.A.: Exponential complexity lower bounds for depth 3 arithmetic circuits in algebras of functions over finite fields. In: IEEE (ed.) 39th Annual Symposium on Foundations of Computer Science: proceedings, Palo Alto, California, 1109 Spring Street, Suite 300, Silver Spring, MD 20910, USA, November 8–11, pp. 269–278. IEEE Computer Society Press, Los Alamitos (1998)
Hastad, J.: Almost optimal lower bounds for small depth circuits. In: Proceedings of the 18th Annual ACM Symposium on Theory of Computing, Berkeley, California, May 28-30, pp. 6–20 (1986)
Huang, M., Rao, T.: Interpolation of sparse multivariate polynomials over large finite fields with applications. ALGORITHMS: Journal of Algorithms 33 (1999)
Jackson, J.C., Klivans, A.R., Servedio, R.A.: Learnability beyond ACÔ. In: Proceedings of the 34th Annual ACM Symposium on Theory of Computing (STOC 2002), New York, May 19–21, pp. 776–784. ACM Press, New York (2002)
Klivans, A.R., Spielman, D.: Randomness efficient identity testing of multivariate polynomials. In: ACM (ed.) Proceedings of the 33rd Annual ACM Symposium on Theory of Computing: Hersonissos, Crete, Greece, New York, NY, USA, July 6–8, pp. 216–223. ACM Press, New York (2001)
Kaltofen, E., Trager, B.M.: Computing with polynomials given by black boxes for their evaluations: Greatest common divisors, factorization, separation of numerators and denominators. Journal of Symbolic Computation 9(3), 300–320 (301–320) (1990)
Linial, N., Mansour, Y., Nisan, N.: Constant depth circuits, Fourier transform and learnability. Journal of the ACM 40(3), 607–620 (1993)
Nisan, N.: Lower bounds for non-commutative computation (extended abstract). In: Proceedings of the 23rd Annual ACM Symposium on Theory of Computing, New Orleans, Louisiana, May 6–8, pp. 410–418 (1991)
Nisan, N., Wigderson, A.: Lower bounds on arithmetic circuits via partial derivatives. CMPCMPL: Computational Complexity 6 (1997)
Ohnishi, H., Seki, H., Kasami, T.: A polynomial time learning algorithm for recognizable series. TIEICE: IEICE Transactions on Communications/ Electronics/Information and Systems (1994)
Schapire, R.E., Sellie, L.M.: Learning sparse multivariate polynomials over a field with queries and counterexamples. Journal of Computer and System Sciences 52(2), 201–213 (1996)
Schwartz, J.T.: Fast probabilistic algorithms for verification of polynomial identities. Journal of the ACM 27(4), 701–717 (1980)
Sudan, M., Trevisan, L., Vadhan, S.: Pseudorandom generators without the XOR lemma. JCSS: Journal of Computer and System Sciences 62 (2001)
Shpilka, A., Wigderson, A.: Depth-3 arithmetic circuits over fields of characteristic zero. CMPCMPL: Computational Complexity 10 (2001)
Zippel, R.: Probabilistic algorithms for sparse polynomials. In: Ng, K.W. (ed.) EUROSAM 1979 and ISSAC 1979. LNCS, vol. 72. Springer, Heidelberg (1979)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Klivans, A.R., Shpilka, A. (2003). Learning Arithmetic Circuits via Partial Derivatives. In: Schölkopf, B., Warmuth, M.K. (eds) Learning Theory and Kernel Machines. Lecture Notes in Computer Science(), vol 2777. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45167-9_34
Download citation
DOI: https://doi.org/10.1007/978-3-540-45167-9_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40720-1
Online ISBN: 978-3-540-45167-9
eBook Packages: Springer Book Archive