Abstract
There are two main families of on-line algorithms depending on whether a relative entropy or a squared Euclidean distance is used as a regularizer. The difference between the two families can be dramatic. The question is whether one can always achieve comparable performance by replacing the relative entropy regularization by the squared Euclidean distance plus additional linear constraints. We formulate a simple open problem along these lines for the case of learning disjunctions.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Khardon, R., Roth, D., Servedio, R.: Efficiency versus convergence of Boolean kernels for on-line learning algorithms. In: Advances in Neural Information Processing Systems 14, pp. 423–430. MIT Press, Cambridge (2001)
Kivinen, J., Warmuth, M.K.: Additive versus exponentiated gradient updates for linear prediction. Information and Computation 132(1), 1–64 (1997)
Kivinen, J., Warmuth, M.K., Auer, P.: The perceptron algorithm vs. winnow: linear vs. logarithmic mistake bounds when few input variables are relevant. Artificial Intelligence 97, 325–343 (1997)
Littlestone, N.: Learning when irrelevant attributes abound: A new linear-threshold algorithm. Machine Learning 2, 285–318 (1988)
Long, P.M., Wu, X.: Mistake bounds for maximum entropy discrimination. In: Advances in Neural Information Processing Systems, vol. 17, MIT Press, Cambridge (December 2004)
Warmuth, M.K., Vishwanathan, S.V.N.: Leaving the span. In: Auer, P., Meir, R. (eds.) COLT 2005. LNCS, vol. 3559, pp. 366–381. Springer, Heidelberg (2005); A longer journal version is in preperation
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Warmuth, M.K. (2006). Can Entropic Regularization Be Replaced by Squared Euclidean Distance Plus Additional Linear Constraints. In: Lugosi, G., Simon, H.U. (eds) Learning Theory. COLT 2006. Lecture Notes in Computer Science(), vol 4005. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11776420_48
Download citation
DOI: https://doi.org/10.1007/11776420_48
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-35294-5
Online ISBN: 978-3-540-35296-9
eBook Packages: Computer ScienceComputer Science (R0)