Abstract
Stochastic online learning algorithms typically exhibit slow convergence speed, but their solutions of moderate accuracy often suffice in practice. Since the outcomes of these algorithms are random variables, not only their accuracy but also their probability of achieving a certain accuracy, called confidence, is important. We show that a rather simple aggregation of outcomes from parallel dual averaging runs can provide a solution with improved confidence, and it can be controlled by the number of runs, independently of the length of learning processes.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hoeffding, W.: Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association 58(301), 13–30 (1963)
Lee, S., Wright, S.: Manifold identification of dual averaging methods for regularized stochastic online learning. In: Proceedings of the 28th International Conference on Machine Learning, pp. 1121–1128 (2011)
Nesterov, Y.: Primal-dual subgradient methods for convex problems. Mathematical Programming 120, 221–259 (2009)
Rockafellar, R.T., Wets, R.J.B.: On the interchange of subdifferentiation and conditional expectation for convex functionals. Stochastics 7(3), 173–182 (1982)
Xiao, L.: Dual averaging methods for regularized stochastic learning and online optimization. Journal of Machine Learning Research 11, 2543–2596 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lee, S. (2012). Improving Confidence of Dual Averaging Stochastic Online Learning via Aggregation. In: Glimm, B., Krüger, A. (eds) KI 2012: Advances in Artificial Intelligence. KI 2012. Lecture Notes in Computer Science(), vol 7526. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33347-7_20
Download citation
DOI: https://doi.org/10.1007/978-3-642-33347-7_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33346-0
Online ISBN: 978-3-642-33347-7
eBook Packages: Computer ScienceComputer Science (R0)