Abstract
We address the problem of online de-noising a stream of input points. We assume that the clean data is embedded in a linear subspace. We present two online algorithms for tracking subspaces and, as a consequence, de-noising. We also describe two regularization schemas which improve the resistance to noise. We analyze the algorithms in the loss bound model, and specify some of their properties. Preliminary simulations illustrate the usefulness of our algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Delmas, J.P., Cardoso, J.F.: Performance analysis of an adaptive algorithm for tracking dominant subspaces. IEEE Transactions on Signal Processing 46, 3054–3057 (1998)
Kumaresanand, R., Tufts, D.W.: Estimating the angles of arrival of multiple plane waves. IEEE Transactions on Aerospace and Electronic Systems 19, 134–139 (1983)
Haimovich, A.M., Bar-Ness, Y.: An eigenanalysis interference canceler. IEEE Transactions on Signal Processing 39, 76–84 (1991)
Kivinen, J., Warmuth, M.K., Hassibi, B.: The p-norm generalization of the lms algorithm for adaptive filtering. In: Proc. 13th IFAC Symposium on System Identification (2003)
Sayed, A.H.: Fundementals of Adaptive Filtering. Wiley-Interscience, Chichester (2003)
Tax, D.M.J.: One-class classification; Concept-learning in the absence of counter-examples. PhD thesis, Delft University of Technology (2001)
Zou, H., Hastie, T., Tibshirani, R.: Sparse principal component analysis (2004)
Weinberger, K.Q., Blitzer, J.C., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. In: NIPS 18 (2005)
Shalev-Shwartz, S., Singer, Y., Ng, A.Y.: Online and batch learning of pseudo-metrics. In: Proc. of the 21st international conf. on Machine learning, p. 94. ACM Press, New York (2004)
Tsuda, K., Rätsch, G., Warmuth, M.K.: Matrix exponentiated gradient updates for on-line learning and bregman projection. Jornal of Machine Learning Research 6, 995–1018 (2005)
Golub, G.H., Van Loan, C.F.: Matrix computations. John Hopkins University Press, Baltimore (1989)
Kivinen, J., Helmbold, D.P., Warmuth, M.: Relative loss bounds for single neurons. IEEE Transactions on Neural Networks 10(6), 1291–1304 (1999)
Crammer, K., Dekel, O., Shalev-Shwartz, S., Singer, Y.: Online passive aggressive algorithms. Advances in Neural Information Processing Systems 16 (2003)
Censor, Y., Zenios, S.A.: Parallel Optimization: Theory, Algorithms, and Applications. Oxford University Press, New York (1997)
Crammer, K., Singer, Y.: On the learnability and design of output codes for multiclass problems. Machine Learning 47 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Crammer, K. (2006). Online Tracking of Linear Subspaces. In: Lugosi, G., Simon, H.U. (eds) Learning Theory. COLT 2006. Lecture Notes in Computer Science(), vol 4005. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11776420_33
Download citation
DOI: https://doi.org/10.1007/11776420_33
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-35294-5
Online ISBN: 978-3-540-35296-9
eBook Packages: Computer ScienceComputer Science (R0)