Abstract
We consider the following sparse representation problem: represent a given matrix X∈ℝm×N as a multiplication X=AS of two matrices A∈ℝm×n (m≤n<N) and S∈ℝn×N, under requirements that all m×m submatrices of A are nonsingular, and S is sparse in sense that each column of S has at least n−m+1 zero elements. It is known that under some mild additional assumptions, such representation is unique, up to scaling and permutation of the rows of S. We show that finding A (which is the most difficult part of such representation) can be reduced to a hyperplane clustering problem. We present a bilinear algorithm for such clustering, which is robust to outliers. A computer simulation example is presented showing the robustness of our algorithm.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alekseev, V.M., Tihomirov, V.M., Fomin, S.V.: Optimal Control. Nauka, Moscow (1979) (in Russian)
Bradley, P.S., Mangasarian, O.L.: k-Plane clustering. J. Glob. Optim. 16(1), 23–32 (2000)
Bradley, P.S., Mangasarian, O.L., Street, W.N.: Clustering via concave minimization. In: Advances in Neural Information Processing Systems, vol. 9, pp. 368–374. MIT Press, Cambridge (1997). See also Mathematical Programming Technical report 96-03 (May 1996)
Chen, S., Donoho, D., Saunders, M.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20(1), 33–61 (1998)
Cichocki, A., Amari, S.: Adaptive Blind Signal and Image Processing. Wiley, Chichester (2002)
Donoho, D., Elad, M.: Optimally sparse representation in general (nonorthogonal) dictionaries via l 1 minimization. Proc. Natl. Acad. Sci. U.S.A. 100(5), 2197–2202 (2003)
Georgiev, P., Theis, F., Cichocki, A.: Sparse component analysis and blind source separation of underdetermined mixtures. IEEE Trans. Neural Netw. 16(4), 992–996 (2005)
Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. Wiley, New York (2001)
Lee, T.-W., Lewicki, M.S., Girolami, M., Sejnowski, T.J.: Blind source separation of more sources than mixtures using overcomplete representaitons. IEEE Signal Process. Lett. 6(4), 87–90 (1999)
Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 40, 788–791 (1999)
Rubinov, A.M., Ugon, J.: Skeletons of finite sets of points. Research working paper 03/06, School of Information Technology & Mathematical Sciences, University of Ballarat (2003)
Theis, F.J., Lang, E.W., Puntonet, C.G.: A geometric algorithm for overcomplete linear ICA. Neurocomputing 56, 381–398 (2004)
Waheed, K., Salem, F.: Algebraic overcomplete independent component analysis. In: Proceedings of International Conference ICA 2003, Nara, Japan, pp. 1077–1082
Zibulevsky, M., Pearlmutter, B.A.: Blind source separation by sparse decomposition in a signal dictionary. Neural Comput. 13(4), 863–882 (2001)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Georgiev, P., Pardalos, P. & Theis, F. A bilinear algorithm for sparse representations. Comput Optim Appl 38, 249–259 (2007). https://doi.org/10.1007/s10589-007-9043-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-007-9043-y