Abstract
Given a set of vectors F={f 1,…,f m } in a Hilbert space \(\mathcal {H}\), and given a family \(\mathcal {C}\) of closed subspaces of \(\mathcal {H}\), the subspace clustering problem consists in finding a union of subspaces in \(\mathcal {C}\) that best approximates (is nearest to) the data F. This problem has applications to and connections with many areas of mathematics, computer science and engineering, such as Generalized Principal Component Analysis (GPCA), learning theory, compressed sensing, and sampling with finite rate of innovation. In this paper, we characterize families of subspaces \(\mathcal {C}\) for which such a best approximation exists. In finite dimensions the characterization is in terms of the convex hull of an augmented set \(\mathcal {C}^{+}\). In infinite dimensions, however, the characterization is in terms of a new but related notion; that of contact half-spaces. As an application, the existence of best approximations from π(G)-invariant families \(\mathcal {C}\) of unitary representations of Abelian groups is derived.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
A. Aldroubi, C. Cabrelli, D. Hardin, U. Molter, Optimal shift-invariant spaces and their Parseval frame generators. Appl. Comput. Harmon. Anal. 273–283 (2007).
A. Aldroubi, C. Cabrelli, U. Molter, Optimal nonlinear models for sparsity and sampling. J. Fourier Anal. Appl. 14, 793–812 (2008).
A. Aldroubi, K. Gröchenig, Non-uniform sampling in shift-invariant spaces. SIAM Rev. 43, 585–620 (2001).
A. Aldroubi, A. Sekmen, Reduction and null space algorithms for the subspace clustering problem (2010, submitted). arXiv:1010.2198.
M. Bownik, The structure of shift-invariant subspaces of L 2(ℝn). J. Funct. Anal. 177, 282–309 (2000).
P. Bradley, O. Mangasarian, k-plane clustering. J. Glob. Optim. 16, 23–32 (2000).
G. Chen, G. Lerman, Foundations of a multi-way spectral clustering framework for hybrid linear modeling. Found. Comput. Math. 9, 517–558 (2009).
E. Candès, J. Romberg, T. Tao, Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52, 489–509 (2006).
J. Costeira, T. Kanade, A multibody factorization method for independently moving objects. Int. J. Comput. Vis. 29, 159–179 (1998).
J. Dixmier, Les C*-algèbres et leurs représentations (Gauthier-Villars, Paris, 1969).
P.L. Dragotti, M. Vetterli, T. Blu, Sampling moments and reconstructing signals of finite rate of innovation: Shannon meets Strang-Fix. IEEE Trans. Signal Process. 55, 1741–1757 (2007).
C. Eckart, G. Young, The approximation of one matrix by another of lower rank. Psychometrica 1, 211–218 (1936).
R. Gribonval, M. Nielsen, Sparse decompositions in unions of bases. IEEE Trans. Inf. Theory 49, 3320–3325 (2003).
D. Han, D.R. Larson, Wandering vector multipliers for unitary groups. Trans. Am. Math. Soc. 353, 3347–3370 (2001).
D. Han, D.R. Larson, Frames, Bases and, Group Representations. Mem. Am. Math. Soc., vol. 147 (697) (2000).
D. Han, Frame representations and Parseval duals with applications to Gabor frames. Trans. Am. Math. Soc. 360, 3307–3326 (2008).
E. Hernandez, D. Labate, G. Weiss, A unified characterization of reproducing systems generated by a finite family, II. J. Geom. Anal. 12, 615–662 (2002).
J. Ho, M. Yang, J. Lim, K. Lee, D. Kriegman, Clustering appearances of objects under varying illumination conditions. In Proceedings of International Conference on Computer Vision and Pattern Recognition, vol. 1 (2003), pp. 11–18.
T. Kanade, D.D. Morris, Factorization methods for structure from motion. Philos. Trans. R. Soc. Lond. A 356, 1153–1173 (1998).
K. Kanatani, Y. Sugaya, Multi-stage optimization for multi-body motion segmentation. IEICE Trans. Inf. Syst. 335–349 (2003).
G. Kutyniok, D. Labate, The theory of reproducing systems on locally compact Abelian groups. Colloq. Math. 106, 197–220 (2006).
G. Lerman, T. Zhang, Probabilistic recovery of multiple subspaces in point clouds by geometric ℓ p minimization. Preprint (2010).
Y. Lu, M.N. Do, A theory for sampling signals from a union of subspaces. IEEE Trans. Signal Process. 56, 2334–2345 (2008).
Y. Ma, A.Y. Yang, H. Derksen, R. Fossum, Estimation of subspace arrangements with applications in modeling and segmenting mixed data. SIAM Rev. 50, 413–458 (2008).
I. Maravic, M. Vetterli, Sampling and reconstruction of signals with finite rate of innovation in the presence of noise. IEEE Trans. Signal Process. 53, 2788–2805 (2005).
M. Mishali, Y.C. Eldar, Blind multi-band signal reconstruction: compressed sensing for analog signals. IEEE Trans. Signal Process. 57, 993–1009 (2009).
H. Rauhut, K. Schnass, P. Vandergheynst, Compressed sensing and redundant dictionaries. IEEE Trans. Inf. Theory 54, 2210–2219 (2008).
R. Vidal, Y. Ma, S. Sastry, Generalized principal component analysis (GPCA). IEEE Trans. Pattern Anal. Mach. Intell. 27, 1–15 (2005).
J. Yan, M. Pollefeys, A general framework for motion segmentation: independent, articulated, rigid, non-rigid, degenerate and nondegenerate. In ECCV, vol. 4 (2006), pp. 94–106.
T. Zhang, A. Szlam, Y. Wang, G. Lerman, Randomized Hybrid Linear Modeling by Local Best-Fit Flats. CVPR, San Francisco (2010).
S. Smale, D.X. Zhou, Shannon sampling II. Connections to learning theory. Appl. Comput. Harmon. Anal. 19, 285–302 (2005).
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Wolfgang Dahmen.
The research of A. Aldroubi is supported in part by NSF Grant DMS-0807464.
Rights and permissions
About this article
Cite this article
Aldroubi, A., Tessera, R. On the Existence of Optimal Unions of Subspaces for Data Modeling and Clustering. Found Comput Math 11, 363–379 (2011). https://doi.org/10.1007/s10208-011-9086-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10208-011-9086-4