Unsupervised feature selection based on joint spectral learning and general sparse regression

Abstract

Unsupervised feature selection is an important machine learning task since the manual annotated data are dramatically expensive to obtain and therefore very limited. However, due to the existence of noise and outliers in different data samples, feature selection without the help of discriminant information embedded in the annotated data is quite challenging. To relieve these limitations, we investigate the embedding of spectral learning into a general sparse regression framework for unsupervised feature selection. Generally, the proposed general spectral sparse regression (GSSR) method handles the outlier features by learning the joint sparsity and the noisy features by preserving the local structures of data, jointly. Specifically, GSSR is conducted in two stages. First, the classic sparse dictionary learning method is used to build the bases of original data. After that, the original data are project to the basis space by learning a new representation via GSSR. In GSSR, robust loss function $\ell _{2,r}{-}{norm}(0<r\le 2)$ and $\ell _{2,p}-{norm}(0<p\le 1)$ instead of the traditional F norm and least square loss function are simultaneously considered as the reconstruction term and sparse regularization term for sparse regression. Furthermore, the local topological structures of the new representations are preserved by spectral learning based on the Laplacian term. The overall objective function in GSSR is optimized and proved to be converging. Experimental results on several publicly datasets have demonstrated the validity of our algorithm, which outperformed the state-of-the-art feature selections in terms of classification performance.