Unsupervised feature selection via discrete spectral clustering and feature weights

Abstract

Most of the existing unsupervised feature selection methods learn the cluster structure through spectral clustering, and then use various regression models to introduce the data matrix into the indicator matrix to obtain feature selection matrix. In these methods, the clustering indicator matrix is usually continuous value, which is not the best choice for the matrix in terms of its supervising role in feature selection. Based on this, unsupervised feature selection via discrete spectral clustering and feature weights (FSDSC) is proposed in this paper. First, FSDSC integrates regression model and spectral clustering in a unified framework for feature selection, and introduces a feature weight matrix, which intuitively expresses the importance of each feature with its diagonal elements. Compared with the common feature selection matrix that requires constraints such as sparse regular items, the appearance of the feature weight matrix reduces the complexity of the model and simplifies the calculation process of feature evaluation. Secondly, for the value of the indicators matrix, the spectral clustering is improved to obtain a discrete clustering indicator matrix, which provides clearer guidance information for feature selection. Finally, in order to avoid trivial solutions, the transformation matrix is constrained by orthogonal constraint. The combination of the orthogonal regression model and spectral clustering enables the algorithm to perform feature selection and manifold information learning at the same time, thereby preserving the local geometric structure of data. Compared with other excellent unsupervised feature selection algorithms, the experimental results prove the effectiveness of the proposed algorithm.

Introduction

With the rapid development of information technology, a large amount of high-dimensional data has been generated in various fields. These high-dimensional data always contain some noise and redundant features, which increase the difficulty of data processing [1]. The research in the fields of machine learning, image processing, data mining [2], and pattern recognition [3] is used to process the high-dimensional data. It is necessary to reduce the dimensionality of high-dimensional data [4]. Dimension reduction can not only decrease the time cost of calculation and improve calculation efficiency, but also reduce the space pressure of calculation. Feature extraction [5] and feature selection [6] are two common dimensionality reduction methods. Feature extraction converts all features into fewer new features to replace the original features [7]. Feature selection selects some representative features to form a subset according to a certain standard, to obtain a compressed data representation [8]. In addition, it can retain the semantic information of data [9] and has stronger interpretability [10]. Nowadays, feature selection is constantly developing [11]. However, most of the high-dimensional data in real life are unlabeled, which is difficult for direct feature selection. Therefore, how to obtain pseudo-labels that are closer to the real labels is a challenging research. This paper proposes an unsupervised feature selection method via discrete spectral clustering and feature weights (FSDSC), which uses a discrete clustering indicator matrix as a pseudo-label to provide clearer discriminative information for feature selection. At the same time, the feature subset is extracted based on the feature weight matrix. Compared with traditional methods that need to impose constraints on the feature selection matrix, such as sparse regularization item, the feature weight matrix reduces the complexity of model and reduces the calculation amount of feature evaluation. Specifically, FSDSC integrates regression models and spectral clustering in a unified framework, introduces a feature weight matrix in the framework, and then performs feature selection through this framework. The matrix is a diagonal matrix, each diagonal element of which intuitively represents the weight of each feature, so that the algorithm can easily select a subset of features. Orthogonal constraint is imposed on the transformation matrix. Compared with least square regression, orthogonal regression model can preserve more discriminative information and avoid trivial solutions. In addition, FSDSC improves the spectral clustering method to obtain a discrete indicator matrix, which provides more accurate guidance information for feature selection. The rest of this paper is organized as follows. Section 2 is mainly about the introduction of some related work. Section 3 introduces the proposed model, optimization method and convergence analysis in detail. Section 4 presents the experimental results and comparative analysis of FSDSC and the compared algorithms on the same datasets. The conclusion is summarized in Section 5.