Coupled compressed sensing inspired sparse spatial-spectral LSSVM for hyperspectral image classification
Introduction
The ample information contained in the hyperspectral data can be used to classify classes of the same species, making the Hyperspectral Image Classification (HIC) very attractive in recent years. Among the supervised HIC approaches, Support Vector Machine (SVM) obtains excellent classification result, because it makes a tradeoff between bias and variance via using a compact topology which is established by a small number of carefully selected support vectors [1]. SVM can handle large input spaces efficiently, work with a relatively small number of labeled training samples and deal with noisy samples in a robust way [2], [3]. And various studies are presented to further improve the performance of SVM in recent years. Firstly, multiple-output support vector regression (MSVR) model is designed for addressing the multi-variate application problems [4], [5]. And Fuzzy SVM model is also proposed for efficiently dealing with outliers or noises in classification [6]. Furthermore, due to the importance of the parameters in SVM, many effective parameters optimization methods of SVM are proposed in recent years. The most popular way among them is the grid search, which exhaustively searches on the parameters space for the validation error minimization [7], [8]. Besides grid search, hybrid algorithm such as hybrid comprehensive learning particle swarm optimizer with Broyden–Fletcher–Goldfarb–Shanno (CLPSO–BFGS) algorithm [9], memetic algorithm such as particle swarm optimization and pattern search (PSO–PS) based memetic algorithm [10], and firefly algorithm based methods [4], [11] are proposed to tune the parameters of SVM [9], [10], support vector regression (SVR) [11] and MSVR [4] respectively. These methods greatly improve the stability of the parameter settings, and thus the SVM tuning parameters by them can obtain good generalization performances. Therefore, SVM is well-suited for HIC and has demonstrated excellent performance in terms of accuracy and robustness in recent years [12], [13]. Furthermore, SVM methods for HIC were used in many areas, such as prostate cancer detection [14], land cover classification [15] and plant diseases detection [16], and have obtained good effects.
Although the generalization ability of SVM is very well, the computational complexity of SVM is high, for it solves a set of inequalities constrained quadratic programming. Least Square Support Vector Machine (LS-SVM) is a modified version of the standard SVM, which replaces the inequality constraints with equality constraints when solving quadratic programming [17]. Therefore, LS-SVM is more computationally attractive than SVM and has been applied to HIC [18], [19]. However, LS-SVM does not perform model selection and loses the sparseness of SVM, so the storage cost, computation cost and prediction error of generalization all increase when it is used in HIC. Even though many pruning algorithms have been developed to impose the sparseness on original LS-SVM [20], [21], these approaches need to iteratively omit the training samples and retrain the reduced LS-SVM. Several works aim for alleviating this problem by imposing sparsity on LS-SVM [22], [23]. These techniques are capable of iteratively constructing a sparse LS-SVM when training. However, the iterative training cost is prohibitive. Moreover, the convergence of these algorithms are dependent on the success of optimization algorithms.
In this paper, inspired by the recently developed Compressed Sensing (CS) theory [24], [25], a compact LS-SVM with sparse topology is established to realize accurate classification of hyperspectral images. This sparse model can be considered as a low-dimensional measurement of the original LS-SVM. The sparse topology can be then obtained by learning a compressive measurement matrix from training data and then reducing the useless support vectors by solving a Multiple Measurement Vector (MMV) optimization problem via CS technology. On the other hand, one can observe that neighboring hyperspectral pixels likely belong to the same class. That is, there is spatial homogeneity in the labels of hyperspectral images, which is beneficial for classifying hyperspectral images. Therefore, incorporating spatial information into the classification can improve the classification accuracy [26], [27], [28]. In paper [26], [27], [28], the spatial neighbors set of a given hyperspectral pixel is defined as a small window centered on the given pixel. However, for the pixels lying on the edges of numerous classes, this spatial homogeneity assumption is invalid. In the tiny window, there are some noisy pixels which do not belong to the same class of the center pixel. Taking these noisy pixels as spatial neighbors of the center pixel will involve noisy information in the hyperspectral pixels and decrease the classification accuracy of hyperspectral image. In our work, the LS-SVM is regularized by casting a local adaptive spatial homogeneity assumption on hyperspectral images. The hyperspectral pixel is redefined both in the spectral domain and spatial domain by adaptively selecting its spatial neighbors according to the edge-map. The weighted sum of spectral and spatial features is utilized to construct a Spatial-Spectral Least Square Support Vector Machine (SS-LSSVM) model in this study. And then a Coupled Compressed Sensing inspired Sparse SS-LSSVM (CCS4-LSSVM) for HIC is advanced. By combining spectral information and adaptively extracted spatial information together, CCS4-LSSVM cannot only avoid the speckle-like misclassification of original LS-SVM but also reduce the influence of noisy pixels.
Compared with the available HIC approaches, our proposed CCS4-LSSVM has the following characteristics: (1) CCS4-LSSVM is more computationally attractive than SVM, for it solves a linear system instead of a quadratic programming. (2) The performance of CCS4-LSSVM for HIC is comparable with SVM due to its sparse topology. (3) CCS4-LSSVM is constructed via a one-step strategy and the designed compressive measurement matrix coupled with the dictionary matrix guarantees the high incoherence with dictionary, which avoids the iterative selection of important support vectors and makes a rapid and high-accuracy HIC possible. (4) By combining spectral information with the adaptively extracted spatial neighbors together, CCS4-LSSVM can avoid the influence of noisy pixels and the speckle-like misclassification of the original LS-SVM. Some experiments are conducted on several hyperspectral data to compare the proposed method with its counterparts, and the results show that it can achieve higher classification accuracy than Spatial-Spectral SVM (SS-SVM) and Spatial-Spectral LSSVM (SS-LSSVM).
The remainder of the paper is organized as follows: Section 2 depicts the proposed CCS4-LSSVM. In Section 3, some experiments are conducted to investigate the performance of our proposed method. A conclusion is presented in Section 4.
Section snippets
Coupled Compressed Sensing inspired Sparse Spatial-Spectral Least Square Support Vector Machine (CCS4-LSSVM) for HIC
CS provides a new information acquisition and processing framework that allows us to reconstruct sparse or compressible signals from a small set of measurements. Assume that a set of signals is compressible under a dictionary , where is the number of rows that contain nonzero elements. Many applications matching the properties of CS involve distribution acquisition of multiple correlated signals. The multiple signal case where all l involved signals are
Experimental results and discussions
In this section, the performance of our proposed CCS4-LSSVM is evaluated on three hyperspectral image data downloaded from http://www.ehu.es/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes.
Conclusions
In order to achieve accurate and rapid classification for hyperspectral images with a small number of training data, a Coupled Compressed Sensing inspired Sparse Spatial-Spectral Least Squares Support Vector Machine (CCS4-LSSVM) is proposed based on recently developed CS theory. By casting a sparse assumption on the support vectors, CCS4-LSSVM can derive a sparse topology from a coupled compressive matrix stemmed from the dictionary and obtained by MMV optimization algorithm. Experimental
Acknowledgements
This work was supported by the National Basic Research Program of China (973 Program) under Grant No. 2013CB329402, the Fundamental Research Funds for the Central Universities BDY021429, the Huawei Innovation Research Program, the Kunshan innovation institute of Xidian University, National Science Foundation of China under Grant Nos. 91438103, 91438201, 61072108, 61173090, 51207002, NCET-10-668, the Foreign Scholars in University Research and Teaching Programs (No. B07048), the fundamental
References (40)
- et al.
Multiple-output support vector regression with a firefly algorithm for interval-valued stock price index forecasting
Knowl.-Based Syst.
(2014) - et al.
Multi-step-ahead time series prediction using multiple-output support vector regression
Neurocomputing
(2014) - et al.
Tuning SVM parameters by using a hybrid CLPSO–BFGS algorithm
Neurocomputing
(2010) - et al.
A PSO and pattern search based memetic algorithm for SVMs parameters optimization
Neurocomputing
(2013) - et al.
A kernel functions analysis for support vector machines for land cover classification
Int. J. Appl. Earth Obs.
(2009) - et al.
Early detection and classification of plant diseases with support vector machines based on hyperspectral reflectance
Comput. Electron. Agric.
(2010) - et al.
Improved sparse least-squares support vector machine
Neurocomputing
(2006) - et al.
Beyond one-step-ahead forecasting: evaluation of alternative multi-step-ahead forecasting models for crude oil price
Energy Econ.
(2013) - et al.
Does restraining end effect matter in EMD-based modeling framework for time series prediction? Some experimental evidences
Neurocomputing
(2014) - et al.
Interval forecasting of electricity demand: a novel bivariate EMD-based support vector regression modeling framework
Int. J. Electr. Power Energy Syst.
(2014)
Classification of hyperspectral remote sensing images with support vector machines
IEEE Trans. Geosci. Remote Sens.
An Introduction to Support Vector Machines
Robust support vector method for hyperspectral data classification and knowledge discovery
IEEE Trans. Geosci. Remote Sens.
A kernel fuzzy c-means clustering-based fuzzy support vector machine algorithm for classification problems with outliers or noises
IEEE Trans. Fuzzy Syst.
Model selection for primal SVM
Mach. Learn.
Electricity load forecasting using support vector regression with memetic algorithms
Sci. World J.
Multi-channel morphological profiles for classification of hyperspectral images using support vector machines
Sensors
A spatial-contextual support vector machine for remotely sensed image classification
IEEE Trans. Geosci. Remote Sens.
Hyperspectral imaging and quantitative analysis for prostate cancer detection
J. Biomed. Opt.
Cited by (43)
Monitoring and detecting coal miners' fatigue status using MPA-LSSVM in the vision of smart mine
2023, Process Safety and Environmental ProtectionA large-scale hyperspectral dataset for flower classification
2022, Knowledge-Based SystemsCitation Excerpt :Compared with RGB images, hyperspectral images (HSI) of the natural scene can effectively describe the spectral distribution and provide intrinsic and discriminative spectral information of the scene [14–21]. The rich spectral details in HSI can show the deterministic information about the lighting and material, which is beneficial to classification [22–24], object detection [25] and material analysis [26,27]. However, most of the existing hyperspectral image datasets are not suitable for classification.
Hyperspectral image classification based on discriminative locality preserving broad learning system
2020, Knowledge-Based SystemsA feature selection approach for hyperspectral image based on modified ant lion optimizer
2019, Knowledge-Based SystemsImproving deep ensemble vehicle classification by using selected adversarial samples
2018, Knowledge-Based Systems