Collaborative learning for hyperspectral image classification

Abstract

Recently, collaborative learning (CL) is introduced to combine active learning (AL) with semi-supervised learning (SSL), and solve the problem of limited training samples. In this paper, we proposed a novel CL framework for hyperspectral image classification, in which AL and SSL are collaboratively integrated using clustering (CLUC). CLUC attempts to obtain more diversity and higher confidence of additional training samples in both AL and SSL. Note that clustering methods, which are used separately to enhance AL or SSL, are utilized to integrate these two learning process in CLUC. First, all unlabeled samples are assigned into clusters. Based on the clustering result, clustering-based query (CBQ) for both AL and SSL, and CBQ-based pseudo-labeling (CBQPL) for SSL are designed for CLUC. Second, the most and secondary uncertain samples in each cluster are selected by CBQ for AL and SSL, respectively, to ensure their informativeness. Third, CBQPL assigns the selected secondary uncertain samples with the same label as the most uncertain one, which is manually-labeled in AL within the same cluster. CBQPL makes the confidence of pseudo-labeling rely on the clustering results. We evaluate the performance of CLUC on three real hyperspectral images. The performance of the proposed method is tested under different numbers of labeled samples and compared with several approaches. We can observe from the experimental results that CLUC have superiority in classification maps and objective metrics with limited training samples.

Introduction

Hyperspectral image (HSI) often consists of rich spectral-spatial information. HSI classification is one of the main challenges in the interpretation of remote sensing data, which has been widely used in various applications, such as precision agriculture [1], target detection [2], urban planning [3], and land-cover identification [4], [5]. Given a set of observations (i.e., pixel vectors in a hyperspectral image), the goal of classification is to assign a unique label to each pixel vector so that it is well-defined by a given class [6]. Various supervised machine learning algorithms have been widely used in HSI classification, e.g., multinomial logistic regression (MLR) [7], support vector machines (SVMs) [8], artificial neural networks(ANN) [9], multiple feature learning [10], [11], sparse representation [12], [13], and so on. However, HSI classification often suffers from the problem of limited training samples, and the manually-labeling is time-consuming and expensive. Recently, some advanced machine learning techniques, i.e., active learning (AL) [14], semi-supervised learning (SSL) [15], and spectral-spatial classification [16], [17], have been widely developed to solve this problem.

AL involves an iterative learning process of human-machine interaction. It starts with a small-size set of training set, and then iteratively selects the newly-added training samples from the unlabeled samples, which are subsequently labeled by human. The research of AL focuses on the query strategy to rank the most informative training samples for the current classifier [18], [19]. These strategies are grouped into three main types: committee-based [14], [20], [21], large margin-based [22], and posterior probability-based [23] query heuristics. In contrast to AL, SSL leverages the unlabeled data together with the initial training set for the training of the classifier. It attempts to exploit the structural information in the feature space to facilitate the learning process without additional human efforts. The SSL methods are grouped into the following categories: self-learning [24], [25], co-training [26], generative probabilistic models [27], semi-supervised SVMs [28], and graph-based SSL [29].

AL and SSL approaches work based on different mechanisms, but both aim to provide promising classification accuracy and alleviate the cost of human efforts. Thus, it is reasonable to combine the AL an SSL learning process. Recently, some methods combining AL and SSL have been developed and demonstrated promising performance [24], [30], [31], [32], [33]. Very recently, AL and SSL have been integrated as a collaborative learning process, in which the selected unlabeled samples are collaboratively labeled by the human experts and machines/classifiers. In [34], collaborative active and semi-supervised learning (CASSL) acquires more confidently labeled samples by integrating AL and SSL. In [35], a random-walker-based verification strategy separates the unlabeled samples into the low-confident and high-confident data sets, and manually-labeled and pseudo-labeled samples are used together to train the final classifier.

These collaborative learning methods present promising classification performances with limited manually-labeled samples. However, there still exist several aspects that may be improved. First, the precision of pseudo-labeling relies on the current classification results and the strategy to select the most confident samples. Second, for the SSL learning process, actually the most confident samples are selected and pseudo-labeled, which provides redundant information for the classifiers. Third, the manually-marked labels are just assigned to the selected unlabeled samples in AL, and we wish to make full use of these labels marked by human.

We note that, clustering has been extensively incorporated into AL or SSL separately to improve the performance of HSI classification [29], [31], [36], [37], [38], [39], [40], [41], [42]. Considering only image statistics, clustering algorithms automatically partition the image into homogeneous and coherent groups. In AL, the classifier is retrained with new training set, and it is promised to select two or more samples at each AL iteration. These AL query strategies, such as margin sampling (MS) [43], Multiclass level uncertainty (MCLU) [36], max entropy (ME) [15], breaking ties (BT) [44], and Kullback–Leiber divergence maximization (KL-Max) [45], [46], consider only the uncertainty of unlabeled samples. And, the queried samples may be redundant to each other. Therefore, many studies have introduced clustering methods into AL to reduce the redundancy by seeking the most dissimilarity among those ranked samples [36], [37], [38], [39]. Different from AL, the unlabeled data in SSL are leveraged with the initial labeled data to train the classifier. SSL attempts to exploit the structural information in feature space to facilitate the learning process without additional manual-labeling. In [31], [40], clustering approaches are utilized to improve/construct the SSL learning process. In [29], [41], cluster/manifold regularization is included in graph-based SSL methods according to the assumption of labeling smoothness.

In this paper, we propose a novel collaborative learning framework using clustering (CLUC), in which the clustering procedure plays the similar role as a `bridge' in integrating the AL and SSL learning process. First, all the unlabeled pixels of an HSI image are grouped into a set of clusters using superpixel algorithm. Recently, superpixel techniques have been developed a lot in computer vision [47], [48], [49]. And superpixel methods are combined with segmentation-based classifiers for HSI classification [50], [51], [52], [53], [54], [55]. Superpixel segmentation is virtually a clustering method considering not only feature similarity but also spatial adjacency. After superpixel segmentation, an image is segmented into non-overlapping regions/clusters. Then, clustering-based query (CBQ) heuristic is designed by combining the breaking BT query strategy with the superpixel clustering result. Second, the most and the secondary uncertain samples in each superpixel are actively selected by the CBQ strategy for the AL and SSL learning process, respectively, to ensure their informativeness for the classifier. Third, CBQ-based pseudo-label (CBQPL) assigns the secondary uncertain unlabeled samples (in SSL) with the same label as the manually-labeled sample (in AL) within the same superpixel. And, both the manually-labeled and CBQPL-labeled samples are used to train the classifier at each iteration of CLUC.

An obvious difference of CLUC from those CL frameworks is that it aims to find the most confident samples that have the same labels with the manually-labeled samples in the CL learning process, instead of pseudo-labeling the unlabeled samples using the current classification result. This will bring several advantages in the following aspects. First, the CBQ strategy selects the most uncertain samples for both AL and SSL, which can provide more information than those aforementioned CL learning framework. Second, the designed CBQPL strategy makes the accuracy of pseudo-labeling rely on the effectiveness of clustering method, instead of the performance of classifiers. In this paper, using effective superpixel algorithm ensures a promising CBQPL labeling process. Additionally, based on the superpixel segmentation, the CBQ heuristic considers the diversity of active sampling process in both feature and spatial domains. Therefore, CBQ promotes more uncertainty of queried samples in both AL and SSL learning processes. Moreover, the CBQ strategy within each cluster is still biased sampling, which partly promotes the accuracy of CBQPL. Mutinomial logistic regression (MLR) is used to model the posterior probability distributions at each CLUC iteration. The variable splitting and augmented Lagrangian (LORSAL) [56] algorithm, followed by a multi-level logistic (MLL) [15] regularization, i.e., the LMLL method, is adopted as the basic classifier in this paper. The performance of CLUC is tested on three real hyperspectral images. Experimental results demonstrate that the proposed CLUC framework can achieve competitive classification performance with limited manually-labeled training samples.

The rest of this paper is organized as follows. Section 2 reviews the related work. Section 3 presents a detailed description of the proposed CLUC method. The experimental results of CLUC are given in Section 4. Section 5 concludes this paper and suggests some future work.