Cross domain boosting for information fusion in heterogeneous sensor-cyber sources

Highlights

• We propose Cross Domain Boosting (CD-Boost) to fuse information in HSCS.

• The weighted boosting loss objective of CD-Boost can fuse information by assigning different weights to each source sample.

• The manifold smoothness regularization of CD-Boost can prevent overfitting and further improve the accuracy.

Abstract

Image classification is an active research area in signal processing and pattern recognition, with extensively academic and industrial applications. Existing image classification methods usually assumes consistent image source for training and testing which, however, seldom holds for practical applications. We define the scenario of images being captured from diverse sources (e.g. Internet, surveillance cameras, and mobile phones) as heterogeneous sensor-cyber sources (HSCS). We indicate that, information fusion in HSCS is able to make full use of the diversity of images, and thus improves the generalization of classifier. A novel algorithm named Cross Domain Boosting (CD-Boost) is proposed to fuse information in HSCS. Our CD-Boost algorithm has two characteristics, weighted loss objective and manifold smoothness regularization. Concretely, the weighted boosting loss objective can fuse information by assigning different weights to each source sample, and the weights are determined by minimizing the difference between the data distribution in HSCS. Furthermore, when the images are scarce, a manifold smoothness regularization can prevent overfitting and further improve the accuracy. The experimental results on real data demonstrate that our algorithm outperforms existing methods.

Introduction

Image classification has long attracted intensive interest in signal processing and pattern recognition, because its results are the basis for web content analysis, image retrieval, human–computer interaction, and biometrics. The first step of training an image classifier is to collect images that are usually captured from heterogeneous sources in real-world scenarios. They are widely obtained from webcam, digital single lens reflex camera, Internet, mobile phone or collected and annotated publicly databases, which are called heterogeneous sensor-cyber sources (HSCS) [1], [2]. Images from HSCS are captured with various hardware settings, under varying illumination and noise conditions, and post-processed with different image techniques. So they have different distributions. Nevertheless, the default assumption in many learning scenarios is that training data and testing data are drawn from the same distribution. If the model is trained and tested on HSCS, a mismatch of image distributions would result in the performance of classifier degradation sharply [3], [4], [5]. We observe that images from HSCS have a diversity characteristic and reside on different feature spaces, and by fusing them, we can obtain a more complete feature space than individual feature space. We therefore propose that training an image classifier utilizing the images from HSCS are able to improve the classifier generalization. The generalization of a classifier is useful in many real-world applications. For example, lab-trained image classifiers based on images downloaded from Internet can be applied into mobile phones in outdoor environment. Traditional methods [6], [7] need lots of training samples captured with mobile phone cameras. On one hand, however, collecting images captured from mobile phone cameras involves tremendous manual work and monetary cost; on the other hand, there are a large amount of readily available images from cyber source. Intuitively, we hope to utilize the images from cyber source to improve the generalization of classifier. Namely, we want to fuse information by utilizing images from cyber source and a small amount of images captured with mobile phone cameras, and the classifier works well on mobile camera images.

Information fusion is key to make full use of the diversity of images from HSCS, and thus improves the generalization of classifier. Many problems arise when fusing information in HSCS. One of these problems is that the same objects appear significantly different across several sources, which is caused by many factors, such as pose, illumination or image resolution. Fig. 1 shows the same set of object classes captured from heterogeneous sources. Although the images from HSCS appear significantly different, the relationship among heterogeneous sources is that these images reside on an intrinsically low dimension manifold in the high dimension feature space [8]. Thus, it is reasonable to believe that information fusion of images from HSCS can improve the generalization of classifier. What we need is a well defined information fusion principle so that make full use of the diversity of images from HSCS.

Fortunately, the above challenge is considered by the techniques of domain adaptation or covariate shift which involve heterogeneous sources, training samples in the source domain and testing samples in the target domain. A fruitful line of work has been proposed for this task [9], [10], [11]. Some have focused on learning adapted classifier parameters by minimizing classification error using a small amount of labeled samples in the target domain [12], [13]. Other methods derive new feature representations [14], [15] or learn a transformation between the features in the various domains [16], [17].

In this paper, we propose a novel Cross Domain Boosting (CD-Boost) algorithm for information fusion in HSCS. The images from HSCS can be viewed as data drawn from different distributions, and by matching the different distributions, each of the samples can be assigned a weight which represents its probability ratio of belonging to target and source sample distribution. The weight for each sample can help classifier fuse the information of HSCS. This inspires a weighted boosting objective. Furthermore, the lack of sufficient target samples may result in overfitting problem. We therefore design a manifold smoothness regularization item, which enforces the smoothness in local data manifold, into the optimization objective. These two characteristics constitute the proposed CD-Boost algorithm. The proposed algorithm is verified on the images captured from HSCS, and the experimental results clearly demonstrate the effectiveness of the proposed CD-Boost algorithm.

The remainder of this paper is organized as follows: in Section 2, we introduce related work; in Section 3, the CD-Boost algorithm is presented in detail; the experimental results are given in Section 4, and finally in Section 5 we conclude this paper.