Filtered shallow-deep feature channels for pedestrian detection

Abstract

The semantic segmentation task is highly related to detection and apparently can provide complementary information for detection. In this paper, we propose integrating deep semantic segmentation feature maps into the original pedestrian detection framework which combines feature channels with AdaBoost classifiers. Firstly, we develop shallow-deep channels by concatenating shallow hand-crafted and deep segmentation channels to capture appearance clues as well as semantic attributes. Then a set of manually designed filters are utilized on the new channels to generate more response feature maps. Finally a cascade AdaBoost classifier is learned for hard negatives selection and pedestrian detection. With abundant feature information, our proposed detector achieves superior results on Caltech USA 10x and ETH dataset.

Introduction

Pedestrian detection has been a hot research topic in the past few years [1], [2], [3], [33]. It is aiming for returning bounding boxes which enclose pedestrians of given images. A majority of works follow all or parts of the pipeline including preprocessing, foreground segmentation, object classification, refinement and tracking [30]. Recent research demonstrates that there emerges two dominant pedestrian detection methods, namely deep learning incurred convolutional neural network (CNN) [5], [6], [17] and boosted decision trees with hand crafted features [2], [3] (e.g. Histogram of Gradient (HOG), Aggregated Channel Features (ACF)). In some scenarios, the straightforward usage of the deep learning framework [6] works no better than traditional methods [2]. Inspired by the close correlation between segmentation and detection [1], [4] and recent advances in the semantic segmentation field [10], [11], [12], many researchers attempt to utilize segmentation to aid the detection task. Cadena et al. [8] apply the Support Vector Machine (SVM) classifier to train segmentation features of proposals which are generated by ACF detector; then take the product of SVM score and original ACF score as the final acceptance decision. Based on the Regions with CNN features (R-CNN) framework, Gidaris and Komodakis [4] concatenate multi-region CNN features and CNN-based semantic segmentation-aware features as object representations. Hariharan et al. [9] extract features from bounding boxes and foreground regions by training separate networks.

In above-mentioned works, segmentation feature extraction based on coarse bounding box annotations [4] or simple foreground segmentation [9] may lead to insufficient description of objects in scenes. The public dataset of fine road scene attribute annotations [13] and the recent development in CNN [10], [11], [12] make it possible for us to learn more accurate semantic segmentation feature maps. Due to complicated scene layout exemplified by severe occlusion and dense crowd in natural images, however, it is still difficult to obtain the pedestrian bounding box directly from the semantic segmentation feature map. Relying on elaborately designed deep learning model, previous methods [1], [7] are able to achieve excellent results. In fact, the R-CNN framework [2], [6] is even surpassed by a straightforward scheme in which the conventional shallow features ACF are combined with boosted decision trees. In this paper, we therefore consider using filtered shallow-deep features that fuse traditional ACF and deep semantic segmentation feature channels under the conventional framework [1], [5], [7].

As shown in Fig. 1, the semantic segmentation feature map (SSFM) describes each pixel with a probability vector of semantic classes, including all objects in road scenes such as person, tree, road, pole, etc. With the capability of describing pixel-wise semantic attributes, SSFM somewhat provides auxiliary information complementary to low-level appearance features [2], [3], which further improves the detection performance.

By concatenating HOG+LUV channels (as those in ACF [3]) and SSFM (shown in Fig. 1), shallow-deep channels have both appearance descriptive power at the image level and deep semantic cues at the pixel level. Then similar to the convolution operation in deep learning, a family of rectangular hand-designed filters with the same size are operated on the shallow-deep feature channels. Consequently, more abstract and compact feature maps are gained after using manual filters on original shallow-deep feature channels. Finally, the feature vector as the final feature is fed into the decision trees. The flowchart of our method is shown in Fig. 2.

The contributions of this paper are summarized as follows:

• To our knowledge, this is the first work that integrates the deep semantic segmentation feature maps into the process of learning the traditional ACF detector.

• A set of hand-designed rectangular filters analogous to checkboard [2] are applied on the shallow-deep channels for more discriminative feature information. Superior performance is achieved with a small number of our checkboard-like filters used.

• Filtered shallow-deep feature channels achieve a superior result on Caltech pedestrian dataset with a log-average miss rate 16.87 %, which significantly outperforms all hand-crafted features [3], [5], [15], [16] and some CNN based methods [6], [17].

The remainder of this paper is structured as follows. In the next section, we review the related work. Then we introduce our proposed method in Section 3 and evaluate our method on two benchmarks in Section 4. Finally, the conclusion is presented in Section 5.