DrlNet: Blind object proposal quality assessment with discriminative response learning

Abstract

Object proposal quality assessment without ground truth as reference is a challenging task. Some existing methods measure the quality with hand-crafted metrics for subjective metrics, such as objectness and foreground confidence. Recently, deep learning is adopted for direct assessment for quantifiable metric, such as Intersection over Union (IoU). However, we find that IoU, the commonly used quality metric, is far from fully describing the quality of an object proposal. Proposals with the same IoU score may carry totally different amount of discriminative attribute. We introduce a new metric named Discriminative Information Richness (DIR) to characterize the discriminative degree of the given object proposal. DIR is derived from the response intensity of the projected deep feature maps, whose high correlation response indicates the discriminative regions. Besides, we design a convolutional neural network named DrlNet to simultaneously predict IoU scores and perceive the richness of the identification information. DrlNet is defined as a multi-metric joint deep regression network for both spatial covering prediction and discriminative information richness perception. Compared with the solely IoU based models, DrlNet can provide more comprehensive quality assessment. We perform comprehensive experiments on both PASCAL VOC dataset and COCO dataset. The experimental results show that our DrlNet performs well on both proposal selection and object detection tasks. Particularly, experimental results on COCO dataset demonstrate the good generalization ability of the proposed model.

Introduction

In the past few decades, academia has witnessed rapid development of object detection [1], especially with the boost of deep learning [2]. In Convolutional Neural Network (CNN) based two-stage object detection approaches, such as R-CNN [3], SPP-Net [4], Fast R-CNN [5] and Faster R-CNN [6], proposal algorithms are widely used for generating object candidates.

Actually, besides object detection, object proposal algorithms, which aim to provide bounding box candidates with high object covering confidence, have played important roles in many other high-level computer vision tasks, such as object segmentation [7], [8], visual tracking [9], [10], action detection [11], [12], et al. Like region-wise processing strategy [13], [14], [15], [16], [17], which are widely used in computer vision tasks, proposal based processing strategy is another commonly adopted processing manner. The object proposal algorithms generate massive target hypothesises, which can specify the processing targets and greatly narrow the information space to be processed. While, it also shows the facts that, the quality of object proposals will directly influence the performance of such subsequent tasks. A small candidate pool with high quality proposals will greatly boost the performance and efficiency of subsequent steps of the application algorithm. Conversely, poor proposals may even significantly degrade application performance.

Given ground truth file, proposal quality can be easily estimated with the commonly used IoU metric. However, there is usually no available ground truth file for direct quality assessment in real-world applications, which makes it a blind assessment problem. Actually, no-reference object proposal quality ranking has been studied in proposal generating methods [18], [19], [20], [21] for good proposal suggestion. However, these build-in ranking modules can only provide relative superiority orders, but hardly give a metric based prediction, such as predicting the IoU score. To cover this concern, Wu et al. [22] proposed a generic proposal evaluator (GPE), which can directly predict the IoU score of the given object proposal. However, taking IoU as the only metric can hardly distinguish the quality difference of proposals covering the different parts of the object.

In Fig. 1 (a), we present a cat image, whose ground truth object box is shown in Fig. 1 (b). Two bounding box candidates, which share the same IoU values and equally cover the right and left body of the cat, are presented in Fig. 1 (c) and Fig. 1 (d), respectively. Apparently, GPE can not effectively evaluate the relative superiority of the two candidate boxes since they have the same IoU scores. While, we can easily notice that the proposal in Fig. 1 (c) can provide more discriminative information which can help distinguish the category of target object better than that in Fig. 1 (d), so proposal in Fig. 1 (c) should be given higher quality evaluation. Richness of discriminative information is a vital factor in proposal quality assessment, which also has broad prospects of application to high-level tasks.

Taking unsupervised and weakly supervised detection tasks for example, discriminative candidates can definitely provide more non-redundant and valuable information for training. Accordingly, the introduction of discriminative judgment helps the detector to capture the key attributes efficiently and reduces the cost of computation and storage.

Hence, screening out the region proposal with rich discriminative information from massive candidates is an important and promising task. However, richness of discriminative information has never been comprehensively explored, especially as a quality indicator. Actually, there are no existing detection or recognition databases which have annotated samples including quantitative scores for discriminative degree. One of the main obstacles is how to define a suitable metric to quantitatively compute the discrimination of the given proposal. And then, another subsequent problem is how to predict this metric without ground truth as reference. In this paper, our contribution mainly reflects as follows:

(1) We introduce a new metric named Discriminative Information Richness (DIR) to characterize the discriminative degree of the given proposal.

(2) A blind quality evaluation method within discriminative response learning framework is proposed, which can simultaneously perceive the richness of the identification information and the target covering in the candidate area.