Typicality ranking: beyond accuracy for video semantic annotation

Abstract

In video annotation, the typicalities or relevancy degrees of relevant samples to a certain concept are generally different. Thus we argue that it is more reasonable to rank typical relevant samples higher than non-typical ones. However, generally the labels of the training data only differentiate relevant of irrelevant; that is to say, typical or non-typical training samples have the same contribution to the learning process. Therefore, the learned scores of the unlabeled data cannot well measure the typicality. Accordingly, three pre-processing approaches are proposed to relax the labels of the training data to real-valued typicality scores. Then the typicality scores of the training data are propagated to unlabeled data using manifold ranking. Meanwhile, we propose to use a novel criterion, Average Typicality Precision (ATP), to replace the frequently used one, Average Precision (AP), for evaluating the performance of video typicality ranking algorithms. Though AP cares the number of relevant samples at the top of the annotation rank list, it actually does not care the typicality order of these samples, while which was taken into consideration of the evaluation strategy ATP. Experiments conducted on the TRECVID data set demonstrate that this typicality ranking scheme is more consistent with human perception than normal accuracy based ranking schemes.