Automatic image annotation with real-world community contributed data set

Abstract

With the massive explosion of social multimedia community, social images have become very popular in our daily life. The image-associated labels are a valuable resource for automatic image annotation, but they tend to be unreliable. In this paper, we exploit the problem of image annotation from real-world community contributed images and their associated incorrect, insufficient, and personalized labels. We present SNTag, a novel semantic neighborhood learning method, on which image annotation task can be efficiently carried out in real-world scenario. First, we propose to use image-associated labels as the supervising information to guide the replenishment of training images, which enable the labels for training image not only more sufficient, but also more correct. Then, the “semantic balanced neighborhood” for image is generated, thus enabling the presence of more rare labels in image label list. Furthermore, we generate “semantic consistent neighborhood” within corresponding “semantic balanced neighborhood”. The retrieved neighbor images are not only visually alike but also semantically related. Contrary to earlier work, these neighbors are retrieved from the same subspace by the integration of metric learning embedded in multiple labels and sparse reconstruction. Based on the neighbor set, we propose a novel algorithm to assign the optimal labels to the image, which is more robust to noise. We conduct extensive experiments on several standard real-world benchmark data sets downloaded from community websites. The experimental results demonstrate that it outperforms the current state-of-the-art methods.