Automatic Image Annotation Based on Generalized Relevance Models

Abstract

This paper presents a generalized relevance model for automatic image annotation through learning the correlations between images and annotation keywords. Different from previous relevance models that can only propagate keywords from the training images to the test ones, the proposed model can perform extra keyword propagation among the test images. We also give a convergence analysis of the iterative algorithm inspired by the proposed model. Moreover, to estimate the joint probability of observing an image with possible annotation keywords, we define the inter-image relations through proposing a new spatial Markov kernel based on 2D Markov models. The main advantage of our spatial Markov kernel is that the intra-image context can be exploited for automatic image annotation, which is different from the traditional bag-of-words methods. Experiments on two standard image databases demonstrate that the proposed model outperforms the state-of-the-art annotation models.

Appendix

This appendix presents the proof of Theorem 1. First, we can obtain from Eq. 10 that

$$ F_t(W)= (\alpha S)^t F_0(W)+ (1-\alpha)\sum\limits_{i=0}^{t-1}(\alpha S)^i F_0(W). $$

(22)

Since the sum of each row of S is bounded by 1 (i.e. $\sum_{J\in \mathcal{U}}P(J|I)\leq 1$), from the Perron–Frobenius theorem [23], we know the spectral radius of S or ρ(S) ≤ 1. Moreover, since 0 < α < 1 (i.e. ρ(αS) < 1), we have

$$ \lim\limits_{t \to \infty}(\alpha S)^t =0,~\lim\limits_{t \to \infty}\sum\limits_{i=0}^{t-1}(\alpha S)^i =(I-\alpha S)^{-1}, $$

(23)

where I is the identity matrix. Hence, it follows from Eq. 22 that the sequence {F _t (W)} will converge to

$$ F^*(W)=\lim\limits_{t \to \infty}F_t(W)=(1-\alpha)(I-\alpha S)^{-1}F_0(W), $$

(24)

when t → ∞. That is, we have proven the theorem.