Learning in an image retrieval scheme that uses hidden annotations is investigated. Compared with low level visual features that are straightforward functions of the raw pixel values, hidden annotations are higher level hidden semantic attributes. In the proposed scheme, a small set of images is manually labeled with several hidden annotations. For each annotation, a support vector machine (SVM) classifier is trained using the images labeled with it as positive examples and others as negative examples. Based on the trained SVMs, the annotations are propagated to the unlabeled images in the database. To perform relevance feedback in the annotation space, a probabilistic re-weighting algorithm is proposed. Experimental results on a general-purpose database of 10,000 images demonstrate the potential of hidden annotation-based image retrieval and the superiority of the proposed relevance feedback algorithm over two existing algorithms.