For the traditional content-based image retrieval system, the number of irrelevant images for a given query image is significantly more than that of relevant images in an image repository. Therefore, the numbers of negative samples and positive samples are highly unbalanced, which makes the traditional binary classifiers ineffective. In this paper, our proposed modified AdaBoost-based one-class support vector machine (OCSVM) ensemble is utilized to deal with the aforesaid problem. In our proposed method, the weight update formula of training data for AdaBoost is modified to make AdaBoost fit for combining the results of OCSVMs even though OCSVM is regarded as a strong classifier. Compared with the other three related methods, our proposed approach exhibits better performance on the three benchmark image databases.