The content-based image retrieval (CBIR) has been an active research field for which several feature extraction, classification and retrieval techniques have been proposed up to date. However, when the database size grows larger, it is a common fact that the overall retrieval performance significantly deteriorates. In this paper, we propose collective network of (evolutionary) binary classifiers (CNBC) framework to achieve a high retrieval performance even though the training (ground truth) data may not be entirely present from the beginning and thus the system can only be trained incrementally. The CNBC framework basically adopts a “Divide and Conquer” type approach by allocating several networks of binary classifiers (NBCs) to discriminate each class and performs evolutionary search to find the optimal binary classifier (BC) in each NBC. In such an evolution session, the CNBC body can further dynamically adapt itself with each new incoming class/feature set without a full-scale re-training or re-configuration. Both visual and numerical performance evaluations of the proposed framework over benchmark image databases demonstrate its scalability; and a significant performance improvement is achieved over traditional retrieval techniques.