An image retrieval system is a technique for browsing, searching and retrieving images from a big database of digital images. In this paper, we propose a new content-based image retrieval system that can solve the object and scene recognition problems and categorize similar images. The proposed model consists of a deep structure support vector machine with Gaussian mixture model, which is combined with human-like top-down selective attention model using growing fuzzy topology adaptive resonant theory (GFTART) network and scene understanding using GIST. The results suggest that the proposed model has better performance than other recent methods used in this field.