How to accurately and efficiently localize objects in images is a challenging computer vision problem. In this article, a novel top-down fine-grained saliency object localization method based on deep-learned features is proposed, which can localize the same object in input image as the query image. The query image and its three subsample images are used as top-down cues to guide saliency detection. We ameliorate Convolutional Neural Network (CNN) using the fast VGG network (VGG-f) and retrained on the Pascal VOC 2012 dataset. Experiment on the FiFA dataset demonstrates that the proposed algorithm can effectively localize the saliency region and find the same object (human face) as the query. Experiments on the David1 and Face1 sequences conclusively prove that the proposed algorithm is able to effectively deal with different challenging factors including appearance and scale variations, shape deformation and partial occlusion.