With the development of the economic and the popularity of smartphones, location-based service is receiving more and more attention. It can be used inside a building where GPS signals are often unavailable. Because of its low deployment cost, vision-based indoor localization is becoming popular in the complicated indoor environment. However, in order to increase the accuracy of indoor localization, the database should be as large as possible. But in online phase, the query image retrieving would be more time-consuming. Therefore, we propose a fast cluster-based GIST (C- GIST) image retrieval method to reduce the time overhead of image retrieval. Compared with the existing indoor localization system, the proposed method utilizing video data could reduce the computational complexity evidently, which is much more convenient. The experiment results show that the proposed method is applicable in the complicated indoor environment, whose localization error less than 2 meters is nearly 70%. The error performance of the proposed method is slightly worse than the traditional method. Nevertheless, the proposed method decreases the computational complexity of image retrieval significantly.