Graph ranking is one of popular and successful technique for information retrieval. However, conventional graph ranking has two shortcomings when deployed for social image search. First, as social tags are noisy and incomplete, using that, the initial ranked list of images is inaccurate and impacts the following visual re-ranking. Another tough issue is how to conduct query-sensitive image re-ranking when multiple visual feature sets are available. In this work, we propose a sparse multigraph ranking framework, in which multiple graphs built on different visual features are integrated to simultaneously encode the image ranking. In particular, a sparse constraint is imposed on the fusion of different features, hoping to select a compact yet informative combination of features for different queries. To deal with the highly noisy issue inherent in social tags, a tag refinement solution along with word embedding is utilized to derive the more accurate initial ranking list, which services as the supervision signal for the proposed graph ranking framework. Extensive experimental analyses and evaluations on NUS-WIDE dataset demonstrate the proposed method can achieve state-of-the-art performance.