Inferring user search goals for a query can be very useful in improving search engine relevance and user experience. Although the research on analyzing user goals or intents for text search has received much attention, little has been proposed for image search. In this paper, we propose a novel approach to infer user search goals in image search by mining search engine query logs with semi-supervised spectral clustering. We combine the visual information of the clicked images with user click information by using graph-based models and then cluster the images with spectral clustering to capture user image-search goals. Experimental results based on a popular commercial search engine demonstrate the effectiveness of the proposed method.