Humans tend to understand image scene by recognizing visual elements, then conjecturing and inferring based on them, hence are able to search relevant images. In this paper, we concern about the problem of complex image retrieval by reasoning image dense captions, which is similar to the way of human perception for searching images. Specifically, we transform the problem of complex image retrieval into a dense captioning and scene graph matching issue by using structured language descriptions for retrieval. Experimental results on a novel proposed large-scale content-based image retrieval dataset demonstrate the rationality and effectiveness of our method.