In this paper, a novel image quality assessment (IQA) metric for the multi-exposure image fusion (MEF) is proposed by using contrast information. Specifically, the proposed approach firstly performs the measurements of contrast structure similarity and contrast saturation similarity based on the observation that human perception is sensitive to contrast information inherited in the MEF and reference images. Then, considering that different reference images contribute differently to the MEF image, the weights are adaptively assigned to each reference image according to its relevance to the MEF image. A standard deviation based pooling strategy and multi-scale scheme are subsequently used to generate the final MEF image quality score. Experimental results have shown that the proposed metric produces high consistency with human perception of the MEF image quality and outperforms the state-of-the-art quality metric.