In this paper, we propose an iterative Wiener filter which can simultaneously perform interpolation and restoration by using non-local means to directly model the correlation between the desired high-resolution image and observed low-resolution image. A novel mechanism is proposed to control the decay speed of the correlation function while iteratively updating both estimated correlation and high-resolution image. During the iterations, the image is decomposed into patches with similar intensities at initial iterations and the patches are connected naturally with good convergence. Experimental results show that the proposed algorithm is able to produce natural image structures, and provides better PSNR and visual quality than the state-of-the-art algorithms using the sparse representation and natural image priors.