In this paper we propose a method for solving various imaging inverse problems via complexity regularization that leverages existing image compression techniques. Lossy compression has already been proposed in the past for Gaussian denoising - the simplest inverse problem. However, extending this approach to more complicated inverse problems (e.g., deblurring, inpainting, etc.) seemed to result in intractable optimization tasks. In this work we address this difficulty by decomposing the complicated optimization problem via the Half Quadratic Splitting approach, resulting in a sequential solution of a simpler l₂-regularized inverse problem followed by a rate-distortion optimization, replaced by an efficient compression technique. In addition, we suggest an improved complexity regularizer that quantifies the average block-complexity in the restored signal, which in turn, extends our algorithm to rely on averaging multiple decompressed images obtained from compression of shifted images. We demonstrate the proposed scheme for inpainting of corrupted images, using leading image compression techniques such as JPEG2000 and HEVC.