To have a unique solution of an ill-posed inverse problem, the usual way is to embed prior information in terms of regularizer or smoothness criterion. In this work, both the inverse mechanism (the relationship of blur and sharp patches) and the smoothness prior are learned simultaneously from the image itself, in multiple scales. We have shown experimentally that the proposed method outperform the existing state-of-the-art techniques on high noise environment and produce comparable result otherwise; moreover, it is almost three times faster than existing ones.