Deep light-weight image super resolution networks that provide a high performance have numerous real-life applications, such as mobile devices and multimedia systems. Hence, analyzing the capability of such deep networks in providing a similar performance between the cases that they are applied to the images with and without distributions similar to that of the training is crucial. In this paper, we carry out the robustness analysis of the deep state-of-the-art light-weight super resolution networks by proposing and using three metrics that are based on the statistical information of the super resolved images in both pixel level and feature level. The results of our metrics for the deep state-of-the-art light-weight super resolution networks demonstrate the behavior of such networks against realistic distribution shift in the test dataset.