Combining Non-local Sparse and Residual Channel Attentions for Single Image Super-resolution Across Modalities

Abstract

Single image super-resolution (SISR) is an ill-posed problem that aims to generate a high-resolution (HR) image from a single low-resolution (LR) image. The main objective of super-resolution is to add relevant high-frequency detail to complement the available low-frequency information. Classical techniques such as non-local similarity and sparse representations have shown promising results in the SISR task. Nowadays, deep learning techniques such as convolutional neural networks (CNN) can extract deep features to improve the SISR results. However, CNN does not explicitly consider similar information in the image. Hence, we employ the non-local sparse attention (NLSA) module in the CNN framework such that it can explore the non-local similarity within an image. We consider sparsity in the non-local operation by focusing on a particular group named attention bin among many groups of features. NLSA is intended to retain the long-range of non-local operation modeling capacity while benefiting from the efficiency and robustness of sparse representation. However, NLSA focuses on similarity in spatial dimension by neglecting any channel-wise significance. Hence, we try to rescale the channel-specific features adaptively while taking into account channel interdependence by using residual channel attention. We combine the advantages of non-local sparse attention (NLSA) and residual channel attention to produce competitive results in different image modalities such as optical color images, depth maps, and X-Ray without re-training.