Image super-resolution via a densely connected recursive network
Introduction
Single image super-resolution (SISR) refers to the process of recovering high-resolution (HR) images from low-resolution (LR) inputs. With the increasing ubiquity of closed-circuit cameras and remote communication techniques, SISR has attracted considerable research attention. The potential applications of SISR include intelligent surveillance systems, remote sensing, medical image enhancement, and telecommunication.
Deep network-based approaches [1], [2], [3], [4], [5] have been widely studied for the SISR task. Deep learning methods intend to reconstruct HR outputs with clear details by learning an end-to-end non-linear mapping between LR and HR images. Deep networks have significantly advanced the reconstruction performance of the existing literature. Increasing the network depth is proven effective for improving the performance of deep models. However, the computation and storage complexities increase rapidly with the increasing network depth. Conducting SISR with a very deep network is costly for practical applications with limited storage space and computation resources. Therefore, balancing the performance and the complexity is of great importance to deep learning methods.
In this paper, a novel network named the densely connected recursive network (DCRN) is proposed to reconstruct high-quality images with fewer parameters and less computation time. The DCRN is derived from the DenseNet [6], forming an enhanced dense unit to adapt the SISR task. Specially, the BN layers are removed from the traditional dense unit to avoid the smoothing operation and reduce the executive consumption, while the SE blocks [7] are employed to learn globally structured features. Besides, we use a recursive architecture to keep the network compact while increasing the network depth. Furthermore, we propose a de-convolution based residual learning approach, which extracts deep features from LR input and achieves the residual output through the de-convolution layer. The execution speed of residual image estimation is largely accelerated by the proposed approach.
In summary, the study makes the following contributions.
- •
We design an advanced dense unit to improve the efficiency and accuracy of densely-connected networks for SISR.
- •
We accelerate the residual learning by using a de-convolution layer skillfully.
- •
The experiments on Set5, Set14, BSD100, and Urban100 demonstrate that the DCRN outperforms the state-of-the-art methods in balancing the reconstruction accuracy, computation speed, and storage requirements.
Section snippets
Deep network-based methods for SISR
Due to the strength in semantic feature learning, deep learning is playing an increasingly significant role in the SISR task [1], [2], [8], [9], [10], [11], [12]. Dong et al. [1] pioneered a CNN model named the SRCNN that learns the non-linear mapping from LR image patches to HR image patches via three fully connected layers. Kim et al. [8] proposed a deep neural network named the very deep neural network (VDSR). The VDSR is the first work to successfully advance the performance of SISR by
Densely connected recursive network
In this section, we describe the proposed DCRN approach. We first introduce the enhanced dense unit in the DCRN and then gradually present the details of the overall framework.
Experimental settings
Datasets and evaluation protocols. Following VDSR [8], our training set comprises 291 images, of which 91 samples are from Yang et al. [14], and the remaining 200 images are from the Berkeley Segmentation Dataset [15]. For testing, we use four popular SISR benchmarks, namely Set5 [16], Set14 [17], BSD100 [15], and Urban100 [18], which contains 5, 14, 100, and 100 testing images, respectively.
Evaluation protocols. We compare the DCRN with the state-of-the-art approaches, including ScSR [14], A+
Conclusion
We propose an efficient network named the DCRN to cope with the SISR task. The DCRN introduces an enhanced dense unit by removing the BN layers and adopting the SE blocks. Besides, a recursive architecture is adopted to control the parameters of the dense blocks. Notably, we propose a de-convolution based residual learning approach to accelerate the executive speed.
Extensive experiments are conducted on Set5, Set14, BSD100, and Urban 100 datasets. The experimental results validate the
Acknowledgment
This project was supported by the NSFC (U1611461 and 61573387).
Zhanxiang Feng received the B.E. degree in automation from Sun Yat-Sen University, China, in 2012. He is currently pursuing the Ph.D. degree in information and communication engineering with Sun Yat-Sen University, China. His research interests include person re-identification, face recognition, face hallucination, image super-resolution, and visual surveillance. He has authored papers in IEEE TIP and ICPR. His ICPR 2016 paper won the enclosed finalist best student paper award.
References (19)
- et al.
Learning a deep convolutional network for image super-resolution
Proceedings of the European Conference on Computer Vision
(2014) - et al.
Image super-resolution via deep recursive residual network
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
(2017) - et al.
Accelerating the super-resolution convolutional neural network
Proceedings of the European Conference on Computer Vision
(2016) - et al.
Face hallucination by deep traversal network
Proceedings of the International Conference on Pattern Recognition
(2016) - et al.
Attention-Aware Face Hallucination via Deep Reinforcement Learning
(2017) - et al.
Densely Connected Convolutional Networks
(2017) - et al.
Squeeze-and-Excitation Networks
(2017) - et al.
Accurate image super-resolution using very deep convolutional networks
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
(2016) - et al.
Deeply-recursive convolutional network for image super-resolution
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
(2016)
Cited by (11)
Wavelet-based residual attention network for image super-resolution
2020, NeurocomputingCitation Excerpt :Single image super-resolution (SR), defined as restoring high-resolution (HR) images from corresponding low-resolution (LR) versions, is an important technique of image processing in computer vision community. Deep learning-based image SR has accumulated substantial attention in recent years [1–7]. It has been extended to various real-world applications, e.g., medical imaging [8–12], video surveillance [13–17], remote sensing [18,19], prerequisite for image classification [20,21], detection [22], recognition [23,24], and denoising [25].
RBPNET: An asymptotic Residual Back-Projection Network for super-resolution of very low-resolution face image
2020, NeurocomputingCitation Excerpt :Li et al. [13] present a two-channel convolutional neural network (SDSR) to restore the general outline of the image and detailed texture information simultaneously. Image super-resolution via a Densely Connected Recursive Network (DCRN) [14] is proposed to reconstruct high-quality images with fewer parameters and less computation time. So far, RCAN [15] (Very Deep Residual Channel Attention Networks) is performed best on single generic image super-resolution in PNSR and SSIM in case of 4 × super-resolution or less.
Single image super-resolution with multi-level feature fusion recursive network
2019, NeurocomputingCitation Excerpt :As the network deepens, the computing cost and the number of parameters increase dramatically. Therefore, recursive learning is introduced into the super-resolution field to reduce the model parameters [22,23,29,30], which refers to applying the same modules multiple times in a recursive manner. DRCN [22] introduces recursive layers as inference net to the network, so that model parameters do not increase while more recursions are used (up to 16).
G-GANISR: Gradual generative adversarial network for image super resolution
2019, NeurocomputingResearch Progress of Single Image Super-resolution Reconstruction Technology
2022, Zidonghua Xuebao/Acta Automatica Sinica
Zhanxiang Feng received the B.E. degree in automation from Sun Yat-Sen University, China, in 2012. He is currently pursuing the Ph.D. degree in information and communication engineering with Sun Yat-Sen University, China. His research interests include person re-identification, face recognition, face hallucination, image super-resolution, and visual surveillance. He has authored papers in IEEE TIP and ICPR. His ICPR 2016 paper won the enclosed finalist best student paper award.
Jianhuang Lai received his M.Sc. degree in applied mathematics in 1989 and his Ph.D. in mathematics in 1999 from Sun Yat-Sen University, China. He joined Sun Yat-Sen University in 1989 as an Assistant Professor, where currently, he is a Professor in School of Data and Computer Science. His current research interests are in the areas of computer vision, pattern recognition and its applications. He has published over 250 scientific papers in the international journals and conferences on image processing and pattern recognition, e.g. IEEE TPAMI, IEEE TNN, IEEE TIP, IEEE TSMC (Part B), Pattern Recognition, ICCV, CVPR and ICDM. He serves as a deputy director of the Image and Graphics Association of China and also serves as a standing director of the Image and Graphics Association of Guangdong. He is also the deputy director of Computer Vision Committee, China Computer Federation (CCF).
Xiaohua Xie received the B.S. degree in mathematics and applied mathematics from Shantou University in 2005, the M.S. degree in information and computing science and the Ph.D. degree in applied mathematics from Sun Yat-Sen University, China, in 2007 and 2010, respectively. He was an Associate Professor with Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences. He is currently a Associate Professor with Sun Yat-Sen University. He has authored or co-authored over 30 papers in prestigious international journals and conferences. His current research fields cover image processing, computer vision, pattern recognition, and computer graphics.
Junyong Zhu received his M.S. and Ph.D. degrees in applied mathematics in the school of Mathematics and Computational Science from Sun Yat-Sen University, Guangzhou, PR China, in 2010 and 2014, respectively. He has working toward the post-doctoral research in the Department of Information Science and Technology, Sun Yat-Sen University. Currently, he is a Associate Research Professor with Sun Yat-Sen University. His current research interests include heterogeneous face recognition, visual transfer learning using partial labeled or unlabeled auxiliary data and non-linear clustering. He has authored and co-authored papers in international journals and conferences such as IEEE TIFS, PR, ICIP, AMFG and ICDM. His cooperative ICDM 2010 paper won the Honorable Mention for Best Research Paper Awards and his CCBR 2012 paper won the Best Student Paper Awards.