Jiawang Dan
ABSTRACT
Deep convolutional neural networks have made remarkable achievements in single-image super-resolution tasks in recent years. However, current methods do not consider the characteristics of super-resolution that the adjacent areas carry similar information. In this paper, we propose a scale adaptive and lightweight super-resolution with a selective hierarchical residual network (SHRN), which utilizes the repeated texture features. Specifically, SHRN is stacked by several selective hierarchical residual blocks (SHRB). The SHRB mainly contains a hierarchical feature fusion structure (HFFS) and a selective feature fusion structure (SFFS). The HFFS uses multiple branches to obtain multiscale features due to the varying texture size of objects. The SFFS fuses features of adjacent branches to select effective information. Plenty of experiments demonstrate that our lightweight model achieves better performance against other methods by extracting scale adaptive features and utilizing the repeated texture structure.
- Chao Dong, Chen Change Loy, Kaiming He, and X. Tang. 2014. Learning a Deep Convolutional Network for Image Super-Resolution. In ECCV.Google Scholar
- Jiwon Kim, J. Lee, and Kyoung Mu Lee. 2016. Accurate Image Super-Resolution Using Very Deep Convolutional Networks. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), 1646–1654.Google Scholar
- J. Kim, J. Lee, and Kyoung Mu Lee. 2016. Deeply-Recursive Convolutional Network for Image Super-Resolution. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), 1637–1645.Google Scholar
- Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. 2017. Enhanced Deep Residual Networks for Single Image Super-Resolution. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2017), 1132–1140.Google Scholar
- Yulun Zhang, Kunpeng Li, K. Li, L. Wang, B. Zhong, and Yun Fu. 2018. Image Super-Resolution Using Very Deep Residual Channel Attention Networks. In ECCV.Google Scholar
- Ying Tai, Jian Yang, and X. Liu. 2017. Image Super-Resolution via Deep Recursive Residual Network. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), 2790–2798.Google ScholarCross Ref
- Ying Tai, Jian Yang, X. Liu, and Chunyan Xu. 2017. MemNet: A Persistent Memory Network for Image Restoration. 2017 IEEE International Conference on Computer Vision (ICCV) (2017), 4549–4557.Google Scholar
- Namhyuk Ahn, Byungkon Kang, and Kyung-Ah Sohn. 2018. Fast, Accurate, and, Lightweight Super-Resolution with Cascading Residual Network. ArXiv abs/1803.08664 (2018).Google Scholar
- Xiangxiang Chu, Bo Zhang, Hailong Ma, R. Xu, Jixiang Li, and Qingyuan Li. 2019. Fast, Accurate and Lightweight Super-Resolution with Neural Architecture Search. ArXiv abs/1901.07261 (2019).Google Scholar
- Juncheng Li, F. Fang, Kangfu Mei, and Guixu Zhang. 2018. Multi-scale Residual Network for Image Super-Resolution. In ECCV.Google Scholar
- A. Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. Commun. ACM 60 (2012), 84 – 90.Google ScholarDigital Library
- T. Zhang, Guo-Jun Qi, Bin Xiao, and Jingdong Wang. 2017. Interleaved Group Convolutions. 2017 IEEE International Conference on Computer Vision (ICCV) (2017), 4383–4392.Google Scholar
- Guotian Xie, Jingdong Wang, Ting Zhang, Jianhuang Lai, Richang Hong, and G. Qi. 2018. IGCV2: Interleaved Structured Sparse Convolutional Neural Networks. arXiv: Computer Vision and Pattern Recognition (2018).Google Scholar
- K. Sun, Mingjie Li, Dong Liu, and Jingdong Wang. 2018. IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks. In BMVC.Google Scholar
- J. Carreira, H. Madeira, and J. Silva. 1998. Xception: A Technique for the Experimental Evaluation of Dependability in Modern Computers. IEEE Trans. Software Eng. 24 (1998), 125–136.Google ScholarDigital Library
- A. Howard, Menglong Zhu, Bo Chen, D. Kalenichenko, W. Wang, Tobias Weyand, M. Andreetto, and H. Adam. 2017. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. ArXiv abs/1704.04861 (2017).Google Scholar
- Mark Sandler, A. Howard, Menglong Zhu, A. Zhmoginov, and Liang-Chieh Chen. 2018. MobileNetV2: Inverted Residuals and Linear Bottlenecks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), 4510–4520.Google ScholarCross Ref
- X. Zhang, X. Zhou, Mengxiao Lin, and Jian Sun. 2018. ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), 6848–6856.Google Scholar
- Ningning Ma, X. Zhang, Hai-Tao Zheng, and Jian Sun. 2018. ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design. In ECCV.Google Scholar
- F. Yu and V. Koltun. 2016. Multi-Scale Context Aggregation by Dilated Convolutions. CoRR abs/1511.07122 (2016).Google Scholar
- F. Yu, V. Koltun, and T. Funkhouser. 2017. Dilated Residual Networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), 636–644.10 The Name of the Title is Hope Woodstock ’18, June 03–05, 2018, Woodstock, NYGoogle Scholar
- Xiang Li, Wenhai Wang, Xiaolin Hu, and Jian Yang. 2019. Selective Kernel Networks. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019), 510–519.Google ScholarCross Ref
- Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), 770–778.Google Scholar
- Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. 2016. Identity Mappings in Deep Residual Networks. ArXiv abs/1603.05027 (2016).Google Scholar
- Felix Heide, Douglas Lanman, D. Reddy, J. Kautz, K. Pulli, and D. Luebke. 2014. Cascaded displays: spatiotemporal superresolution using offset pixel layers. ACM Trans. Graph. 33 (2014), 60:1–60:11.Google ScholarDigital Library
- Zhong Zhen-jie, Huai Li-bo, and W. Qi. 2020. Application Research of Overexposure Image Restoration Algorithm Based on Dynamic Convolution Template. Proceedings of the 2020 the 4th International Conference on Innovation in Artificial Intelligence (2020).Google Scholar
- Pourya Shamsolmoali, Masoumeh Zareapoor, Junhao Zhang, and J. Yang. 2019. Image super resolution by dilated dense progressive network. Image Vis. Comput. 88 (2019), 9–18.Google ScholarDigital Library
- F. Yu, Dequan Wang, and Trevor Darrell. 2018. Deep Layer Aggregation. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), 2403–2412.Google Scholar
- Gustav Larsson, M. Maire, and Gregory Shakhnarovich. 2017. FractalNet: Ultra-Deep Neural Networks without Residuals. ArXiv abs/1605.07648 (2017).Google Scholar
- Christian Szegedy,W. Liu, Y. Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, D. Erhan, V. Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), 1–9.Google ScholarCross Ref
- S. Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. ArXiv abs/1502.03167 (2015).Google Scholar
- Christian Szegedy, V. Vanhoucke, S. Ioffe, Jon Shlens, and Z. Wojna. 2016. Rethinking the Inception Architecture for Computer Vision. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), 2818–2826.Google Scholar
- Christian Szegedy, S. Ioffe, V. Vanhoucke, and Alexander Amir Alemi. 2017. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. ArXiv abs/1602.07261 (2017).Google Scholar
- L. Itti and C. Koch. 2001. Computational modelling of visual attention. Nature Reviews Neuroscience 2 (2001), 194–203.Google ScholarCross Ref
- L. Itti, C. Koch, and E. Niebur. 2009. A Model of Saliency-Based Visual Attention for Rapid Scene Analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20 (2009), 1254–1259.Google ScholarDigital Library
- H. Larochelle and Geoffrey E. Hinton. 2010. Learning to combine foveal glimpses with a third-order Boltzmann machine. In NIPS.Google Scholar
- V. Mnih, N. Heess, A. Graves, and K. Kavukcuoglu. 2014. Recurrent Models of Visual Attention. In NIPS.Google Scholar
- B. Olshausen, C. Anderson, and D. V. Van Essen. 1993. A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information. In The Journal of neuroscience : the official journal of the Society for Neuroscience.Google Scholar
- Fei Wang, Mengqing Jiang, Chen Qian, S. Yang, Cheng Li, H. Zhang, Xiaogang Wang, and X. Tang. 2017. Residual Attention Network for Image Classification. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), 6450–6458.Google ScholarCross Ref
- Jie Hu, L. Shen, Samuel Albanie, Gang Sun, and Enhua Wu. 2020. Squeeze-and-Excitation Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 42 (2020), 2011–2023.Google ScholarDigital Library
- Jongchan Park, S. Woo, Joon-Young Lee, and In-So Kweon. 2018. BAM: Bottleneck Attention Module. In BMVC.Google Scholar
- S. Woo, Jongchan Park, Joon-Young Lee, and In-So Kweon. 2018. CBAM: Convolutional Block Attention Module. In ECCV.Google Scholar
- C. Ledig, L. Theis, Ferenc Huszár, J. Caballero, Andrew Aitken, Alykhan Tejani, J. Totz, Zehan Wang, and W. Shi. 2017. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), 105–114.Google ScholarCross Ref
- Chao Dong, Chen Change Loy, and X. Tang. 2016. Accelerating the Super-Resolution Convolutional Neural Network. In ECCV.Google Scholar
- W. Shi, J. Caballero, Ferenc Huszár, J. Totz, A. Aitken, R. Bishop, D. Rueckert, and Zehan Wang. 2016. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), 1874–1883.Google ScholarCross Ref
- Muhammad Haris, Gregory Shakhnarovich, and Norimichi Ukita. 2018. Deep back-projection networks for superresolution. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1664–1673.Google Scholar
- Yulun Zhang, Yapeng Tian, Y. Kong, B. Zhong, and Yun Fu. 2018. Residual Dense Network for Image Super-Resolution. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), 2472–2481.Google Scholar
- Wei-Sheng Lai, Jia-Bin Huang, N. Ahuja, and Ming-Hsuan Yang. 2017. Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), 5835–5843.Google Scholar
- Wei-Sheng Lai, Jia-Bin Huang, N. Ahuja, and Ming-Hsuan Yang. 2019. Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 41 (2019), 2599–2613.Google ScholarCross Ref
- Sachin Mehta, M. Rastegari, Anat Caspi, L. Shapiro, and Hannaneh Hajishirzi. 2018. ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation. ArXiv abs/1803.06815 (2018).Google Scholar
- Zhaowen Wang, Ding Liu, Jianchao Yang, Wei Han, and T. Huang. 2015. Deep Networks for Image Super-Resolution with Sparse Prior. 2015 IEEE International Conference on Computer Vision (ICCV) (2015), 370–378.Google Scholar
- Mehdi S. M. Sajjadi, B. Schölkopf, and M. Hirsch. 2017. EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis. 2017 IEEE International Conference on Computer Vision (ICCV) (2017), 4501–4510.Google Scholar
- Kai Zhang, W. Zuo, and Lei Zhang. 2018. Learning a Single Convolutional Super-Resolution Network for Multiple Degradations. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), 3262–3271.Google ScholarCross Ref
- V. Nair and Geoffrey E. Hinton. 2010. Rectified Linear Units Improve Restricted Boltzmann Machines. In ICML. 9 Woodstock ’18, June 03–05, 2018, Woodstock, NY Trovato and Tobin,Google ScholarDigital Library
- Saining Xie, Ross B. Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. 2017. Aggregated Residual Transformations for Deep Neural Networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), 5987–5995.Google ScholarCross Ref
- Shanghua Gao, Ming-Ming Cheng, Kai Zhao, Xinyu Zhang, Ming-Hsuan Yang, and P. Torr. 2021. Res2Net: A New Multi-Scale Backbone Architecture. IEEE Transactions on Pattern Analysis and Machine Intelligence 43 (2021), 652–662. 8 The Name of the Title is Hope Woodstock ’18, June 03–05, 2018, Woodstock, NY.Google ScholarDigital Library
- Liang-Chieh Chen, G. Papandreou, I. Kokkinos, Kevin Murphy, and A. Yuille. 2018. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence 40 (2018), 834–848.Google ScholarCross Ref
- R. Timofte, Eirikur Agustsson, L. Gool, Ming-Hsuan Yang, Lei Zhang, Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, Kyoung Mu Lee, Xintao Wang, Yapeng Tian, K. Yu, Yulun Zhang, Shixiang Wu, Chao Dong, L. Lin, Y. Qiao, Chen Change Loy, W. Bae, Jae Jun Yoo, Yoseob Han, J. C. Ye, Jae-Seok Choi, M. Kim, Yuchen Fan, J. Yu, Wei Han, Ding Liu, Haichao Yu, Zhangyang Wang, Humphrey Shi, X. Wang, T. Huang, Yunjin Chen, Kai Zhang, W. Zuo, Z. Tang, Linkai Luo, S. Li, M. Fu, L. Cao, Wen Heng, G. Bui, Truc Le, Ye Duan, D. Tao, Ruxin Wang, Xu Lin, Jianxin Pang, Jinchang Xu, Y. Zhao, Xiangyu Xu, Jin shan Pan, Deqing Sun, Y. Zhang, X. Song, Yuchao Dai, X. Qin, X. Huynh, Tiantong Guo, H. Mousavi, T. Vu, V. Monga, C. Cruz, K. Egiazarian, V. Katkovnik, Rakesh Mehta, A. Jain, Abhinav Agarwalla, C. Praveen, R. Zhou, Hongdiao Wen, C. Zhu, Zhiqiang Xia, Z. Wang, and Q. Guo. 2017. NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2017), 1110–1121.Google Scholar
- Marco Bevilacqua, A. Roumy, C. Guillemot, and M. Alberi-Morel. 2012. Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding. In BMVC.Google Scholar
- Roman Zeyde, Michael Elad, and M. Protter. 2010. On Single Image Scale-Up Using Sparse-Representations. In Curves and Surfaces.Google Scholar
- D. Martin, Charless C. Fowlkes, D. Tal, and Jitendra Malik. 2001. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 2 (2001), 416–423 vol.2.Google ScholarCross Ref
- Jia-Bin Huang, Abhishek Singh, and N. Ahuja. 2015. Single image super-resolution from transformed self-exemplars. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), 5197–5206.Google ScholarCross Ref
- Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980 (2015).Google Scholar
- Zhou Wang, A. Bovik, H. R. Sheikh, and E. P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13 (2004), 600–612.Google ScholarDigital Library
- Jae-Seok Choi and M. Kim. 2017. A Deep Convolutional Neural Network with Selection Units for Super-Resolution. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2017), 1150–1156.Google ScholarCross Ref
- T. Tong, Gen Li, Xiejie Liu, and Qinquan Gao. 2017. Image Super-Resolution Using Dense Skip Connections. 2017 IEEE International Conference on Computer Vision (ICCV) (2017), 4809–4817.Google Scholar
- Yifan Wang, L. Wang, Hongyu Wang, and P. Li. 2019. End-to-End Image Super-Resolution via Deep and Shallow Convolutional Networks. IEEE Access 7 (2019), 31959–31970.Google ScholarCross Ref
- T. Dai, Jianrui Cai, Yong-Bing Zhang, S. Xia, and Lei Zhang. 2019. Second-Order Attention Network for Single Image Super-Resolution. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019), 11057–11066.Google Scholar
- Xiangxiang Chu, Bo Zhang, R. Xu, and Hailong Ma. 2020. Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search. ArXiv abs/1901.01074 (2020).Google Scholar
Index Terms
- Scale adaptive and lightweight super-resolution with a selective hierarchical residual network
Index terms have been assigned to the content through auto-classification.
Recommendations
Deep Residual Attention Network for Spectral Image Super-Resolution
Computer Vision – ECCV 2018 WorkshopsAbstractSpectral imaging sensors often suffer from low spatial resolution, as there exists an essential tradeoff between the spectral and spatial resolutions that can be simultaneously achieved, especially when the temporal resolution needs to be ...
Spatial-Spectral Deep Residual Network for Hyperspectral Image Super-Resolution
AbstractRecently, single hyperspectral image super-resolution (SR) methods based on deep learning have been extensively studied. However, there has been limited technical development focusing on single hyperspectral image super-resolution due to the high-...
Attention hierarchical network for super-resolution
AbstractDeep neural networks with attention mechanism for super-resolution (SR) have achieved good SR performance by focusing on the high-frequency components of images. However, during the SR process, it is difficult for these networks to obtain multi-...
Comments