Skip to main content
SpringerLink
Log in

LCRCA: image super-resolution using lightweight concatenated residual channel attention networks

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Images that are more similar to the original high-resolution images can be generated by deep neural network-based super-resolution methods than the non-learning-based ones, but the huge and sometimes redundant network structure and parameters make them unbearable. To get high-quality super-resolution results in computation resource-limited scenarios, we propose a lightweight skip concatenated residual channel attention network, LCRCA for image super-resolution. Specifically, we design a light but efficient deep residual block (DRB) which can generate more precise residual information by using more convolution layers under the same computation budget. To enhance the feature maps of DRB, an improved channel attention mechanism named statistical channel attention (SCA) is proposed by introducing channel statistics. Besides, compared with the commonly used skip connections, we propose to use skip concatenation (SC) to build information flows for feature maps of different layers. Finally, DRB, SCA, and SC are efficiently used to form the proposed network LCRCA. Experiments on four test sets show that our method can gain up to 3.2 dB and 0.12 dB over the bicubic interpolation and the representative lightweight method FERN, respectively, and can recover image details more accurately than the compared algorithms. Code can be found at https://github.com/pengcm/LCRCA.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. The PSNR/SSIM value below each detail-enlarged subgraph is the quantitative result of the corresponding algorithm on the entire test image, not the subgraph

References

  1. Agustsson E, Timofte R (2017) Ntire 2017 challenge on single image super-resolution: Dataset and study. In: IEEE conference on computer vision and pattern recognition workshops (CVPRW). IEEE, Honolulu, pp 1122–1131

  2. Ahn N, Kang B, Sohn KA (2018) Fast accurate, and lightweight super-resolution with cascading residual network. In: Proceedings of the European conference on computer vision (ECCV). Springer International Publishing, Munich, pp 256–272

  3. Ba J, Caruana R (2014) Do deep nets really need to be deep? arXiv:1312.6184

  4. Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: 23rd british machine vision conference, Guildford, England

  5. Cao Y, Wang C, Song C, Tang Y, Li H (2021) Real-time super-resolution system of 4k-video based on deep learning. arXiv:2107.05307

  6. Chen L, Pan J, Jiang J, Zhang J, Han Z, Bao L (2021a) Multi-stage degradation homogenization for super-resolution of face images with extreme degradations. IEEE Trans Image Process 30:5600–5612. https://doi.org/10.1109/TIP.2021.3086595

    Article  Google Scholar 

  7. Chen W, Yao P, Gai S, Da F (2021b) Multi-scale feature aggregation network for image super-resolution. Appl Intell :1–10

  8. Chen Y, He F, Wu Y, Hou N (2017) A local start search algorithm to compute exact hausdorff distance for arbitrary point sets. Pattern Recogn 67:139–148

    Article  Google Scholar 

  9. Chen Y, Liu L, Phonevilay V, Gu K, Xia R, Xie J, Zhang Q, Yang K (2021c) Image super-resolution reconstruction based on feature map attention mechanism. Appl Intell :1–14

  10. Choi JS, Kim M (2017) A deep convolutional neural network with selection units for super-resolution. In: IEEE conference on computer vision and pattern recognition workshops (CVPRW). IEEE, Honolulu, pp 1150–1156

  11. Dai T, Cai J, Zhang Y, Xia S, Zhang L (2019) Second-order attention network for single image super-resolution. In: IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, Long Beach, pp 11057–11066

  12. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: Proceedings of the european conference on computer vision (ECCV). Springer International Publishing, Zurich, pp 184–199

  13. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: Proceedings of the European conference on computer vision (ECCV). Springer International Publishing, Amsterdam, pp 391–407

  14. Dong W, Zhou C, Wu F, Wu J, Shi G, Li X (2021) Model-guided deep hyperspectral image super-resolution. IEEE Trans Image Process 30:5754–5768. https://doi.org/10.1109/TIP.2021.3078058

    Article  Google Scholar 

  15. Du X (2021) Single image super-resolution using global enhanced upscale network. Appl Intell :1–7

  16. Fang C, Zhu Y, Liao L, Ling X (2021) Tsrgan: Real-world text image super-resolution based on adversarial learning and triplet attention. Neurocomputing 455:88–96. https://doi.org/10.1016/j.neucom.2021.05.060, https://www.sciencedirect.com/science/article/pii/S0925231221008134

    Article  Google Scholar 

  17. Feng X, Li X, Li J (2021) Multi-scale fractal residual network for image super-resolution. Appl Intell 51(4):1845–1856

    Article  Google Scholar 

  18. Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32(200):675–701

    Article  Google Scholar 

  19. Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics, JMLR Workshop and Conference Proceedings, Fort Lauderdale, FL, USA, pp 315– 323

  20. Greff K, Srivastava R, Schmidhuber J (2017) Highway and residual networks learn unrolled iterative estimation. arXiv:1612.07771

  21. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: IEEE international conference on computer vision (ICCV). IEEE, Santiago, pp 1026–1034

  22. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, Las Vegas, pp 770-778

  23. Hu J, Shen L, Albanie S, Sun G, Wu E (2020a) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell 42:2011–2023

    Article  Google Scholar 

  24. Hu X, Fan Z, Jia X, Li Z, Zhang X, Qi L, Xuan Z (2021) Towards effective learning for face super-resolution with shape and pose perturbations. https://doi.org/10.1016/j.knosys.2021.106938. https://www.sciencedirect.com/science/article/pii/S095070512100201X, vol 220, p 106938

  25. Hu Y, Li J, Huang Y, Gao X (2020b) Channel-wise and spatial feature modulation network for single image super-resolution. IEEE Trans Circ Syst Vid Technol 30(11):3911–3927. https://doi.org/10.1109/TCSVT.2019.2915238

    Article  Google Scholar 

  26. Huang JB, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: IEEE conference on computer vision and pattern recognition (CVPR), Boston, pp 5197-5206

  27. Hui Z, Wang X, Gao X (2018) Fast and accurate single image super-resolution via information distillation network. In: IEEE/CVF conference on computer vision and pattern recognition. IEEE, Salt Lake City, pp 723-731. https://doi.org/10.1109/CVPR.2018.00082

  28. Hui Z, Gao X, Yang Y, Wang X (2019) Lightweight image super-resolution with information multi-distillation network. In: 27th ACM international conference on multimedia (MM), Association for Computing Machinery, Nice, France, MM-19, pp 2024–2032. https://doi.org/10.1145/3343031.3351084

  29. Hui Z, Gao X, Wang X (2020) Lightweight image super-resolution with feature enhancement residual network. Neurocomputing 404:50–60

    Article  Google Scholar 

  30. Isallari M, Rekik I (2021) Brain graph super-resolution using adversarial graph neural network with application to functional brain connectivity, vol 71, p 102084. https://doi.org/10.1016/j.media.2021.102084. https://www.sciencedirect.com/science/article/pii/S1361841521001304

  31. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: IEEE conference on computer vision and pattern recognition (CVPR), IEEE, Las Vegas, pp 1646-1654

  32. Kim J, Lee JK, Lee KM (2016) Deeply-recursive convolutional network for image super-resolution. In: IEEE conference on computer vision and pattern recognition (CVPR), IEEE, Las Vegas, pp 1637–1645

  33. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:14126980

  34. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  35. Kumar V, Gupta P (2012) Importance of statistical measures in digital image processing. Int J Emerg Technol Adv Eng 2(8):56–62

    Google Scholar 

  36. Lai WS, Huang JB, Ahuja N, Yang MH (2017) Deep laplacian pyramid networks for fast and accurate super-resolution

  37. Lee CY, Xie S, Gallagher PW, Zhang Z, Tu Z (2015) Deeply-supervised nets. arXiv:1409.5185

  38. Li B, Liu J, Wang B, Qi Z, Shi Y (2019) S-lwsr: Super lightweight super-resolution network. arXiv:1909.10774

  39. Liang Y, He F, Li H (2019) An asymmetric and optimized encryption method to protect the confidentiality of 3d mesh model. Adv Eng Inform 42:100963

    Article  Google Scholar 

  40. Liang Y, He F, Zeng X (2020) 3d mesh simplification with feature preservation based on whale optimization algorithm and differential evolution. Integr Comput-Aid Eng (Preprint):417– 435

  41. Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: IEEE conference on computer vision and pattern recognition workshops (CVPRW), IEEE, Honolulu, pp 1132-1140

  42. Liu B, Ait-Boudaoud D (2020) Effective image super resolution via hierarchical convolutional neural network. Neurocomputing 374:109–116

    Article  Google Scholar 

  43. Lu E, Hu X (2021) Image super-resolution via channel attention and spatial attention. Appl Intell :1–9

  44. Martin DR, Fowlkes CC, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: 8th IEEE international conference on computer vision (ICCV 2001), Vancouver, Canada, vol 2. pp 416-423

  45. Nemenyi PB (1963) Distribution-free multiple comparisons. Princeton University

  46. Paszke A, Gross S, Chintala S, Chanan G, Yang E, Devito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in pytorch

  47. Qin J, Huang Y, Wen W (2020) Multi-scale feature fusion residual network for single image super-resolution. Neurocomputing 379:334–342

    Article  Google Scholar 

  48. Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: IEEE conference on computer vision and pattern recognition (CVPR), IEEE, Las Vegas, pp 1874-1883

  49. Park Sung Cheol, Park Min Kyu, Kang Moon Gi (2003) Super-resolution image reconstruction: a technical overview. IEEE Signal Process Mag 20(3):21–36

    Article  Google Scholar 

  50. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), IEEE, Boston, pp 1-9

  51. Tai Y, Yang J, Liu X, Xu C (2017) Memnet: A persistent memory network for image restoration. In: IEEE international conference on computer vision (ICCV), IEEE, Venice, pp 4549–4557

  52. Timofte R, Smet VD, Gool LV (2014) A+: Adjusted anchored neighborhood regression for fast super-resolution. In: 12th Asian conference on computer vision (ACCV), Singapore

  53. Tong T, Li G, Liu X, Gao Q (2017) Image super-resolution using dense skip connections. In: IEEE international conference on computer vision (ICCV), IEEE, Venice, pp 4809–4817. https://doi.org/10.1109/ICCV.2017.514

  54. Wang C, Li Z, Shi J (2019) Lightweight image super-resolution with adaptive weighted learning network. arXiv:1904.02358

  55. Wang F, Jiang M, Qian C, Yang S, Li C, Zhang H, Wang X, Tang X (2017) Residual attention network for image classification. In: IEEE conference on computer vision and pattern recognition (CVPR), IEEE, Honolulu, pp 6450–6458. https://doi.org/10.1109/CVPR.2017.683

  56. Wang H, Hu Q, Wu C, Chi J, Yu X, Wu H (2021a) Dclnet: Dual closed-loop networks for face super-resolution. Knowl-Based Syst 222:106987. https://doi.org/10.1016/j.knosys.2021.106987, https://www.sciencedirect.com/science/article/pii/S0950705121002501

    Article  Google Scholar 

  57. Wang Q, Gao Q, Wu L, Sun G, Jiao L (2021b) Adversarial multi-path residual network for image super-resolution. IEEE Trans Image Process 30:6648–6658. https://doi.org/10.1109/TIP.2021.3096089

    Article  Google Scholar 

  58. Wenlong Z, Yihao L, Dong C, Qiao Y (2021) Ranksrgan: Generative adversarial networks with ranker for image super-resolution. IEEE Trans Pattern Anal Mach Intell. Early access. https://doi.org/10.1109/TPAMI.2021.3096327

  59. Wu Y, He F, Zhang D, Li X (2015) Service-oriented feature-based data exchange for cloud-based design and manufacturing. IEEE Trans Serv Comput 11(2):341–353

    Article  Google Scholar 

  60. Xiong C, Shi X, Gao Z, Wang G (2021) Attention augmented multi-scale network for single image super-resolution. Appl Intell 51(2):935–951

    Article  Google Scholar 

  61. Xue J, Zhao YQ, Bu Y, Liao W, Chan JCW, Philips W (2021) Spatial-spectral structured sparse low-rank representation for hyperspectral image super-resolution. IEEE Trans Image Process 30:3084–3097. https://doi.org/10.1109/TIP.2021.3058590

    Article  MathSciNet  Google Scholar 

  62. Yan Y, Ren W, Hu X, Li K, Shen H, Cao X (2021) Srgat: Single image super-resolution with graph attention network. IEEE Trans Image Process 30:4905–4918. https://doi.org/10.1109/TIP.2021.3077135

    Article  Google Scholar 

  63. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European conference on computer vision (ECCV), Springer International Publishing, Munich, pp 294- 310

  64. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2021) Residual dense network for image restoration. IEEE Trans Pattern Anal Mach Intell 43:2480–2495

    Article  Google Scholar 

  65. Zhao X, Liao Y, Li Y, Zhang T, Zou X (2019) Fc2n: Fully channel-concatenated network for single image super-resolution. arXiv:1907.03221

  66. Zhu L, Zhan S, Zhang H (2019) Stacked u-shape networks with channel-wise attention for image super-resolution. Neurocomputing 345:58–66

    Article  Google Scholar 

  67. Zou Y, Zhang L, Liu C, Wang B, Hu Y, Chen Q (2021) Super-resolution reconstruction of infrared images based on a convolutional neural network with skip connections. Opt Lasers Eng 146:106717

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by National Natural Science Foundation of China (61671126).

Author information

Authors and Affiliations

  1. University of Electronic Science and Technology of China, Chengdu, China

    Changmeng Peng, Xiaoyang Huang, Zhizhong Fu & Xiaofeng Li

  2. Macro Net Communication Co., Ltd, Chongqing, China

    Pei Shu

Authors
  1. Changmeng Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pei Shu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoyang Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhizhong Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaofeng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhizhong Fu.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peng, C., Shu, P., Huang, X. et al. LCRCA: image super-resolution using lightweight concatenated residual channel attention networks. Appl Intell 52, 10045–10059 (2022). https://doi.org/10.1007/s10489-021-02891-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02891-5

Keywords

Search

Navigation