Skip to main content
SpringerLink
Log in

Perception-oriented Single Image Super-Resolution Network with Receptive Field Block

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In recent years, deep learning has been widely applied to single image super-resolution(SISR). However, the majority of deep learning methods employ the Mean Square Error(MSE) loss as the objective optimization function, and the generated results are frequently too smooth and lack of details. In addition, the high-frequency information of the reconstructed image is severely lost, resulting in a generated image with poor visual effects. In order to address the aforementioned issues, this paper proposes a super-resolution network (RRFDB-GAN) with a receptive field module with a generative adversarial network as the main framework. The network adopts the receptive field block (RFB), which enables it to extract the features in multiple scales and to improve the discriminability. In this paper, the Residual in Residual Dense Block (RRDB) and the Residual of Receptive Field Dense Block (RRFDB) are combined into a new module, called Basic block. This module enhances the capability of feature reconstruction for low-resolution images. During the upsampling stage, a combination of Nearest Neighborhood Interpolation and Sub-pixel convolution is used to reduce the computational complexity and provide additional contextual information for super-resolution reconstruction, while achieving satisfactory performance. Finally, the four Basic blocks are integrated with the upsampling module into a simple end-to-end framework. Extensive experimental results demonstrate that the proposed method in this paper shows more details on the five test sets and outperforms other methods in terms of quantitative metrics and perception assessment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Zhang L, Wu X (2006) An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans Image Process 15(8):2226–2238

    Article  Google Scholar 

  2. Zhang K, Gao X, Tao D, Li X (2012) Single image super-resolution with non-local means and steering kernel regression. IEEE Trans Image Process 21(11):4544–4556

    Article  MathSciNet  Google Scholar 

  3. Yang J, Wright J, Huang T, Ma Y (2008) Image super-resolution as sparse representation of raw image patches. In: 2008 IEEE conference on computer vision and pattern recognition, pp 1–8. IEEE

  4. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  Google Scholar 

  5. Anchored neighborhood regression for fast example-based super-resolution. In: Proceedings of the IEEE international conference on computer vision, pp 1920–1927 (2013)

  6. Timofte R, De Smet V, Van Gool L (2014) A+: adjusted anchored neighborhood regression for fast super-resolution. In: Asian conference on computer vision. Springer, pp 111–126

  7. Timofte R, De Smet V, Van Gool L (2013) Anchored neighborhood regression for fast example-based super-resolution. In: Proceedings of the IEEE international conference on computer vision, pp 1920–1927

  8. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: European conference on computer vision. Springer, pp 184–199

  9. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  10. Dong C, Loy CC, He K, Tang X (2015) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Article  Google Scholar 

  11. Lim B, Son S, Kim H, Nah S, Mu Lee K (2017) Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 136–144

  12. Wang X, Yu K, Dong C, Loy CC (2018) Recovering realistic texture in image super-resolution by deep spatial feature transform. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 606–615

  13. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z et al. (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4681–4690

  14. Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, Qiao Y, Change Loy C (2018) Esrgan: Enhanced super-resolution generative adversarial networks. In: Proceedings of the European conference on computer vision (ECCV) Workshops

  15. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European conference on computer vision (ECCV), pp 286–301

  16. Soh JW, Park GY, Jo J, Cho NI (2019) Natural and realistic single image super-resolution with explicit natural manifold discrimination. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8122–8131

  17. Ahn N, Kang B, Sohn K-A (2018) Fast, accurate, and lightweight super-resolution with cascading residual network. In: Proceedings of the European conference on computer vision (ECCV), pp 252–268

  18. Guo Y, Chen J, Wang J, Chen Q, Cao J, Deng Z, Xu Y, Tan M (2020) Closed-loop matters: Dual regression networks for single image super-resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5407–5416

  19. Liu J, Zhang W, Tang Y, Tang J, Wu G (2020) Residual feature aggregation network for image super-resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2359–2368

  20. Niu B, Wen W, Ren W, Zhang X, Yang L, Wang S, Zhang K, Cao X, Shen H (2020) Single image super-resolution via a holistic attention network. In: European conference on computer vision, pp 191–207. Springer

  21. Li Z, Yang J, Liu Z, Yang X, Jeon G, Wu W (2019) Feedback network for image super-resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3867–3876

  22. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets in advances in neural information processing systems (NIPS)

  23. Chen W, Ma Y, Liu X, Yuan Y (2021) Hierarchical generative adversarial networks for single image super-resolution. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 355–364

  24. Hui Z, Li J, Gao X, Wang X (2021) Progressive perception-oriented network for single image super-resolution. Inf Sci 546:769–786

    Article  MathSciNet  Google Scholar 

  25. Li W, Zhou K, Qi L, Lu L, Jiang N, Lu J, Jia J (2021) Best-buddy gans for highly detailed image super-resolution. arXiv preprint arXiv:2103.15295

  26. Ma C, Rao Y, Lu J, Zhou J (2021) Structure-preserving image super-resolution. IEEE Trans Pattern Anal Mach Intell

  27. Huang J-H, Wang H-K, Liao Z-W (2020) Hfd-srgan: Super-resolution generative adversarial network with high-frequency discriminator. In: 2020 IEEE international conference on systems, man, and cybernetics (SMC), pp 3148–3153. IEEE

  28. Park S-J, Son H, Cho S, Hong K-S, Lee S (2018) Srfeat: Single image super-resolution with feature discrimination. In: Proceedings of the European conference on computer vision (ECCV), pp 439–455

  29. Sajjadi MS, Scholkopf B, Hirsch M (2017) Enhancenet: Single image super-resolution through automated texture synthesis. In: Proceedings of the IEEE international conference on computer vision, pp 4491–4500

  30. Shang T, Dai Q, Zhu S, Yang T, Guo Y (2020) Perceptual extreme super-resolution network with receptive field block. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp 440–441

  31. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European conference on computer vision. Springer, pp 391–407

  32. Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1874–1883

  33. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1646–1654

  34. Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3147–3155

  35. Lai W-S, Huang J-B, Ahuja N, Yang M-H (2017) Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 624–632

  36. Mechrez R, Talmi I, Shama F, Zelnik-Manor L (2018) Maintaining natural image statistics with the contextual loss. In: Asian conference on computer vision. Springer, pp 427–443

  37. Ma C, Yang C-Y, Yang X, Yang M-H (2017) Learning a no-reference quality metric for single-image super-resolution. Comput Vis Image Underst 158:1–16

    Article  Google Scholar 

  38. Mittal A, Soundararajan R, Bovik AC (2012) Making a “completely blind” image quality analyzer. IEEE Signal Process Lett 20(3):209–212

    Article  Google Scholar 

  39. Zhang K, Gool LV, Timofte R (2020) Deep unfolding network for image super-resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3217–3226

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (Grant No. U1908214, 61906032), Special Project of Central Government Guiding Local Science and Technology Development (Grant No. 2021JH6/10500140), Program for the Liaoning Distinguished Professor, Program for Innovative Research Team in University of Liaoning Province (Grant No. LT2020015), Program for Innovative Research Team in University of Liaoning Province (Grant No. LT2020015), the Science and Technology Innovation Fund of Dalian (Grant No. 2020JJ25CY001), the Fundamental Research Funds for the Central Universities under grant DUT21TD107.

Author information

Authors and Affiliations

  1. National and Local Joint Engineering Laboratory of Computer Aided Design, School of Software Engineering, Dalian University, Dalian, China

    Wei Zhang, Wanshu Fan, Dongsheng Zhou & Qiang Zhang

  2. School of Computer Science and Technology, Dalian University of Technology, Dalian, China

    Yaqing Hou, Xin Yang, Dongsheng Zhou, Qiang Zhang & Xiaopeng Wei

Authors
  1. Wei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yaqing Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wanshu Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dongsheng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xiaopeng Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dongsheng Zhou.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, W., Hou, Y., Fan, W. et al. Perception-oriented Single Image Super-Resolution Network with Receptive Field Block. Neural Comput & Applic 34, 14845–14858 (2022). https://doi.org/10.1007/s00521-022-07341-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07341-y

Keywords

Search

Navigation