Skip to main content
SpringerLink
Log in

The improved deep plug-and-play super-resolution with residual-in-residual dense block for arbitrary blur kernels

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Single-image super-resolution (SISR) reconstruction has highly academic and practical values. The deep plug-and-play super-resolution (DPSR) framework has been proposed to super-resolve low-resolution (LR) images with arbitrary blur kernels. However, DPSR does not make full use of hierarchical features from original LR images, thereby achieving relatively-low performance, such as getting low average peak signal to noise ratio (PSNR) and structural similarity (SSIM) values. Considering residual-in-residual dense block (RRDB) can exploit hierarchical features, in this paper, firstly, RRDB is introduced to design an improved DPSR (IDPSR) framework with RRDB for arbitrary blur kernels. Secondly, the RRDB is adopted to replace the deep feature extraction part in DPSR in order to extract abundant local features, which makes the network capacity higher benefiting from the dense connections. The residual learning in different levels in RRDB can obtain high quality images. Finally, the test experiments are based on Set5, Set14, Urban100 and BSD100 datasets. The experimental results show that, under different blur kernels and different scale factors, PSNR and SSIM values of our proposed method increase by 0.34dB and 0.68%, respectively; under different noise levels, the average PSNR and SSIM values increase by 0.27dB and 1.01%, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data Availability

The toy tasks dataset analyzed during the current study are available publicly in the Set5 repository (http://people.rennes.inria.fr/Aline.Roumy/results/SR_BMVC12.html), Set14 repository (https://sites.google.com/site/romanzeyde/research-interests), BSD100 repository (https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/) and Urban100 repository (https://sites.google.com/site/jbhuang0604/publications/struct_sr).

References

  1. Agustsson E, Timofte R (2017) Ntire 2017 challenge on single image super-resolution: Dataset and study. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 126–135

  2. Ahn N, Kang B, Sohn KA (2018) Fast, accurate, and lightweight super-resolution with cascading residual network. In: Proceedings of the European Conference on Computer Vision (ECCV)

  3. Bevilacqua M, Roumy A, Guillemot C, et al (2012) Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding. In: British Machine Vision Conference (BMVC), Guildford, Surrey, United Kingdom, https://doi.org/10.5244/C.26.135, https://hal.inria.fr/hal-00747054

  4. Chan SH, Wang X, Elgendy OA (2017) Plug-and-play admm for image restoration: fixed-point convergence and applications. IEEE Trans Comput Imag 3(1):84–98. https://doi.org/10.1109/TCI.2016.2629286

    Article  MathSciNet  Google Scholar 

  5. Dong C, Loy CC, He K, et al (2014) Learning a deep convolutional network for image super-resolution. In: Fleet D, Pajdla T, Schiele B, et al (eds) Computer Vision – ECCV 2014. Springer International Publishing, Cham, pp 184–199, https://doi.org/10.1007/978-3-319-10593-2_13

  6. Dong W, Zhang L, Shi G et al (2013) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process 22(4):1620–1630. https://doi.org/10.1109/TIP.2012.2235847

    Article  MathSciNet  MATH  Google Scholar 

  7. Efrat N, Glasner D, Apartsin A, et al (2013) Accurate blur models vs. image priors in single image super-resolution. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp 2832–2839

  8. Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: Gordon G, Dunson D, Dudík M (eds) Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, vol 15. PMLR, Fort Lauderdale, FL, USA, pp 315–323, http://proceedings.mlr.press/v15/glorot11a.html

  9. He K, Zhang X, Ren S, et al (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 770–778

  10. Huang G, Liu Z, van der Maaten L, et al (2017) Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 4700–4708

  11. Huang JB, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 5197–5206

  12. Hui Z, Wang X, Gao X (2018) Fast and accurate single image super-resolution via information distillation network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  13. Jiang X, Wang N, Xin J et al (2021) Learning lightweight super-resolution networks with weight pruning. Neural Networks 144:21–32, https://doi.org/10.1016/j.neunet.2021.08.002, https://www.sciencedirect.com/science/article/pii/S0893608021003075

  14. Jiang X, Wang N, Xin J et al (2022) Toward pixel-level precision for binary super-resolution with mixed binary representation. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2022.3201528

    Article  Google Scholar 

  15. Karras T, Aila T, Laine S, et al (2017) Progressive growing of gans for improved quality, stability, and variation. CoRR abs/1710.10196. arXiv:1710.10196

  16. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1646–1654

  17. Ledig C, Theis L, Huszár F, et al (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4681–4690

  18. Lee CY, Xie S, Gallagher P, et al (2015) Deeply-supervised nets. In: Artificial intelligence and statistics, PMLR, pp 562–570

  19. Lim B, Son S, Kim H, et al (2017) Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp 136–144

  20. Martin D, Fowlkes C, Tal D, et al (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, pp 416–423 vol.2, https://doi.org/10.1109/ICCV.2001.937655

  21. Peleg T, Elad M (2014) A statistical prediction model based on sparse representations for single image super-resolution. IEEE Trans Image Process 23(6):2569–2582. https://doi.org/10.1109/TIP.2014.2305844

    Article  MathSciNet  MATH  Google Scholar 

  22. Schulter S, Leistner C, Bischof H (2015) Fast and accurate image upscaling with super-resolution forests. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3791–3799

  23. Shi W, Caballero J, Ledig C, et al (2013) Cardiac image super-resolution with global correspondence using multi-atlas patchmatch. In: International conference on medical image computing and computer-assisted intervention, Springer, pp 9–16, https://doi.org/10.1007/978-3-642-40760-4_2

  24. Song D, Xu C, Jia X et al (2020) Efficient residual dense block search for image super-resolution. Proc AAAI Conf Artif Intell 34(07):12007–12014

    Google Scholar 

  25. Timofte R, De Smet V, Van Gool L (2013) Anchored neighborhood regression for fast example-based super-resolution. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp 1920–1927

  26. Timofte R, De Smet V, Van Gool L (2015) A+: Adjusted anchored neighborhood regression for fast super-resolution. In: Cremers D, Reid I, Saito H et al (eds) Computer Vision - ACCV 2014. Springer International Publishing, Cham, pp 111–126. https://doi.org/10.1007/978-3-319-16817-3_8

    Chapter  Google Scholar 

  27. Timofte R, Rothe R, Van Gool L (2016) Seven ways to improve example-based single image super resolution. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, pp 1865–1873

  28. Wang X, Yu K, Wu S, et al (2018) Esrgan: Enhanced super-resolution generative adversarial networks. In: Proceedings of the European Conference on Computer Vision (ECCV) Workshops, pp 0

  29. Xin J, Wang N, Jiang X et al (2020) Binarized neural network for single image super resolution. In: Vedaldi A, Bischof H, Brox T et al (eds) Computer Vision - ECCV 2020. Springer International Publishing, Cham, pp 91–107

    Chapter  Google Scholar 

  30. Yang CY, Ma C, Yang MH (2014) Single-image super-resolution: A benchmark. In: Fleet D, Pajdla T, Schiele B et al (eds) Computer Vision - ECCV 2014. Springer International Publishing, Cham, pp 372–386. https://doi.org/10.1007/978-3-319-10593-2_25

    Chapter  Google Scholar 

  31. Zeyde R, Elad M, Protter M (2012) On single image scale-up using sparse-representations. In: Boissonnat JD, Chenin P, Cohen A et al (eds) Curves and Surfaces. Springer, Berlin Heidelberg, Berlin, Heidelberg, pp 711–730. https://doi.org/10.1007/978-3-642-27413-8_47

    Chapter  Google Scholar 

  32. Zhang K, Gao X, Tao D et al (2012) Single image super-resolution with non-local means and steering kernel regression. IEEE Transactions on Image Processing 21(11):4544–4556. https://doi.org/10.1109/TIP.2012.2208977

    Article  MathSciNet  MATH  Google Scholar 

  33. Zhang K, Zuo W, Zhang L (2018a) Learning a single convolutional super-resolution network for multiple degradations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3262–3271

  34. Zhang K, Zuo W, Zhang L (2019) Deep plug-and-play super-resolution for arbitrary blur kernels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 1671–1681

  35. Zhang L, Wu X (2006) An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Transactions on Image Processing 15(8):2226–2238. https://doi.org/10.1109/TIP.2006.877407

    Article  Google Scholar 

  36. Zhang Y, Tian Y, Kong Y, et al (2018b) Residual dense network for image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2472–2481

  37. Zou WWW, Yuen PC (2012) Very low resolution face recognition problem. IEEE Transactions on Image Processing 21(1):327–340. https://doi.org/10.1109/TIP.2011.2162423

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China (51905416, 51804249), Xi’an Science and Technology Program (2022JH-RGZN-0041), Qin Chuangyuan Scientists + Engineers Team Construction Program in Shaanxi Province(2022KXJ-38), the Natural Science Basic Research Program of Shaanxi (Grant No. 2021JQ-574) and Scientific Research Plan Projects of Shaanxi Education Department20JK0758).

Author information

Authors and Affiliations

  1. College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an, 710054, Shaanxi Province, China

    Chao Xu

  2. College of Electrical and Control Engineering, Xi’an University of Science and Technology, Xi’an, 710054, Shaanxi Province, China

    Xiaoling Yang, Xiangdong Huang & Hongguang Pan

  3. CHN Energy Shendong Coal Group Design Company, Shenmu, 719315, Shaanxi Province, China

    Shan Li

  4. College of Artificial Intelligence, Xi’an Jiaotong University, Xi’an, 710049, Shaanxi Province, China

    Xinyu Lei

Authors
  1. Chao Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoling Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiangdong Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hongguang Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xinyu Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongguang Pan.

Ethics declarations

Conflict of interest

We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, C., Yang, X., Li, S. et al. The improved deep plug-and-play super-resolution with residual-in-residual dense block for arbitrary blur kernels. Pattern Anal Applic 26, 1657–1670 (2023). https://doi.org/10.1007/s10044-023-01192-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-023-01192-6

Keywords

Search

Navigation