Skip to main content
SpringerLink
Log in

Compact and progressive network for enhanced single image super-resolution—ComPrESRNet

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

The use of deep convolutional neural networks (CNNs) for single image super-resolution (SISR) in the recent years has led to numerous vision-based applications. Complementing the growing interest in the computer vision community embracing such networks, there is an unmet demand of reduced computational complexity. Despite being state-of-the-art for SISR tasks, CNN-based models need to be compact and efficient to account for applications running on low-cost deployment devices that have limited computation resources. While it is common to note that many state-of-the-art SISR approaches stack large number of convolutional layers in order to enhance their SR performance, there is proportional increase in the computational complexity. We propose a computationally efficient, compact and enhanced progressive network for SISR task which we hereafter refer as ComPrESRNet. The architectural changes employs a progressive learning strategy with a novel design of Enhanced densely connected parallel residual network (EDPRN) which simultaneously extracts rich features from the low-resolution (LR) observation while reducing the total number of parameters to 2.61M making it compact in nature that is suitable for low computational platform. The novelty of the proposed model stems from the inclusion of (i) densely connected ResBlock to extract rich features in the LR observation, (ii) extended global residual learning approach which stabilizes the training process effectively and also helps network to further improve the SR performance and (iii) progressive upscaling module which can generate an SR image of size \(\times 4\) and \(\times 8\) of original LR image. The robustness of the proposed method is further demonstrated on four different benchmark testing datasets consisting of natural scenes and urban landscape to exemplify the different applications. The superior performance over other state-of-the-art methods is also illustrated in this work for an upscaling factor \(\times 4\) and \(\times 8\) despite the lower computational complexity. The code of the paper is available at https://github.com/Vishal2188/Compactand-Progressive-Networkfor-Enhanced-SISR---ComPrESRNet.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Availability of data and material

Not Applicable

Notes

  1. https://github.com/Vishal2188/Compact-and-Progressive-Network-for-Enhanced-SISR---ComPrESRNet

  2. https://github.com/jbhuang0604/SelfExSR.

  3. http://cv.snu.ac.kr/research/VDSR/.

  4. http://cv.snu.ac.kr/research/DRCN/.

  5. http://vllab.ucmerced.edu/wlai24/LapSRN/.

  6. https://twitter.app.box.com/s/lcue6vlrd01ljkdtdkhmfvk7vtjhetog.

  7. https://github.com/nmhkahn/CARN-pytorch.

  8. https://github.com/Zheng222/IMDN.

  9. https://github.com/YuanfeiHuang/DeFiAN.

  10. https://github.com/fperazzi/proSR.

  11. https://github.com/MIVRC/MSRN-PyTorch.

  12. https://github.com/HyeongseokSon1/SRFeat.

  13. https://github.com/JWSoh/NatSR.

  14. https://github.com/ChaofWang/AWSRN.

  15. https://github.com/alterzero/DBPN-Pytorch.

References

  1. Agustsson, E., Timofte, R.: Ntire 2017 challenge on single image super-resolution: dataset and study. In: IEEE Conference on CVPR Workshops, vol. 3, p. 2 (2017)

  2. Ahn, N., Kang, B., Sohn, K.A.: Fast, accurate, and lightweight super-resolution with cascading residual network. In: ECCV), pp. 252–268 (2018)

  3. Anwar, S., Khan, S., Barnes, N.: A deep journey into super-resolution: A survey. CoRR abs/1904.07523 (2019)

  4. Barron, J.T.: A more general robust loss function. arXiv preprint arXiv:1701.03077 (2017)

  5. Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: BMVC (2012)

  6. Chollet, F.: Xception: Deep learning with depthwise separable convolutions. In: IEEE conference on CVPR, pp. 1251–1258 (2017)

  7. Chudasama, V., Upla, K.: E-prosrnet: An enhanced progressive single image super-resolution approach. CVIU p. 103038 (2020)

  8. Dai, T., Cai, J., Zhang, Y., Xia, S.T., Zhang, L.: Second-order attention network for single image super-resolution. In: IEEE Conference on CVPR, pp. 11065–11074 (2019)

  9. Dang-Nguyen, D.T., Pasquini, C., Conotter, V., Boato, G.: Raise: a raw images dataset for digital image forensics. In: ACM Multimedia Systems Conference, pp. 219–224. ACM (2015)

  10. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Article  Google Scholar 

  11. Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: ECCV, pp. 391–407 (2016)

  12. Fang, F., Li, J., Zeng, T.: Soft-edge assisted network for single image super-resolutionn. In: IEEE Transactions on Image Processing, vol. 29, pp. 4656–4668. IEEE (2020)

  13. Freeman, I., Roese-Koerner, L., Kummert, A.: Effnet: An efficient structure for convolutional neural networks. In: IEEE International Conference on Image Processing (ICIP), pp. 6–10. IEEE (2018)

  14. Freeman, W.T., Pasztor, E.C., Jones, T.R.: Example-based super-resolution. IEEE Comp. Gr. Appl. 22(02), 56–65 (2002)

    Article  Google Scholar 

  15. Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: IEEE Conference on ICCV, pp. 349–356. IEEE (2009)

  16. Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: IEEE Conference on ICCV, pp. 349–356 (2009)

  17. Guo, Y., Chen, J., Wang, J., Chen, Q., Cao, J., Deng, Z., Xu, Y., Tan, M.: Closed-loop matters: Dual regression networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2020)

  18. Haris, M., Shakhnarovich, G., Ukita, N.: Deep back-projection networks for super-resolution. In: IEEE conference on CVPR, pp. 1664–1673 (2018)

  19. Hayat, K.: Super-resolution via deep learning. arXiv preprint arXiv:1706.09077 (2017)

  20. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE conference on CVPR, pp. 770–778 (2016)

  21. Hore, A., Ziou, D.: Image quality metrics: Psnr vs. ssim. In: International conference on Pattern recognition (icpr), pp. 2366–2369. IEEE (2010)

  22. Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. IEEE Conference on CVPR pp. 2261–2269 (2017)

  23. Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: IEEE Conference on CVPR, pp. 5197–5206 (2015)

  24. Huang, Y., Li, J., Gao, X., Hu, Y., Lu, W.: Interpretable detail-fidelity attention network for single image super-resolution. IEEE Trans. Image Process. 30, 2325–2339 (2021)

    Article  Google Scholar 

  25. Hui, Z., Gao, X., Yang, Y., Wang, X.: Lightweight image super-resolution with information multi-distillation network. In: Proceedings of the 27th ACM International Conference on Multimedia (ACM MM), pp. 2024–2032 (2019)

  26. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: IEEE Conference on CVPR, pp. 1646–1654 (2016)

  27. Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. IEEE Conference on CVPR pp. 1637–1645 (2016)

  28. Kim, K.I., Kwon, Y.: Single-image super-resolution using sparse regression and natural image prior. IEEE Trans. Pattern Anal. Mach. Intell. 32(6), 1127–1133 (2010)

    Article  Google Scholar 

  29. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  30. Lai, W.S., Huang, J.B., Ahuja, N., Yang, M.H.: Deep laplacian pyramid networks for fast and accurate super-resolution. In: IEEE Conference on CVPR (2017)

  31. Lan, R., Sun, L., Liu, Z., Lu, H., Su, Z., Pang, C., Luo, X.: Cascading and enhanced residual networks for accurate single-image super-resolution. IEEE Transactions on Cybernetics pp. 1–11 (2020)

  32. Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: IEEE conference on CVPR, pp. 4681–4690 (2017)

  33. Li, J., Fang, F., Mei, K., Zhang, G.: Multi-scale residual network for image super-resolution. In: ECCV, pp. 517–532 (2018)

  34. Li, X., Sun, Y., Yang, Y., Miao, C.: Symmetrical residual connections for single image super-resolution. ACM Tran. Multimed. Comput. Commun. Appl. (TOMM) 15(1), 1–10 (2019)

    Article  Google Scholar 

  35. Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. IEEE Conference on CVPR Workshops pp. 1132–1140 (2017)

  36. Lin, Z., Shum, H.Y.: Fundamental limits of reconstruction-based superresolution algorithms under local translation. IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 83–97 (2004)

    Article  Google Scholar 

  37. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: IEEE Conference on ICCV, vol. 2, pp. 416–423. IEEE (2001)

  38. Nasrollahi, K., Moeslund, T.B.: Super-resolution: a comprehensive survey. MVA 25(6), 1423–1468 (2014)

    Google Scholar 

  39. Park, S.J., Son, H., Cho, S., Hong, K.S., Lee, S.: Srfeat: Single image super-resolution with feature discrimination. In: ECCV, pp. 439–455 (2018)

  40. Peleg, T., Elad, M.: A statistical prediction model based on sparse representations for single image super-resolution. IEEE Trans. Image Process. 23(6), 2569–2582 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  41. Prajapati, K., Chudasama, V., Upla, K.: A light weight convolutional neural network for single image super-resolution. Proc. Comput. Sci. 171, 139–147 (2020)

    Article  Google Scholar 

  42. Sajjadi, M.S., Schölkopf, B., Hirsch, M.: Enhancenet: Single image super-resolution through automated texture synthesis. In: IEEE Conference on ICCV, pp. 4501–4510. IEEE (2017)

  43. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., Wang, Z.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: IEEE Conference on CVPR, pp. 1874–1883 (2016)

  44. Shi, W., Du, H., Mei, W., Ma, Z.: (sarn)spatial-wise attention residual network for image super-resolution. Vis. Comput. (2020). https://doi.org/10.1007/s00371-020-01903-8

    Article  Google Scholar 

  45. Singh, A., Porikli, F., Ahuja, N.: Super-resolving noisy images. In: IEEE Conference on CVPR, pp. 2846–2853 (2014)

  46. Soh, J.W., Park, G.Y., Jo, J., Cho, N.I.: Natural and realistic single image super-resolution with explicit natural manifold discrimination. In: IEEE Conference on CVPR, pp. 8122–8131 (2019)

  47. Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In: IEEE Conference on CVPR, vol. 1 (2017)

  48. Timofte, R., Agustsson, E., Van Gool, L., Yang, M.H., Zhang, L.: Ntire 2017 challenge on single image super-resolution: Methods and results. In: IEEE Conference on CVPR Workshops, pp. 114–125 (2017)

  49. Timofte, R., De, V., Van Gool, L.: Anchored neighborhood regression for fast example-based super-resolution. In: IEEE Conference on ICCV, pp. 1920–1927. IEEE (2013)

  50. Tong, T., Li, G., Liu, X., Gao, Q.: Image super-resolution using dense skip connections. In: IEEE Conference on ICCV, pp. 4809–4817. IEEE (2017)

  51. Trottier, L., Gigu, P., Chaib-draa, B., et al.: Parametric exponential linear unit for deep convolutional neural networks. In: IEEE International Conference on Machine Learning and Applications, pp. 207–214. IEEE (2017)

  52. Tsai, R.Y., Huang, T.S.: Multiframe image restoration and registration. In: Advances in computer vision and image processing, pp. 317–339 (1984)

  53. Wang, C., Li, Z., Shi, J.: Lightweight image super-resolution with adaptive weighted learning network. arXiv preprint arXiv:1904.02358 (2019)

  54. Wang, Y., Perazzi, F., McWilliams, B., Sorkine-Hornung, A., Sorkine-Hornung, O., Schroers, C.: A fully progressive approach to single-image super-resolution. In: IEEE Conference on CVPR Workshops (2018)

  55. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  56. Yang, C.Y., Ma, C., Yang, M.H.: Single-image super-resolution: A benchmark. In: ECCV, pp. 372–386. Springer (2014)

  57. Yang, J., Lin, Z., Cohen, S.: Fast image super-resolution based on in-place example regression. In: IEEE Conference on CVPR, pp. 1059–1066. IEEE (2013)

  58. Yang, W., Zhang, X., Tian, Y., Wang, W., Xue, J.H., Liao, Q.: Deep learning for single image super-resolution: A brief review. IEEE Trans. Multimed. (2019)

  59. Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: International conference on curves and surfaces, pp. 711–730. Springer (2010)

  60. Zhang, D., Shao, J., Shen, H.T.: Kernel attention network for single image super-resolution. ACM Trans. Multimed. Comput. Commun. Appl. (TOMM) 16(3), 1–15 (2020)

    Article  Google Scholar 

  61. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: ECCV, pp. 286–301 (2018)

  62. Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: IEEE Conference on CVPR, pp. 2472–2481 (2018)

Download references

Funding

Not Applicable

Author information

Authors and Affiliations

  1. Sardar Vallabhbhai National Institute of Technology (SVNIT), Surat, India

    Vishal Chudasama & Kishor Upla

  2. Norwegian University of Science and Technology (NTNU), Gjøvik, Norway

    Kiran Raja, Raghavendra Ramachandra & Christoph Busch

Authors
  1. Vishal Chudasama
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kishor Upla
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kiran Raja
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Raghavendra Ramachandra
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Christoph Busch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kishor Upla.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Code availability

Yes (https://github.com/Vishal2188/Compact-and-Progressive-Network-for-Enhanced-SIS R---ComPrESRNet)

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chudasama, V., Upla, K., Raja, K. et al. Compact and progressive network for enhanced single image super-resolution—ComPrESRNet. Vis Comput 38, 3643–3665 (2022). https://doi.org/10.1007/s00371-021-02193-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02193-4

Keywords

Search

Navigation