Skip to main content
SpringerLink
Log in

Improved edge-guided network for single image super-resolution

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In recent years, deep learning has been successfully applied to image super-resolution. It is still a challenge to reconstruct high-frequency details from low-resolution images. However, many works lack attention to the high-frequency part. We find that edge prior information can be used to extract high-frequency parts and applying soft edges to image reconstruction has achieved great results. Inspired by this, we focus on how to make full use of edge information to generate high-frequency details. We propose an improved edge-guided neural network for single image super-resolution (IEGSR), which makes full use of the edge prior information to reconstruct images with more abundant high-frequency information. For high-frequency information, we propose an edge-net to generate image edges better. For low-frequency information, we propose a global and local feature extraction module (GLM) to reconstruct the texture details. For the fusion of high-frequency information and low-frequency information, we propose a progressive fusion method, which can greatly reduce the number of parameters. Extensive experimental results demonstrate that our method can obtain images with sharper details. Applying our model to the Manga109 test set, the PSNR value of 4 times image super-resolution is as high as 39.02.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Agustsson E, Timofte R (2017) NTIRE 2017 challenge on single image super-resolution: Dataset and study. In computer vision and pattern recognition workshops, pp. 1122–1131

    Google Scholar 

  2. Almalioglu Y, Ozyoruk KB, Gokce A, Incetan K, Gokceler GI, Simsek MA, Ararat K, Chen RJ, Durr NJ, Mahmood F, Turan M (2020) Endol2h: Deep super-resolution for capsule endoscopy. IEEE Trans Med Imaging 39(12):4297–4309

    Article  Google Scholar 

  3. Anagün Y, Isik S, Seke E (2019) Srlibrary: Comparing different loss functions for super-resolution over various convolutional architectures. J Vis Commun Image Represent 61:178–187

    Article  Google Scholar 

  4. Arbelaez P, Maire M, Fowlkes CC, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916

    Article  Google Scholar 

  5. Bevilacqua M, Roumy A, Guillemot C, Alberimorel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In british machine vision conference, pp. 1–10

  6. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In european conference on computer vision, pp. 184–199

  7. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In european conference on computer vision, pp. 391–407

  8. Dong W, Zhang L, Shi G, Li X (2013) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process. 22(4):1620–1630

    Article  MathSciNet  Google Scholar 

  9. Fang F, Li J, Zeng T (2020) Soft-edge assisted network for single image super-resolution. IEEE Trans Image Process 29:4656–4668

    Article  Google Scholar 

  10. Haris M, Shakhnarovich G, Ukita N (2018) Deep back-projection networks for super-resolution. In computer vision and pattern recognition, pp. 1664–1673

  11. Hayat K (2018) Multimedia super-resolution via deep learning: A survey. Digital Signal Process 81:198–217

    Article  Google Scholar 

  12. Huang J, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In computer vision and pattern recognition, pp. 5197–5206

  13. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In computer vision and pattern recognition, pp. 1646–1654

  14. Kim J, Lee JK, Lee KM (2016) Deeply-recursive convolutional network for image super-resolution. In computer vision and pattern recognition, pp. 1637–1645

  15. Kim K, Chun SY (2018) Sredgenet: Edge enhanced single image super resolution using dense edge detection network and feature merge network. arXiv: Computer Vision and Pattern Recognition

  16. Lai W, Huang J, Ahuja N, Yang M (2017) Deep laplacian pyramid networks for fast and accurate super-resolution. In computer vision and pattern recognition, pp. 5835–5843

  17. Lai W, Huang J, Ahuja N, Yang M (2019) Fast and accurate image super-resolution with deep laplacian pyramid networks. IEEE Trans Pattern Anal Mach Intell 41(11):2599–2613

    Article  Google Scholar 

  18. Ledig C, Theis L, Huszr F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network. In computer vision and pattern recognition, pp. 105–114

  19. Lee J, Lee J, Yoo HJ (2020) Srnpu: An energy-efficient cnn-based super-resolution processor with tile-based selective super-resolution in mobile devices. IEEE J Emerging Sel Top Circuits Syst 10(3):320–334

    Article  Google Scholar 

  20. Li F, Bai H, Zhao L, Zhao Y (2018) Dual-streams edge driven encoder-decoder network for image super-resolution. IEEE Access 6:33421–33431

    Article  Google Scholar 

  21. Li J, Fang F, Mei K, Zhang G (2018) Multi-scale residual network for image super-resolution. In european conference on computer vision, pp. 527–542

  22. Li J, Yuan Y, Mei K, Fang F (2019) Lightweight and accurate recursive fractal network for image super-resolution. In international conference on computer vision, pp. 0–0

  23. Li Z, Yang J, Liu Z, Yang X, Jeon G, Wu W (2019) Feedback network for image super-resolution. In computer vision and pattern recognition, pp. 3867–3876

  24. Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In computer vision and pattern recognition, pp. 1132–1140

  25. Liu H, Fu Z, Han J, Shao L, Hou S, Chu Y (2019) Single image super-resolution using multi-scale deep encoder-decoder with phase congruency edge map guidance. Inform Sci 473:44–58

    Article  MathSciNet  Google Scholar 

  26. Liu J, Zhang W, Tang Y, Tang J, Wu G (2020) Residual feature aggregation network for image super-resolution. In computer vision and pattern recognition, pp. 2356–2365

  27. Ma J, Yu J, Liu S, Chen L, Li X, Feng J, Chen Z, Zeng S, Liu X, Cheng S (2020) Pathsrgan: Multi-supervised super-resolution for cytopathological images using generative adversarial network. IEEE Trans Med Imaging 39(9):2920–2930

    Article  Google Scholar 

  28. Matsui Y, Ito K, Aramaki Y, Yamasaki T, Aizawa K (2017) Sketch-based manga retrieval using manga109 dataset. Multimed Tools Appl 76(20):21811–21838

    Article  Google Scholar 

  29. Niu B, Wen W, Ren W, Zhang X, Yang L, Wang S, Zhang K, Cao X, Shen H (2020) Single image super-resolution via a holistic attention network. In european conference on computer vision, 12357:191–207

    Google Scholar 

  30. Noh J, Bae W, Lee W, Seo J, Kim G (2019) Better to follow, follow to be better: Towards precise supervision of feature super-resolution for small object detection. In international conference on computer vision, pp. 9724–9733

  31. Pang Y, Cao J, Wang J, Han J (2019) Jcs-net: Joint classification and super-resolution network for small-scale pedestrian detection in surveillance images. IEEE Trans Inf Forensics Secur 14(12):3322–3331

    Article  Google Scholar 

  32. Shamsolmoali P, Zareapoor M, Jain DK, Jain VK, Yang J (2019) Deep convolution network for surveillance records super-resolution. Multimed Tools Appl 78(17):23815–23829

    Article  Google Scholar 

  33. Shi W, Caballero J, Huszar F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In computer vision and pattern recognition, pp. 1874–1883

  34. Shocher A, Cohen N, Irani M (2018) Zero-shot super-resolution using deep internal learning. In computer vision and pattern recognition, pp. 3118–3126

  35. Tai Y, Yang J, Liu X, Xu C (2017) Memnet: A persistent memory network for image restoration. In international conference on computer vision, pp. 4549–4557

  36. Toutounchi F, Izquierdo E (2018) Enhancing digital zoom in mobile phone cameras by low complexity super-resolution. In international conference on multimedia expo workshops, pp. 01–06

  37. Wang C, Li Z, Shi J (2019) Lightweight image super-resolution with adaptive weighted learning network

  38. Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, Qiao Y, Loy CC (2018) ESRGAN: enhanced super-resolution generative adversarial networks. In european conference on computer vision, vol. 11133, pp. 63–79

    Google Scholar 

  39. Wang Y, Perazzi F, Mcwilliams B, Sorkinehornung A, Sorkinehornung O, Schroers C (2018) A fully progressive approach to single-image super-resolution. In computer vision and pattern recognition, pp. 977–97709

  40. Wen Z, Guan J, Zeng T, Li Y (2020) Residual network with detail perception loss for single image super-resolution. Comput Vis Image Underst 199:103007

  41. Xie J, Feris RS, Sun M (2016) Edge-guided single depth image super resolution. IEEE Trans Image Process 25(1):428–438

    Article  MathSciNet  Google Scholar 

  42. Yang W, Feng J, Yang J, Zhao F, Liu J, Guo Z, Yan S (2017) Deep edge guided recurrent residual learning for image super-resolution. IEEE Trans Image Process 26(12):5895–5907

    Article  MathSciNet  Google Scholar 

  43. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In international conference on curves and surfaces, pp. 711–730

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (61876099), in part by the National Key R&D Program of China (2019YFB1311001), in part by the Scientific and Technological Development Project of Shandong Province (2019GSF111002), in part by the Shenzhen Science and Technology Research and Development Funds (JCYJ20180305164401921), in part by the Foundation of Ministry of Education Key Laboratory of System Control and Information Processing (Scip201801), in part by the Foundation of Key Laboratory of Intelligent Computing & Information Processing of Ministry of Education (2018ICIP03), and in part by the Foundation of State Key Laboratory of Integrated Services Networks (ISN20-06). Jie Zhao and Zhenxue Chen contributed equally to this work and should be considered as the co-first authors.

Author information

Authors and Affiliations

  1. School of Control Science and Engineering, Shandong University, Jinan, 250061, China

    Jie Zhao, Zhenxue Chen, Xianming Li & Kai Zhu

  2. Department of Electrical and Computer Engineering, University of Windsor, Windsor, N9B 3P4, Canada

    Q. M. Jonathan Wu

  3. School of Artificial Intelligence, Henan Institute of Science and Technology, Xinxiang, 453003, China

    Lei Cai

Authors
  1. Jie Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhenxue Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Q. M. Jonathan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xianming Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lei Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kai Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenxue Chen.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Jie Zhao and Zhenxue Chen equally contributed to this work.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, J., Chen, Z., Wu, Q.M.J. et al. Improved edge-guided network for single image super-resolution. Multimed Tools Appl 81, 343–365 (2022). https://doi.org/10.1007/s11042-021-11429-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11429-3

Keywords

Search

Navigation