Skip to main content
SpringerLink
Log in

Multi-scale feature aggregation network for Image super-resolution

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

For single image super-resolution technology, Convolutional Neural Network (CNN) has an excellent capability to improve reconstruction results. However, the existing CNN-based SR methods maintain high-quality reconstruction with excessive amounts of parameters and very deep network structures, which not only leads to higher requirements for computational resource and memory storage but also makes it difficult to apply resource-constrained devices. To solve these problems, a multi-scale feature aggregation network (MFAN) is proposed in this paper. In the MFAN, global dual-path is put into use to forward the feature information in low resolution space and medium resolution space respectively. For the sake of enhancing the ability of MFAN feature extraction, we come up a with multi-scale feature extraction module (MSFE), which also contains a multi-path structure. This module is embedded in each global path for image feature extraction. Through the effective combination of global path and MSFE, multi-scale information is extracted and enhanced step by step. Moreover, we design an inter-scale feature projection module (ISFP) based on the back-projection mechanism to effectively integrate the multi-scale feature information from MSFE. Extensive experiments show that the proposed MFAN has achieved competitive results in the qualitative and quantitative evaluation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Zhou Y, Liu D, Huang T (2018) Survey of face detection on low-quality images. In: 13th IEEE international conference on automatic face & gesture recognition, pp 769–773

  2. Zhou Y, Ren D, Emerton N, Lim S, Large T (2020) Image restoration for under-display camera. arXiv:200304857

  3. Li X, Orchard M T (2001) New edge-directed interpolation. IEEE Trans Image Process 10 (10):1521–1527

    Article  Google Scholar 

  4. Wang Q, Tang X, Shum H (2005) Patch based blind image super resolution. In: The tenth IEEE international conference on computer vision, pp 709–716

  5. Zhang K, Gool LV, Timofte R (2020) Deep unfolding network for image super-resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3217–3226

  6. Dai T, Cai J, Zhang Y, Xia S-T, Zhang L (2019) Second-order attention network for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 11065–11074

  7. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: European conference on computer vision, pp 184–199

  8. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European conference on computer vision, pp 286–301

  9. Lan R, Sun L, Liu Z, Lu H, Su Z, Pang C, Luo X (2021) Cascading and enhanced residual networks for accurate single-image super-resolution. IEEE Trans Cybern 51(1):115–125

    Article  Google Scholar 

  10. Niu B, Wen W, Ren W, Zhang X, Yang L, Wang S, Zhang K, Cao X, Shen H (2020) Single Image Super-Resolution via a Holistic Attention Network. arXiv:200808767

  11. Tian C, Zhuge R, Wu Z, Xu Y, Zuo W, Chen C, Lin C -W (2020) Lightweight image super-resolution with enhanced CNN. Knowl-Based Syst 205:106235

    Article  Google Scholar 

  12. Li Z, Yang J, Liu Z, Yang X, Jeon G, Wu W (2019) Feedback network for image super-resolution. In: the Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3867–3876

  13. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European conference on computer vision, pp 391–407

  14. Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1646–1654

  15. Kim J, Kwon Lee J, Mu Lee K (2016) Deeply-recursive convolutional network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1637–1645

  16. Han W, Chang S, Liu D, Yu M, Witbrock M, Huang TS (2018) Image super-resolution via dual-state recurrent networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1654–1663

  17. Wang Y, Perazzi F, McWilliams B, Sorkine-Hornung A, Sorkine-Hornung O, Schroers C (2018) A fully progressive approach to single-image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 864–873

  18. Jiang K, Wang Z, Yi P, Jiang J (2020) Hierarchical dense recursive network for image Super-Resolution. Pattern Recogn 107:107475

    Article  Google Scholar 

  19. Wang Z, Chen J, Hoi S (2020) Deep Learning for Image Super-resolution: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence

  20. Lan R, Sun L, Liu Z, Lu H, Pang C, Luo X (2021) MADNEt: a fast and lightweight network for single-image super resolution. IEEE Trans Cybern 51(3):1443–1453

    Article  Google Scholar 

  21. Guo Y, Chen J, Wang J, Chen Q, Cao J, Deng Z, Xu Y, Tan M (2020) Closed-loop Matters: Dual Regression Networks for Single Image Super-Resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5407–5416

  22. Haris M, Shakhnarovich G, Ukita N (2018) Deep back-projection networks for super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1664–1673

  23. Zhang W, Liu Y, Dong C, Qiao Y (2019) Ranksrgan: Generative adversarial networks with ranker for image super-resolution. In: Proceedings of the IEEE international conference on computer vision, pp 3096–3105

  24. Zhang Z., Wang Z., Lin Z., Qi H. (2019) Image super-resolution by neural texture transfer. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7982– 7991

  25. Tian C, Xu Y, Zuo W, Zhang B, Fei L, Lin C -W (2020) Coarse-to-fine CNN for image super-resolution. IEEE Trans Multimed 23:1489–1502

    Article  Google Scholar 

  26. Feng R, Gu J, Qiao Y, Dong C (2019) Suppressing model overfitting for image super-resolution networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 0–0

  27. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4681–4690

  28. Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, Qiao Y, Change Loy C (2018) Esrgan: Enhanced super-resolution generative adversarial networks. In: Proceedings of the European conference on computer vision, pp 0–0

  29. Lai W-S, Huang J-B, Ahuja N, Yang M-H (2017) Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 624-632

  30. Li J, Fang F, Mei K, Zhang G (2018) Multi-scale residual network for image super-resolution. In: Proceedings of the European conference on computer vision, pp 517–532

  31. Feng X, Li X, Li J (2020) Multi-scale fractal residual network for image super-resolution. Appl Intell 51(4):1845–1856

    Article  Google Scholar 

  32. Zhao H, Kong X, He J, Qiao Y, Dong C (2020) Efficient Image Super-Resolution Using Pixel Attention. arXiv:2010.01073

  33. Agustsson E, Timofte R (2017) Ntire 2017 challenge on single image super-resolution: Dataset and study. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 126–135

  34. Bevilacqua M, Guillemot C, Alberi-Morel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: 23rd British machine vision conference

  35. Yang J, Wright J, Huang T S, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  Google Scholar 

  36. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings eighth IEEE international conference on computer vision, pp 416–423

  37. Huang J-B, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5197–s5206

  38. Wang Z, Liu D, Yang J, Han W, Huang T (2015) Deep networks for image super-resolution with sparse prior. In: Proceedings of the IEEE international conference on computer vision, pp 370–378

  39. Lai W-S, Huang J-B, Ahuja N, Yang M-H (2018) Fast and accurate image super-resolution with deep laplacian pyramid networks. IEEE Trans Pattern Anal Mach Intell 41(11):2599–2613

    Article  Google Scholar 

  40. Tong T, Li G, Liu X, Gao Q (2017) Image super-resolution using dense skip connections. In: Proceedings of the IEEE international conference on computer vision, pp 4799–4807

  41. Lim B, Son S, Kim H, Nah S, Mu Lee K (2017) Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 136–144

Download references

Acknowledgements

This work is supported by the Special Project on Basic Research of Frontier Leading Technology of Jiangsu Province of China (Grant Nos. BK20192004C) and Natural Science Foundation of Jiangsu Province of China (Grant Nos. BK20181269). A Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions. Shenzhen Science and Technology Innovation Commission (STIC) under Grant JCYJ20180306174455080.

Author information

Authors and Affiliations

  1. School of Automation, Southeast University, Nanjing, 210096, China

    Wenlong Chen, Pengcheng Yao, Shaoyan Gai & Feipeng Da

  2. Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of Education, Nanjing, Jiangsu, 210096, China

    Wenlong Chen, Pengcheng Yao, Shaoyan Gai & Feipeng Da

  3. Shenzhen Research Institute, Southeast University, Shenzhen, Guangdong, 518063, China

    Feipeng Da

Authors
  1. Wenlong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pengcheng Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shaoyan Gai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Feipeng Da
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Shaoyan Gai or Feipeng Da.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, W., Yao, P., Gai, S. et al. Multi-scale feature aggregation network for Image super-resolution. Appl Intell 52, 3577–3586 (2022). https://doi.org/10.1007/s10489-021-02593-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02593-y

Keywords

Search

Navigation