Abstract
Single image super-resolution (SISR) has achieved great progress based on convolutional neural networks (CNNs) such as generative adversarial network (GAN). However, most deep learning architectures cannot utilize the hierarchical features in original low-resolution images, which may result in the loss of image details. To recover visually high-quality high-resolution images, we propose a novel Multi-recursive residual dense Attention Generative Adversarial Network (MAGAN). Our MAGAN enjoys the ability to learn more texture details and overcome the weakness of conventional GAN-based models, which easily generate redundant information. In particular, we design a new multi-recursive residual dense network as a module in our generator to take advantage of the information from hierarchical features. We also introduce a multi-attention mechanism to our MAGAN to capture more informative features. Moreover, we present a new convolutional block in our discriminator by utilizing switchable normalization and spectral normalization to stabilize the training and accelerate convergence. Experimental results on benchmark datasets indicate that MAGAN yields finer texture details and does not produce redundant information in comparison with existing methods.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Yang, W., Zhang, X., Tian, Y., Wang, W., Xue, J., Liao, Q.: Deep learning for single image super-resolution: a brief review. IEEE Trans. Multimedia 21(12), 3106–3121 (2019)
Zhang, K., Gao, X., Tao, D., Li, X.: Single image super-resolution with non-local means and steering kernel regression. IEEE Trans. Image Process. 21(11), 4544–4556 (2012)
Zhang, L., Wu, X.: An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans. Image Process. 15(8), 2226–2238 (2006)
Hsu, J., Kuo, C., Chen, D.: Image super-resolution using capsule neural networks. IEEE Access 8, 9751–9759 (2020)
Shi, Y., Li, S., Li, W., Liu, A.: Fast and lightweight image super-resolution based on dense residuals two-channel network. In. IEEE International Conference on Image Processing (ICIP) 2019, pp. 2826–2830 (2019)
Haris, M., Shakhnarovich, G., Ukita, N.: Deep back-projection networks for super-resolution. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1664–1673 (2018)
Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_13
Lim, B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 1132–1140 (2017)
Shi, W., et al.: Is the deconvolution layer the same as a convolutional layer? arXiv:1609.07009 (2016)
Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2472–2481 (2018)
Hu, Y., Gao, X., Li, J., Huang, Y., Wang, H.: Single image super-resolution with multi-scale information cross-fusion network. Signal Process. 179, 107831 (2021)
Chang, K., Li, M., Ding, P.L.K., Li, B.: Accurate single image super-resolution using multi-path wide-activated residual network. Signal Process. 172, 107567 (2020)
Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 105–114 (2017)
Sajjadi, M.S.M., Scholkopf, B., Hirsch, M.: EnhanceNet: single image super-resolution through automated texture synthesis. In: The IEEE International Conference on Computer Vision (ICCV), pp. 4501–4510 (2017)
Wang, X., et al.: ESRGAN: enhanced super-resolution generative adversarial networks. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11133, pp. 63–79. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11021-5_5
Soh, J.W., Park, G.Y., Jo, J., Cho, N.I.: Natural and realistic single image super-resolution with explicit natural manifold discrimination. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Goodfellow, I., et al.: Generative adversarial nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 2672–2680. Curran Associates Inc. (2014)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43
Wang, X., Yu, K., Dong, C., Change Loy, C.: Recovering realistic texture in image super-resolution by deep spatial feature transform. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 606–615 (2018)
Jolicoeur-Martineau, A.: The relativistic discriminator: a key element missing from standard GAN. In: International Conference on Learning Representations (2019)
Mechrez, R., Talmi, I., Zelnik-Manor, L.: The contextual loss for image transformation with non-aligned data. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision – ECCV 2018. LNCS, vol. 11218, pp. 800–815. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01264-9_47
Mechrez, R., Talmi, I., Shama, F., Zelnik-Manor, L.: Learning to maintain natural image statistics. arXiv: 1803.04626 (2018)
Wang, Z., Chen, J., Hoi, S.C.H.: Deep learning for image super-resolution: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 43, 3365–3387 (2020)
Bachlechner, T., Majumder, B.P., Mao, H.H., Cottrell, G.W., McAuley, J.: ReZero is all you need: Fast convergence at large depth. arXiv:2003.04887 (2020)
Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: CBAM: convolutional block attention module. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 3–19. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_1
Luo, P., Ren, J., Peng, Z., Zhang, R., Li, J.: Differentiable learning-to-normalize via switchable normalization. In: International Conference on Learning Representations (ICLR) (2019)
Miyato, T., Kataoka, T., Koyama, M., Yoshida, Y.: Spectral normalization for generative adversarial networks. In: International Conference on Learning Representations (ICLR) (2018)
Odena, A., et al.: Is generator conditioning causally related to GAN performance? In: Proceedings of the 35th International Conference on Machine Learning (ICML), pp. 3849–3858 (2018)
Blau, Y., Mechrez, R., Timofte, R., Michaeli, T., Zelnik-Manor, L.: The 2018 PIRM challenge on perceptual image super-resolution. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11133, pp. 334–355. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11021-5_21
Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of the 30th International Conference on Machine Learning (ICML) (2013)
Agustsson, E., Timofte, R.: NTIRE 2017 challenge on single image super-resolution: dataset and study. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 2017, pp. 1122–1131 (2017)
Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: International Conference on Curves and Surfaces, pp. 711–730 (2012)
Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, pp. 416–423 (2001)
Huang, J., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2015, pp. 5197–5206 (2015)
Rad, M.S., Bozorgtabar, B., Marti, U.-V., Basler, M., Ekenel, H.K., Thiran, J.-P.: SROBB: targeted perceptual loss for single image super-resolution. In: The IEEE International Conference on Computer Vision (ICCV), 2019, pp. 2710–2719 (2019)
Zhu, X., Hu, H., Lin, S., Dai, J.: Deformable ConvNets V2: more deformable, better results. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9300–9308 (2019)
Liu, H., Zhao, P., Ruan, Z., Shang, F., Liu, Y.: Large motion video super-resolution with dual subnet and multi-stage communicated upsampling. In: Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI) (2021)
Liu, H., et al.: A single frame and multi-frame joint network for 360-degree panorama video super-resolution. arXiv Preprint arXiv:2008.10320 (2020)
Liu, H., Ruan, Z., Zhao, P., Shang, F., Yang, L., Liu, Y.: Video super resolution based on deep learning: a comprehensive survey. arXiv Preprint arXiv:2007.12928 (2022)
Acknowledgements
This work was supported by the National Natural Science Foundation of China (Nos. 61976164, 61876221, 61876220), and Natural Science Basic Research Program of Shaanxi (Program No. 2022GY-061).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 IFIP International Federation for Information Processing
About this paper
Cite this paper
Yang, L. et al. (2022). Multi Recursive Residual Dense Attention GAN for Perceptual Image Super Resolution. In: Shi, Z., Jin, Y., Zhang, X. (eds) Intelligence Science IV. ICIS 2022. IFIP Advances in Information and Communication Technology, vol 659. Springer, Cham. https://doi.org/10.1007/978-3-031-14903-0_39
Download citation
DOI: https://doi.org/10.1007/978-3-031-14903-0_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-14902-3
Online ISBN: 978-3-031-14903-0
eBook Packages: Computer ScienceComputer Science (R0)
