Skip to main content
Log in

Gated normalization unit for image restoration

  • Original Article
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Image restoration has been an integral part of image processing research with the goal of converting degraded images into clear ones. While some networks have achieved state-of-the-art results through architecture and module design, little attention has been paid to the adaptation of normalization methods in image restoration tasks. Normalization methods are crucial in deep learning. In this work, we attempt to combine gating mechanisms with normalization methods. Gated mechanisms are popular in feature extraction and information filtering, and combining them with normalization methods has potential for designing image restoration algorithms. Firstly, we propose a Simple Gated Attention Unit (SGAU), a block using a simple gating mechanism to validate the potential of gating mechanisms. Then, we propose a new normalization block, Gated Instance Normalization (GIN), and introduce a new normalization method, Global Response Normalization (GRN), for image restoration tasks. Both GIN and GRN combine gating mechanisms with normalization methods for feature extraction, fusion, and integration. Finally, we propose a two-stage network, Gated Normalization Network (GNNet), utilizing GIN and GRN as blocks to effectively extract and filter information. Deep separable convolutions are used in the deep layers to reduce parameters while preserving spatial information, improving local feature perception. An improved cross-stage feature fusion (ICSFF) block is used for feature information transfer between stages, and a supervised attention module (SAM) is used as input to the second stage network from the first stage output. Through various image restoration tasks, we achieve 32.93 dB PSNR on GoPro, 30.42 dB PSNR on HIDE for image deblurring, 39.94 dB PSNR on SIDD for real-world denoising, and good performance in Gaussian white noise denoising and image deraining tasks. Moreover, the GIN and GRN only generated a small number of gated weight and bias parameters, and compared to other multi-stage networks, the model size is reduced, and computational complexity is well balanced.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Algorithm 1
Algorithm 2
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

The data presented in this study are available on request from the first author.

References

  1. Ye J, Zhang Y (2019) MAP-based image denoising with structured sparsity and Gaussian scale mixture. Pattern Anal Appl 22:965–977. https://doi.org/10.1007/s10044-018-0692-5

    Article  MathSciNet  MATH  Google Scholar 

  2. Su Z, Wenbo W, Zhang W (2023) Regularized denoising latent subspace based linear regression for image classification. Pattern Anal Appl 2023:1–18. https://doi.org/10.1007/s10044-023-01149-9

    Article  MATH  Google Scholar 

  3. Joseph Raj AN, Junmin C, Nersisson R et al (2022) Bilingual text detection from natural scene images using faster R-CNN and extended histogram of oriented gradients. Pattern Anal Appl 25(4):1001–1013. https://doi.org/10.1007/s10044-022-01066-3

    Article  Google Scholar 

  4. Zhang K, Zuo W, Zhang L (2018) FFDNet: Toward a fast and flexible solution for CNN-based image denoising. IEEE Trans Image Process 27(9):4608–4622. https://doi.org/10.1109/TIP.2018.2839891

    Article  MathSciNet  MATH  Google Scholar 

  5. Zhang Y, Tian Y, Kong Y et al (2020) Residual dense network for image restoration. IEEE Trans Pattern Anal Mach Intell 43(7):2480–2495

    Article  MATH  Google Scholar 

  6. Zamir SW, Arora A, Khan S et al (2021) Multi-stage progressive image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 14821–14831

  7. Tsai FJ, Peng YT, Tsai CC et al (2022) Banet: a blur-aware attention network for dynamic scene deblurring. IEEE Trans Image Process 31:6789–6799. https://doi.org/10.1109/TIP.2022.3216216

    Article  MATH  Google Scholar 

  8. Mei Y, Fan Y, Zhang Y et al (2023) Pyramid Attention Network for Image Restoration. Int J Comput Vision 2023:1–19. https://doi.org/10.1007/s11263-023-01843-5

    Article  MATH  Google Scholar 

  9. Anwar S, Barnes N (2020) Densely residual laplacian super-resolution. IEEE Trans Pattern Anal Mach Intell 44(3):1192–1204. https://doi.org/10.1109/TPAMI.2020.3021088

    Article  MATH  Google Scholar 

  10. Zhou J, Meng M, Xing J et al (2021) Iterative feature refinement with network-driven prior for image restoration. Pattern Anal Appl 24:1623–1634

    Article  MATH  Google Scholar 

  11. Zhang K, Zuo W, Chen Y et al (2017) Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Trans Image Process 26(7):3142–3155

    Article  MathSciNet  MATH  Google Scholar 

  12. Khan S, Naseer M, Hayat M et al (2022) Transformers in vision: A survey. ACM Comput Surv (CSUR) 54(10s):1–41. https://doi.org/10.1145/3505244

    Article  MATH  Google Scholar 

  13. Liu Z, Lin Y, Cao Y et al (2021) Swin transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 10012–10022

  14. Liang J, Cao J, Sun G et al (2021) Swinir: Image restoration using swin transformer. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1833–1844

  15. Zamir SW, Arora A, Khan S et al (2022) Restormer: Efficient transformer for high-resolution image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5728–5739

  16. Wang Z, Cun X, Bao J et al (2022) Uformer: A general u-shaped transformer for image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 17683–17693

  17. Xue T, Ma P (2023) TC-net: transformer combined with cnn for image denoising. Appl Intell 53(6):6753–6762. https://doi.org/10.1007/s10489-022-03785-w

    Article  MATH  Google Scholar 

  18. Dauphin YN, Fan A, Auli M et al (2017) Language modeling with gated convolutional networks. In: International conference on machine learning, (PMLR), pp 933–941

  19. Kupyn O, Martyniuk T, Wu J et al (2019) Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 8878–8887

  20. Pan J, Dong J, Liu Y et al (2020) Physics-based generative adversarial models for image restoration and beyond. IEEE Trans Pattern Anal Mach Intell 43(7):2449–2462. https://doi.org/10.1109/TPAMI.2020.2969348

    Article  MATH  Google Scholar 

  21. Chen L, Lu X, Zhang J et al (2021) Hinet: Half instance normalization network for image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 182–192

  22. Yue Z, Zhao Q, Zhang L et al (2020) Dual adversarial network: Toward real-world noise removal and noise generation. Computer Vision–ECCV 2020. Springer International Publishing, Cham, pp 41–58

    Chapter  Google Scholar 

  23. Zhang K, Li Y, Zuo W et al (2021) Plug-and-play image restoration with deep denoiser prior. IEEE Trans Pattern Anal Mach Intell 44(10):6360–6376

    Article  MATH  Google Scholar 

  24. Chen L, Chu X, Zhang X et al (2022) Simple baselines for image restoration. Computer Vision–ECCV 2022. Springer International Publishing, Cham, pp 17–33

    Chapter  MATH  Google Scholar 

  25. Ko K, Koh YJ, Kim CS (2022) Blind and compact denoising network based on noise order learning. In: IEEE Trans Image Process, pp 1657–1670

  26. Li J, Yang H, Yi Q et al (2022) Multiple degradation and reconstruction network for single image denoising via knowledge distillation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 558–567

  27. Tian C, Zheng M, Zuo W et al (2024) A cross Transformer for image denoising. Inform Fusion 102:102043

    Article  MATH  Google Scholar 

  28. Tu Z, Talebi H, Zhang H et al (2022) Maxim: Multi-axis mlp for image processing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5769–5780

  29. Hua W, Dai Z, Liu H et al (2022) Transformer quality in linear time. In: International conference on machine learning, (PMLR), pp 9099–9117

  30. Muthusamy D, Sathyamoorthy S (2023) Feature Sampling based on Multilayer Perceptive Neural Network for image quality assessment. Eng Appl Artif Intell 121:106015

    Article  MATH  Google Scholar 

  31. Muthusamy D, Sathyamoorthy S (2022) Deep belief network for solving the image quality assessment in full reference and no reference model. Neural Comput Appl 34(24):21809–21833

    Article  MATH  Google Scholar 

  32. Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1251–1258

  33. Woo S, Debnath S, Hu R et al (2023) Convnext v2: Co-designing and scaling convnets with masked autoencoders. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 16133–16142

  34. Chu X, Chen L, Chen C et al (2022) Improving image restoration by revisiting global information aggregation. Computer Vision–ECCV 2022. Springer International Publishing, Cham, pp 53–71

    Chapter  MATH  Google Scholar 

  35. Zhang H, Sindagi V, Patel VM (2019) Image de-raining using a conditional generative adversarial network. IEEE Trans Circuits Syst Video Technol 30(11):3943–3956. https://doi.org/10.1109/TCSVT.2019.2920407

    Article  MATH  Google Scholar 

  36. Yang W, Tan RT, Feng J et al (2017) Deep joint rain detection and removal from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1357–1366

  37. Fu X, Huang J, Zeng D et al (2017) Removing rain from single images via a deep detail network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3855–3863

  38. Zhang H, Patel VM (2018) Density-aware single image de-raining using a multi-stream dense network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 695–704

  39. Martin D, Fowlkes C, Tal D et al (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistic. In: Proceedings Eighth IEEE International Conference on Computer Vision. pp 416–423

  40. Timofte R, Agustsson E, Van Gool L et al (2017) Ntire 2017 challenge on single image super-resolution: Methods and results. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops. pp 114–125

  41. Agustsson E, Timofte R (2017) Ntire 2017 challenge on single image super-resolution: Dataset and study. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops. pp 126–135

  42. Ma K, Duanmu Z, Wu Q et al (2016) Waterloo exploration database: New challenges for image quality assessment models. IEEE Trans Image Process 26(2):1004–1016

    Article  MathSciNet  MATH  Google Scholar 

  43. Roth S, Black MJ (2005) Fields of experts: A framework for learning image priors. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2: 860–867

  44. Huang JB, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 5197–5206

  45. Franzen R (1999) Kodak lossless true color image suite. source: http://r0k.us/graphics/kodak

  46. Abdelhamed A, Lin S, Brown MS (2018) A high-quality denoising dataset for smartphone cameras. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1692–1700

  47. Nah S, Hyun Kim T, Mu Lee K (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3883–3891

  48. Shen Z, Wang W, Lu X et al (2019) Human-aware motion deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5572–5581

  49. Fu X, Huang J, Ding X et al (2017) Clearing the skies: A deep network architecture for single-image rain removal. IEEE Trans Image Process 26(6):2944–2956. https://doi.org/10.1109/TIP.2017.2691802

    Article  MathSciNet  MATH  Google Scholar 

  50. Jiang K, Wang Z, Yi P et al (2020) Multi-scale progressive fusion network for single image deraining. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 8346–8355

  51. Anwar S, Barnes N, Petersson L (2021) Attention-based real image restoration. IEEE Transactions on Neural Networks and Learning Systems

    MATH  Google Scholar 

  52. Cho SJ, Ji SW, Hong JP et al (2021) Rethinking coarse-to-fine approach in single image deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4641–4650

  53. Cheng S, Wang Y, Huang H et al (2021) Nbnet: Noise basis learning for image denoising with subspace projection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4896–4906

  54. Gu S, Zhang L, Zuo W et al (2014) Weighted nuclear norm minimization with application to image Denoising. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2862–2869

  55. Zhang K, Zuo W, Gu S et al (2017) Learning deep CNN denoiser prior for image Restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3929–3938

  56. Li L, Pan J, Lai WS et al (2020) Dynamic scene deblurring by depth guided model. IEEE Trans Image Process 29:5273–5288

    Article  MATH  Google Scholar 

  57. Zhang K, Luo W, Zhong Y et al (2020) Deblurring by realistic blurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2737–2746

  58. Cai J, Zuo W, Zhang L (2020) Dark and bright channel prior embedded network for dynamic scene deblurring. IEEE Trans Image Process 29:6885–6897

    Article  MATH  Google Scholar 

  59. Purohit K, Suin M, Rajagopalan AN et al (2021) Spatially-adaptive image restoration using distortion-guided networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2309–2319

  60. Su J, Xu B, Yin H (2022) A survey of deep learning approaches to image restoration. Neurocomputing 487:46–65

    Article  MATH  Google Scholar 

Download references

Funding

This work was funded by the Natural Science Research of Jiangsu Higher Education Institutions of China (Grants No. 22KJD140004) and 2022 Qixia District Key R&D Project (Grants No. ZY202224).

Author information

Authors and Affiliations

Authors

Contributions

Qingyu Wang, Haitao Wang, Luyang Zhang, and Yi Jiang wrote the main manuscript text.Xinyao Wang and Qiang Liu prepared Figs. 1, 2, 3, 4, 5, 6, 7, 8 and 9. Dehai Huang and Binding Hu prepared tables.All authors reviewed the manuscript.

Corresponding author

Correspondence to Haitao Wang.

Ethics declarations

Competing interests

The authors declare no competing interests.

Conflict of interest

We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Ethical approval

This declaration is not applicable.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Q., Wang, H., Zang, L. et al. Gated normalization unit for image restoration. Pattern Anal Applic 28, 16 (2025). https://doi.org/10.1007/s10044-024-01393-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10044-024-01393-7

Keywords

Navigation