Skip to main content
SpringerLink
Log in

A nonlocal HEVC in-loop filter using CNN-based compression noise estimation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

High-efficiency video coding (HEVC) effectively reduces the amount of video data while unavoidably introducing compression noise. The in-loop filter can enhance the reconstructed frames at the encoder to prevent compression noise from transmitting to the subsequent frames. The existing incorporated in-loop filters for HEVC do not fully combine the advantages of CNN and video coding priors. Moreover, obtaining the accuracy noise level is essential for adaptively in-loop filtering. To further enhance the compressed video quality in the encoder, we propose a nonlocal in-loop filter for HEVC using a CNN-based compression noise estimation network(CNEN). In the noise estimation part, we adopt a classification network to estimate the noise of compressed HEVC videos according to video content characteristics. In the denoising part, we propose a spatial-temporal nonlocal low-rank(STNLLR) prior by simultaneously exploiting the nonlocal self-similarity of video in spatial and temporal domains. We also propose an adaptive narrow quantization constraint (ANQC) prior by limiting the reconstructed pixel values adaptively according to the quantization parameters(QPs). The experimental results show that CNEN outperforms the existing compression noise estimation methods. Furthermore, our in-loop filter can improve the quality of the HEVC reconstructed frames under AI, LDP, and RA configurations, achieving an average reduction of 4.17%, 10.46%, and 6.10% in the Bjøntegaard Delta Bit Rate (BDBR), respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Wiegand T, Sullivan G J, Bjontegaard G, Luthra A (2003) Overview of the h. 264/avc video coding standard. IEEE Trans Circ Syst Video Technolo 13(7):560–576. https://doi.org/10.1109/TCSVT.2003.815165

    Article  Google Scholar 

  2. Sullivan G J, Ohm J-R, Han W-J, Wiegand T (2012) Overview of the high efficiency video coding (hevc) standard. IEEE Trans Circ Syst Video Technol 22(12):1649–1668. https://doi.org/10.1109/TCSVT.2012.2221191

    Article  Google Scholar 

  3. Foi A, Katkovnik V, Egiazarian K (2007) Pointwise shape-adaptive dct for high-quality denoising and deblocking of grayscale and color images. IEEE Trans Image Process 16(5):1395–1411. https://doi.org/10.1109/TIP.2007.891788

    Article  MathSciNet  Google Scholar 

  4. Zhang J, Xiong R, Zhao C, Zhang Y, Ma S, Gao W (2016) Concolor: Constrained non-convex low-rank model for image deblocking. IEEE Trans Image Process 25(3):1246–1259. https://doi.org/10.1109/TIP.2016.2515985

    Article  MathSciNet  MATH  Google Scholar 

  5. Zhao C, Zhang J, Ma S, Fan X, Zhang Y, Gao W (2016) Reducing image compression artifacts by structural sparse representation and quantization constraint prior. IEEE Trans Circ Syst Video Technol 27(10):2057–2071. https://doi.org/10.1109/TCSVT.2016.2580399

    Article  Google Scholar 

  6. Zhang X, Xiong R, Lin W, Zhang J, Wang S, Ma S, Gao W (2016) Low-rank-based nonlocal adaptive loop filter for high-efficiency video compression. IEEE Trans Circ Syst Video Technol 27(10):2177–2188. https://doi.org/6.10.1109/TCSVT.2016.2581618

    Google Scholar 

  7. Zhang X, Lin W, Xiong R, Liu X, Ma S, Gao W (2016) Low-rank decomposition-based restoration of compressed images via adaptive noise estimation. IEEE Trans Image Process 25(9):4158–4171. https://doi.org/7.10.1109/TIP.2016.2588326

    MathSciNet  MATH  Google Scholar 

  8. Khmag A, Ramli A R, Al-Haddad SAR, Kamarudin N (2018) Natural image noise level estimation based on local statistics for blind noise reduction. Vis Comput 34(4):575–587. https://doi.org/10.1007/s00371-017-1362-0

    Article  Google Scholar 

  9. Xiao J, Tian H, Zhang Y, Zhou Y, Lei J (2018) Blind video denoising via texture-aware noise estimation. Comput Vis Image Underst 169:1–13. https://doi.org/10.1016/j.cviu.2017.11.012

    Article  Google Scholar 

  10. Ma B, Yao J, Le Y, Qin C, Yao H (2020) Efficient image noise estimation based on skewness invariance and adaptive noise injection. IET Image Process 14(7):1393–1401. https://doi.org/10.1049/iet-ipr.2019.1548

    Article  Google Scholar 

  11. Yue H, Liu J, Yang J, Nguyen T, Hou C (2017) Image noise estimation and removal considering the bayer pattern of noise variance. In: 2017 IEEE international conference on image processing (ICIP). IEEE, Beijing, pp 2976–2980

  12. Pimpalkhute V A, Page R, Kothari A, Bhurchandi K M, Kamble V M (2021) Digital image noise estimation using dwt coefficients. IEEE Trans Image Process 30:1962–1972. https://doi.org/10.1109/TIP.2021.3049961

    Article  Google Scholar 

  13. Hu L (2020) Image noise estimation with using pre-trained conventional neural network. In: 2020 IEEE 3rd international conference on automation, electronics and electrical engineering (AUTEEE). IEEE, pp 492–495

  14. Liu H, Xiong R, Liu D, Ma S, Wu F, Gao W (2017) Image denoising via low rank regularization exploiting intra and inter patch correlation. IEEE Trans Circ Syst Video Technol 28(12):3321–3332. https://doi.org/10.1109/TCSVT.2017.2759187

    Article  Google Scholar 

  15. Zhang M, Desrosiers C (2018) High-quality image restoration using low-rank patch regularization and global structure sparsity. IEEE Trans Image Process 28(2):868–879. https://doi.org/10.1109/TIP.2018.2874284

    Article  MathSciNet  MATH  Google Scholar 

  16. Zha Z, Wen B, Yuan X, Zhou J, Zhu C (2021) Image restoration via reconciliation of group sparsity and low-rank models. IEEE Trans Image Process 30:5223–5238. https://doi.org/10.1109/TIP.2021.3078329

    Article  MathSciNet  Google Scholar 

  17. Shen W, Fan Y, Bai Y, Huang L, Shang Q, Liu C, Zeng X (2016) A combined deblocking filter and sao hardware architecture for hevc. IEEE Trans Multimed 18(6):1022–1033. https://doi.org/10.1109/TMM.2016.2532606

    Article  Google Scholar 

  18. Chen Y, Bordes P, Poirier T, Racape F (2017) Optimization of sample adaptive band offset in hevc. In: 2017 Data compression conference (DCC). IEEE, Snowbird, pp 434–434

  19. Zhang J, Jia C, Zhang N, Ma S, Gao W (2016) Structure-driven adaptive non-local filter for high efficiency video coding (hevc). In: 2016 Data compression conference (DCC). IEEE, Snowbird, pp 91–100

  20. Zhao C, Zhang J, Ma S, Fan X, Zhang Y, Gao W (2016) Reducing image compression artifacts by structural sparse representation and quantization constraint prior. IEEE Trans Circ Syst Video Technol 27(10):2057–2071. https://doi.org/10.1109/TCSVT.2016.2580399

    Article  Google Scholar 

  21. Meng X, Jia C, Wang S, Zheng X, Ma S (2018) Optimized non-local in-loop filter for video coding. In: 2018 Picture Coding Symposium (PCS). IEEE, San Francisco, pp 233–237

  22. Jia C, Luo F, Zhang X, Wang S, Wang S, Ma S (2020) Fast non-local adaptive in-loop filter optimization on gpu. IEEE Trans Multimed 23:39–51. https://doi.org/10.1109/TMM.2020.2981185

    Article  Google Scholar 

  23. Park W-S, Kim M (2016) Cnn-based in-loop filtering for coding efficiency improvement. In: 2016 IEEE 12th image, video, and multidimensional signal processing workshop (IVMSP). IEEE, Bordeaux, pp 1–5

  24. Dai Y, Liu D, Wu F (2017) A convolutional neural network approach for post-processing in hevc intra coding. In: International conference on multimedia modeling. Springer, Reykjavik, pp 28–39

  25. Dai Y, Liu D, Zha Z-J, Wu F (2018) A cnn-based in-loop filter with cu classification for hevc. In: 2018 IEEE visual communications and image processing (VCIP). IEEE, Taichung, pp 1–4

  26. Li T, Xu M, Zhu C, Yang R, Wang Z, Guan Z (2019) A deep learning approach for multi-frame in-loop filter of hevc. IEEE Trans Image Process 28(11):5663–5678. https://doi.org/10.1109/TIP.2019.2921877

    Article  MathSciNet  MATH  Google Scholar 

  27. Yang R, Sun X, Xu M, Zeng W (2019) Quality-gated convolutional lstm for enhancing compressed video. In: 2019 IEEE international conference on multimedia and expo (ICME). IEEE, Shanghai, pp 532–537

  28. Pan Z, Yi X, Zhang Y, Jeon B, Kwong S (2020) Efficient in-loop filtering based on enhanced deep convolutional neural networks for hevc. IEEE Trans Image Process 29:5352–5366. https://doi.org/10.1109/TIP.2020.2982534

    Article  MATH  Google Scholar 

  29. Lu G, Zhang X, Ouyang W, Xu D, Chen L, Gao Z (2019) Deep non-local kalman network for video compression artifact reduction. IEEE Trans Image Process 29:1725–1737. https://doi.org/10.1109/TIP.2019.2943214

    Article  MathSciNet  MATH  Google Scholar 

  30. Yu L, Shen L, Yang H, Wang L, An P (2019) Quality enhancement network via multi-reconstruction recursive residual learning for video coding. IEEE Signal Process Lett 26(4):557–561. https://doi.org/10.1109/LSP.2019.2899253

    Article  Google Scholar 

  31. Sun W, He X, Chen H, Sheriff R E, Xiong S (2020) A quality enhancement framework with noise distribution characteristics for high efficiency video coding. Neurocomputing 411:428–441. https://doi.org/10.1016/j.neucom.2020.06.048

    Article  Google Scholar 

  32. Lee S Y, Yang Y, Kim D, Cho S, Oh B T (2020) Offset-based in-loop filtering with a deep network in hevc. IEEE Access 8:213958–213967. https://doi.org/10.1109/ACCESS.2020.3040751

    Article  Google Scholar 

  33. Ding D, Kong L, Wang W, Zhu F (2021) A progressive cnn in-loop filtering approach for inter frame coding. Signal Process Image Commun 94:116201. https://doi.org/10.1016/j.image.2021.116201

    Article  Google Scholar 

  34. Jia C, Wang S, Zhang X, Wang S, Liu J, Pu S, Ma S (2019) Content-aware convolutional neural network for in-loop filtering in high efficiency video coding. IEEE Trans Image Process 28(7):3343–3356. https://doi.org/10.1109/TIP.2019.2896489

    Article  MathSciNet  MATH  Google Scholar 

  35. Chen Z, He X, Ren C, Chen H, Zhang T (2021) Enhanced separable convolution network for lightweight jpeg compression artifacts reduction. IEEE Signal Process Lett. https://doi.org/10.1109/LSP.2021.3090249

  36. Xing Q, Xu M, Li T, Guan Z (2020) Early exit or not: Resource-efficient blind quality enhancement for compressed images. In: Vedaldi A, Bischof H, Brox T, Frahm J-M (eds) Computer Vision – ECCV 2020. Springer International Publishing, Cham, pp 275–292

  37. Huang Z, Sun J, Guo X, Shang M (2021) One-for-all: An efficient variable convolution neural network for in-loop filter of vvc. IEEE Trans Circ Syst Video Technol. https://doi.org/10.1109/TCSVT.2021.3089498

  38. Huang Z, Guo X, Shang M, Gao J, Sun J (2021) An efficient qp variable convolutional neural network based in-loop filter for intra coding. In: 2021 Data compression conference (DCC). IEEE, Snowbird, pp 33–42

  39. Yang R, Timofte R (2021) NTIRE 2021 challenge on quality enhancement of compressed video: Dataset and study. In: IEEE/CVF conference on computer vision and pattern recognition workshops, Nashville

  40. Huang G, Liu Z, Van Der Maaten L, Weinberger K Q (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 4700–4708

  41. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778

  42. Chang S G, Yu B, Vetterli M (2000) Adaptive wavelet thresholding for image denoising and compression. IEEE Trans Image Process 9(9):1532–1546. https://doi.org/10.1109/83.862633

    Article  MathSciNet  MATH  Google Scholar 

  43. Bjøntegaard G (2001) Calculation of average psnr differences between rd-curves (vceg-m33). In: VCEG Meeting (ITU-T SG16 Q. 6), pp 2–4

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (Grant No. 61871279 and GrantNo. 62081330105) and the Fundamental Research Funds for the Central Universities (Grant No. 2021SCU12061).

Author information

Authors and Affiliations

  1. College of Electronics and Information Engineering, Sichuan University, Chengdu, Sichuan, China

    Weiheng Sun, Xiaohai He, Honggang Chen, Shuhua Xiong & Yifei Xu

Authors
  1. Weiheng Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaohai He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Honggang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shuhua Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yifei Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaohai He.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, W., He, X., Chen, H. et al. A nonlocal HEVC in-loop filter using CNN-based compression noise estimation. Appl Intell 52, 17810–17828 (2022). https://doi.org/10.1007/s10489-022-03259-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03259-z

Keywords

Search

Navigation