Skip to main content
SpringerLink
Log in

A video compression artifact reduction approach combined with quantization parameters estimation

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

High Efficiency Video Coding is one of the most widely used Video Coding standards. It could encode videos to bitstream with a high compression rate for transporting, and videos would be reconstructed by decoding bitstream. However, decoded videos with compression artifacts would degrade the viewing experience of users. At present, many methods based on deep learning have been proposed to reduce compression artifacts for quality enhancement. Most methods train their models assuming that quantization parameters (QP) are known. But the QPs of videos are obtained from bitstream, which means QPs are probably unknown if there are only decoded videos. Without QP, training a network for compression artifacts reduction would become difficult. To this end, we propose a video compression artifacts reduction approach combined with quantization parameters estimation (VRQPE). In light of the fact that decoded videos with different QPs have different artifacts, we extract representative sample blocks in video frames to estimate the QPs of these videos. Then, we propose a majority voting to achieve better estimation results. Afterward, we proposed a quality enhancement network to reduce compressed artifacts. Extensive experiments demonstrate that our proposed VRQPE achieves superior performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Zhang X, Xiong R, Lin W, Zhang J, Wang S, Ma S, Gao W (2016) Low-rank-based nonlocal adaptive loop filter for high-efficiency video compression. IEEE Trans Circuits Systems Video Technol 27(10):2177–2188

    Google Scholar 

  2. Li F, Tan W, Yan B (2018) Deep residual network for enhancing quality of the decoded intra frames of hevc. In: 2018 25th IEEE International Conferenceon Image Processing (ICIP) IEEE 3918–3922 39183922

  3. Ma L, Tian Y, Huang T (2018) Residual-based video restoration for hevc intracoding. In: 2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM), IEEE, pp 1–7

  4. Tong J, Wu X, Ding D, Zhu Z, Liu Z (2019) Learning-based multi-frame video quality enhancement. In: 2019 IEEE International Conference on Image Processing (ICIP), IEEE, pp 929–933

  5. Dong C, Deng Y, Loy CC, Tang X (2015) Compression artifacts reduction by a deep convolutional network. In:Proceedings of the IEEE International Conference on Computer Vision, pp 576–584

  6. Dai Y, Liu D, Wu F (2017) A convolutional neural network approach for post-processing in hevc intra coding, in: International Conference on Multimedia Modeling, Springer, pp 28–39

  7. Yang R, Xu M, Wang Z (2017) Decoder-side hevc quality enhancement with scalable convolutional neural network, in: 2017 IEEE International Conference on Multimedia and Expo (ICME), IEEE, pp 817–822

  8. Yang R, Xu M, Wang Z, Li T (2018) Multi-frame quality enhancement for compressed video. In: Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition, pp 6664–6673

  9. Deng J, Wang L, Pu S, Zhuo C (2020) Spatio-temporal deformable convolutionfor compressed video quality enhancement, In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 34, pp 10696–10703

  10. Claus M, van Gemert J (2019) Videnn: Deep blind video denoising, In: Pro-ceedings of the IEEE/CVF Conference on Computer Vision and PatternRecognition (CVPR) Workshops,

  11. Ehret T, Davy A, Morel J-M, Facciolo G, Arias P (2019) Model-blind video denoising via frame-to-frame training, in: Proceedings of the IEEE/CVFConference on Computer Vision and Pattern Recognition, pp 11369–11378

  12. Amaranageswarao G, Deivalakshmi S, Ko SB (2020) Blind compression artifact reduction using dense parallel convolutional neural network. Signal Process Image Commun 89:116009

    Article  Google Scholar 

  13. Caballero J, Ledig C, Aitken A, Acosta A, Totz J, Wang Z, Shi W (2017) Real-time video super-resolution with spatio-temporal networks and motioncompensation. In: Proceedings of the IEEE Conference on Computer Visionand Pattern Recognition, pp 4778–4787

  14. Guo D, Xia Y, Xu L, Li W, Luo X (2021) Remote sensing image super-resolution using cascade generative adversarial nets. Neurocomputing 443:117–130.19

  15. Tao X, Gao H, Shen X, Wang J, Jia J (2018) Scale-recurrent network for deep image deblurring, in: Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition (CVPR)

  16. Zhang H, Wu Y, Zhang L, Zhang Z, Li Y (2020) Image deblurring using tri-segment intensity prior. Neurocomputing 398:265–279

    Article  Google Scholar 

  17. Xu M, Li T, Wang Z, Deng X, Yang R, Guan Z (2018) Reducing complexity of hevc: A deep learning approach. IEEE Trans Image Process 27(10):5044–5059. https://doi.org/10.1109/TIP.2018.2847035.

  18. Sengar SS, Mukhopadhyay S (2020) Motion segmentation-based surveillance video compression using adaptive particle swarm optimization. Neural Comput Appl 32:11443–11457

    Article  Google Scholar 

  19. Huang YL, Chang RF (2000) Error concealment using adaptive multilayer perceptrons (MLPs) for block-based image coding. Neural Comput Appl 9(2):83–92

    Article  Google Scholar 

  20. Shi Z , Feng Y , Zhao M et al (2020)A joint deep neural networks-based method for single nighttime rainy image enhancement. Neural Comput Appl, 32(9)

  21. Paraschiv EG, Ruiz-Coll D, Pantoja M et al (2019) Parallelization and improvement of the MDV-SW algorithm for HEVC intra-prediction coding. J Supercomput 75:1150–1162. https://doi.org/10.1007/s11227-018-2329-2

    Article  Google Scholar 

  22. Piñol P, Migallón H, López-Granado O et al (2015) Slice-based parallel approach for HEVC encoder. J Supercomput 71:1882–1892. https://doi.org/10.1007/s11227-014-1371-y

    Article  Google Scholar 

  23. Lu J, Liong VE, Zhou J (2017) Deep hashing for scalable image search. IEEE Trans Image Process 26(5):2352–2367. https://doi.org/10.1109/TIP.2017.2678163

    Article  MathSciNet  MATH  Google Scholar 

  24. Duan Y, Lu J, Wang Z, Feng J, Zhou J (2019) Learning deep binary descriptor with multi quantization. IEEE Trans Pattern Anal Mach Intell 41(8):1924–1938. https://doi.org/10.1109/TPAMI.2018.2858760

    Article  Google Scholar 

  25. Sun W, He X, Chen H, Sheriff RE, Xiong S (2020) A quality enhancement framework with noise distribution characteristics for high efficiency videocoding. Neurocomputing 41:428–441.20

  26. Lu CT, Wang LL, Shen JH et al (2021) Image enhancement using deep-learning fully connected neural network mean filter. J Supercomput 77:3144–3164. https://doi.org/10.1007/s11227-020-03389-6

    Article  Google Scholar 

  27. Tai Y, Yang J, Liu X, Xu C (2017) Memnet: A persistent memory network forimage restoration. In: Proceedings of the IEEE International Conferenceon Computer Vision (ICCV)

  28. Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Trans Image Process 26(7):3142–3155. https://doi.org/10.1109/TIP.2017.2662206

    Article  MathSciNet  MATH  Google Scholar 

  29. Wang T, Chen M, Chao H (2017) A novel deep learning-based method of improving coding efficiency from the decoder-end for hevc. Data Compression Conference (DCC) 2017:410–419. https://doi.org/10.1109/DCC.2017.42

    Article  Google Scholar 

  30. Liu M, Tang L, Zhong S, Luo H, Peng J (2021) Learning noise-decoupled affine models for extreme low-light image enhancement. Neurocomputing 448:21–29

    Article  Google Scholar 

  31. Yang R, Xu M, Liu T, Wang Z, Guan Z (2018) Enhancing quality for hevc compressed videos. IEEE Trans Circuits Syst Video Technol 29(7):2039–2054

    Article  Google Scholar 

  32. Guan Z, Xing Q, Xu M, Yang R, Liu T, Wang Z Mfqe 2.0: A newapproach for multi-frame quality enhancement on compressed video. IEEE Trans Patt Anal Mach Intell

  33. Chen CC, Zhang N, Ye YY et al (2021) A new framework based on spatio-temporal information for enhancing compressed video. In: 2021 4th International Conference on Information Communication and Signal Processing (ICICSP). IEEE, pp 571–576

  34. Li T, Xu M, Zhu C, Yang R, Wang Z, Guan Z (2019) A deep learning approach for multi-frame in-loop filter of hevc. IEEE Trans Image Process 28(11):5663–5678

    Article  MathSciNet  Google Scholar 

  35. Yang R, Sun X, Xu M, Zeng W (2019) Quality-gated convolutional lstm for enhancing compressed video, in: 2019 IEEE International Conference on Multimedia and Expo (ICME), IEEE, pp 532–537.21

  36. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. IEEE Conf Comput Vis Pattern Recog (CVPR) 2016:770–778. https://doi.org/10.1109/CVPR.2016.90

    Article  Google Scholar 

  37. Xue T, Chen B, Wu J, Wei D, Freeman WT (2019) Video enhancement with task-oriented flow. Int J Comput Vis 127(8):1106–1125

    Article  Google Scholar 

  38. Kim Y, Soh JW, Park J, Ahn B, Lee HS, Moon YS, Cho NI (2019) A pseudo-blind convolutional neural network for the reduction of compression artifacts. IEEE Trans Circuits Syst Video Technol 30(4):1121–1135

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (Grant No. 61871279 and No. 62081330105).

Author information

Authors and Affiliations

  1. College of Electronics and Information Engineering, Sichuan University, Chengdu, 610065, China

    Xin Shuai, Linbo Qing, Weiheng Sun & Xiaohai He

  2. Department of Computer Science, University of Maryland, College Park, MD, 20742, USA

    Mozhi Zhang

Authors
  1. Xin Shuai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Linbo Qing
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mozhi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weiheng Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaohai He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaohai He.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shuai, X., Qing, L., Zhang, M. et al. A video compression artifact reduction approach combined with quantization parameters estimation. J Supercomput 78, 13564–13582 (2022). https://doi.org/10.1007/s11227-022-04412-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04412-8

Keywords

Search

Navigation