Skip to main content
SpringerLink
Log in

An HEVC-compliant perceptual video coding using just noticeable difference

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Video coding removes spatial, temporal, and statistic redundancies. After H.265, to further improve the coding efficiency, many efforts have been dedicated to removing the perceptual redundancy by using human perception-based methods. Just noticeable difference (JND) gives a good approximation for the human visual system and provides a valuable solution to remove the perceptual redundancy for perceptual video coding (PVC). However, there are still problems in the PVC architecture and the JND profile. One is although the whole discrete cosine transform (DCT) block are suppressed, there are still many transform coefficients below the suppression levels which are not adequately suppressed. Another problem is, to the best of our knowledge, most JND profiles are measured by image-based test methods and past display equipment. However, compared to images, videos exhibits temporal characteristics, and the current trend of the display equipment is towards full high definition. To solve these problems, we first propose a high efficiency video coding (HEVC)-compliant PVC architecture, where the coefficients in a DCT block can be adequately suppressed in a whole block manner. Second, we propose a video-based test method to model the temporal masking (TM) effect, called TM-JND. Experimental results show that the proposed TM-JND model can more accurately estimate the JND values for today’s display equipment and videos, avoiding the overestimate of the JND values like other existing models. The proposed PVC architecture achieves a significant bitrate reduction with a negligible subjective quality loss, compared with the HEVC test model HM 16.9.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Bae SH, Kim J, Kim M (2016) HEVC-based perceptually adaptive video coding using a DCT-based local distortion detection probability model. IEEE Trans. Image Process. 25(7):3343–3357. https://doi.org/10.1109/TIP.2016.2568459

    Article  MathSciNet  MATH  Google Scholar 

  2. Bae SH, Kim M (2013) A new DCT-based JND model of monochrome images for contrast masking effects with texture complexity and frequency. In Proc. IEEE Int. Conf. Image Process. (ICIP). Melbourne, Victoria, Australia pp 431–434.

  3. Bae SH, Kim M (2013) A novel DCT-based JND model for luminance adaptation effect in DCT frequency. IEEE Signal Process. Lett. 20(9):893–896. https://doi.org/10.1109/LSP.2013.2272193

    Article  Google Scholar 

  4. Bae SH, Kim M (2014) A novel generalized DCT-based JND profile based on an elaborate CM-JND model for variable block-sized transforms in monochrome images. IEEE Trans. Image Process. 23(8):3227–3240. https://doi.org/10.1109/TIP.2014.2327808

    Article  MathSciNet  MATH  Google Scholar 

  5. Bae SH, Kim M (2017) A DCT-based total JND profile for spatiotemporal and foveated masking effects. IEEE Trans. Circuits Syst. Video Technol. 27(6):1196–1207. https://doi.org/10.1109/TCSVT.2016.2539862

    Article  Google Scholar 

  6. Bossen F (2012) Common test conditions and software reference configurations. In: document JCT-VC of ISO/IEC MPEG and ITU-T VCEG, JCTVC-K1100, 11th Meeting of JCT-VC. JCT-VC, Shanghai, China.

  7. Cui X, Peng Z, Jiang G, Chen F, Yu M (2019) Perceptual video coding scheme using just noticeable distortion model based on entropy filter. Entropy 21(11):1095. https://doi.org/10.3390/e21111095

    Article  Google Scholar 

  8. Hadizadeh H, Rajati A, Bajić IV (2017) Saliency-guided just noticeable distortion estimation using the normalized Laplacian pyramid. IEEE Signal Process. Lett. 24(8):1218–1222. https://doi.org/10.1109/LSP.2017.2717946

    Article  Google Scholar 

  9. ITU (2002) Methodology for the subjective assessment of the quality of television pictures. In: ITU-R BT.500-11. ITU, Geneva, Switzerland.

  10. Jaballah S, Larabi MC, Tahar JB (2018) Asymmetric DCT-JND for luminance adaptation effects: an application to perceptual video coding in MV-HEVC. In: Proc. IEEE Int. Conf. Acoust., Speech and Signal Process. (ICASSP). IEEE, Calgary, AB, Canada pp 1797–1801. https://doi.org/10.1109/ICASSP.2018.8461738

  11. JCT-VC (2020) HM 16.9 reference software https://hevc.hhi.fraunhofer.de/trac/hevc/browser/tags/HM-16.9

  12. Jia Y, Lin W, Kassim AA (2006) Estimating just-noticeable distortion for video. IEEE Trans. Circuits Syst. Video Technol. 16(7):820–829. https://doi.org/10.1109/TCSVT.2006.877397

    Article  Google Scholar 

  13. Jung C, Fu Q, Xue F (2017) Perceptual stereoscopic video coding using disparity just-noticeable-distortion model. J. Vis. Commun. Image Represent. 48:195–204. https://doi.org/10.1016/j.jvcir.2017.06.007

  14. Laird J, Rosen M, Pelz J, Montag E, Daly S (2006) Spatio-velocity CSF as a function of retinal velocity using unstabilized stimuli. In: B.E. Rogowitz, T.N. Pappas, S.J. Daly (eds.) Proc. SPIE 6057, Human Vision and Electronic Imaging XI, vol. 6057, pp. 32–43. International Society for Optics and Photonics, SPIE, Calgary, AB, Canada. https://doi.org/10.1117/12.647870

  15. Li Y, Yang G, Zhu Y, Ding X, Gong R (2018) Probability model-based early merge mode decision for dependent views coding in 3D-HEVC. ACM Trans. Multimedia Comput. Commun. Appl. 14(4) Article 12. https://doi.org/10.1145/3267128

  16. Liu D, Wang Y, Chen Z (2018) Joint foveation-depth just-noticeable-difference model for virtual reality environment. J. Vis. Commun. Image Represent. 56:73–82. https://doi.org/10.1016/j.jvcir.2018.07.015

  17. Luo Z, Song L, Zheng S, Ling N (2013) H.264/advanced video control perceptual optimization coding based on JND-directed coefficient suppression. IEEE Trans. Circuits Syst. Video Technol. 23(6):935–948. https://doi.org/10.1109/TCSVT.2013.2240919

  18. Ma L, Ngan KN, Zhang F, Li S (2011) Adaptive block-size transform based just-noticeable difference model for images/videos. Signal Process. Image Commun. 26(3):162–174. https://doi.org/10.1016/j.image.2011.02.002

  19. Mak CM, Ngan KN (2009) Enhancing compression rate by just-noticeable distortion model for H.264/AVC. In Proc. IEEE Int. Symp. Circuits Syst. (ISCAS) pp 609–612. IEEE, Taipei, Taiwan. https://doi.org/10.1109/ISCAS.2009.5117822

  20. Ott RL, Longnecker MT (2008) An introduction to statistical methods and data analysis, 6th. edn. Duxbury Press, Boston, MA, USA.

    Google Scholar 

  21. Pan Z, Lei J, Zhang Y, Wang FL (2018) Adaptive fractional-pixel motion estimation skipped algorithm for efficient HEVC motion estimation. ACM Trans. Multimedia Comput. Commun. Appl. 14(1), Article 12. https://doi.org/10.1145/3159170

  22. Robson J (1966) Spatial and temporal contrast sensitivity functions of the visual system. J. Opt. Soc. Am. 56(8):1141–1142. https://doi.org/10.1364/JOSA.56.001141

    Article  Google Scholar 

  23. Robson J, Graham N (1981) Probability summation and regional variation in contrast sensitivity across the visual field. Vis. Res. 21(3):409–418. https://doi.org/10.1016/0042-6989(81)90169-3

    Article  Google Scholar 

  24. Valizadeh S, Nasiopoulos P, Ward R (2020) Improving compression efficiency of HEVC using perceptual coding. Multimedia Tools Appl. https://doi.org/10.1007/s11042-020-09442-z

  25. Vidal E, Sturmel N, Guillemot C, Corlay P, Coudoux FX (2017) New adaptive filters as perceptual preprocessing for rate-quality performance optimization of video coding. J. Vis. Commun. Image Represent. 52:124–137. https://doi.org/10.1016/j.image.2016.12.003

  26. Wei Z, Ngan KN (2009) Spatio-temporal just noticeable distortion profile for grey scale image/video in DCT domain. IEEE Trans. Circuits Syst. Video Technol. 19(3):337–346. https://doi.org/10.1109/TCSVT.2009.2013518

  27. Wu J, Li L, Dong W, Shi G, Lin W, Kuo CCJ (2017) Enhanced just noticeable difference model for images with pattern complexity. IEEE Trans. Image Process. 26(6):2682–2693. https://doi.org/10.1109/TIP.2017.2685682

  28. Xiang G, Jia H, Yang M, Zhang X, Huang X, Liu J, Xie X (2018) A perceptually temporal adaptive quantization algorithm for HEVC. J. Vis. Commun. Image Represent. 50:280–289. https://doi.org/10.1016/j.jvcir.2017.11.011

Download references

Acknowledgements

This work was supported by the Natural Science Foundation of Fujian Province of China under Grant 2019J01046.

Author information

Authors and Affiliations

  1. School of Electronic Science and Engineering, Xiamen University, Xiamen, 361005, China

    Jiefeng Guo

Authors
  1. Jiefeng Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiefeng Guo.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, J. An HEVC-compliant perceptual video coding using just noticeable difference. Multimed Tools Appl 81, 1257–1286 (2022). https://doi.org/10.1007/s11042-021-11535-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11535-2

Keywords

Search

Navigation