Skip to main content
SpringerLink
Log in

A Gabor feature-based full reference video quality assessment model based on spatiotemporal slice of videos

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The distortion of video is under the twin impacts of spatial and temporal distortions. Some of the video quality assessment (VQA) metrics assess the video using a combination of these two measures. The accuracy of VQA may be reduced by separating spatial and temporal aspects. The goal of this study was to assess the spatiotemporal structure of video quality by using several VQA databases: (1) The 2D Gabor filter was designed to extract features from both spatial and temporal data, respectively, from video sequences. Moreover, the real and imaginary parts of the 2D Gabor response taken from the reference video and distortion video were then compared with the degree of similarity of the local frame quality. (2) To further enhance the quality of the assessment, we combined the spatial similarity with that of the spatiotemporal similarity and proposed the VQA model, named spatiotemporal slice Gabor feature similarity deviation. (3) In video quality scoring, a sequential pooling strategy was used to assemble the quality indices of frames. (4) Experimental evaluations of the video quality database show that the proposed metric has good consistency with subjective perception and competitive with state-of-the-art full reference video quality assessment models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Xue, W., Lei, Z., Xuanqin, M., Bovik, A.C.: Gradient magnitude similarity deviation: a highly efficient perceptual image quality index. IEEE Trans. Image Process. 23(2), 684–695 (2013). https://doi.org/10.1109/TIP.2013.2293423

    Article  MATH  MathSciNet  Google Scholar 

  2. Bovik, A.C.: Content-weighted video quality assessment using a three-component image model. J. Electron. Imaging 19(1), 011003 (2010). https://doi.org/10.1117/1.3267087

    Article  Google Scholar 

  3. Vu, P. V., Vu, C. T., Chandler, D. M.: A spatiotemporal most-apparent-distortion model for video quality assessment. In: 2011 18th IEEE International Conference on Image Processing, pp. 2505–2508. IEEE, Brussels, Belgium (2011). doi:https://doi.org/10.1109/ICIP.2011.6116171

  4. Pinson, M.H., Wolf, S.: A new standardized method for objectively measuring video quality. IEEE Trans. Broadcast. 50(3), 312–322 (2004). https://doi.org/10.1109/TBC.2004.834028

    Article  Google Scholar 

  5. Seshadrinathan, K., Bovik, A.: Motion tuned spatio-temporal quality assessment of natural videos. IEEE Trans. Image Process. 19, 335–350 (2010). https://doi.org/10.1109/TIP.2009.2034992

    Article  MATH  MathSciNet  Google Scholar 

  6. Choi, L.K., Bovik, A.C.: Video quality assessment accounting for temporal visual masking of local flicker. Signal Process. Image Commun. 67, 182–198 (2018). https://doi.org/10.1016/j.image.2018.06.009

    Article  Google Scholar 

  7. Yan, P., Mou, X., Xue, W.: Video quality assessment via gradient magnitude similarity deviation of spatial and spatiotemporal slices. Paper presented at IS&T/SPIE electronic imaging, San Francisco, California, United States, 94110M, March 2015. doi: https://doi.org/10.1117/12.2083283

  8. Lu, W., et al.: A spatiotemporal model of video quality assessment via 3D gradient differencing. Inf. Sci. 478, 141–151 (2019). https://doi.org/10.1016/j.ins.2018.11.003

    Article  Google Scholar 

  9. Chen, G., Yang, C., Xie, S.: Gradient-based structural similarity for image quality assessment. In: 2006 International Conference on Image Processing, pp. 2929–2932, IEEE, Atlanta, GA (2006). Doi: https://doi.org/10.1109/ICIP.2006.313132

  10. Wang, Z., Simoncelli, E. P., Bovik, A. C.: Multiscale structural similarity for image quality assessment. In: The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, pp. 1398–1402, IEEE, Pacific Grove, CA, USA (2003). doi:https://doi.org/10.1109/ACSSC.2003.1292216

  11. Chandler, D.M., Hemami, S.S.: VSNR: a wavelet-based visual signal-to-noise ratio for natural images. IEEE Trans. Image Process. 16(9), 2284–2298 (2007). https://doi.org/10.1109/TIP.2007.901820

    Article  MathSciNet  Google Scholar 

  12. Chandler, D.M.: Most apparent distortion: full-reference image quality assessment and the role of strategy. J. Electron. Imaging 19(1), 011006 (2010). https://doi.org/10.1117/1.3267105

    Article  Google Scholar 

  13. Sheikh, H.R., Bovik, A.C.: Image information and visual quality. IEEE Trans. Image Process. 15(2), 430–444 (2006)

    Article  Google Scholar 

  14. Wang, Z., Lu, L., Bovik, A.C.: Video quality assessment based on structural distortion measurement. Signal Process. Image Commun. 19(2), 121–132 (2004). https://doi.org/10.1016/S0923-5965(03)00076-6

    Article  Google Scholar 

  15. Yan, P., Mou, X.: Video quality assessment based on motion structure partition similarity of spatiotemporal slice images. J. Electron. Imag. 27(03), 1 (2018). https://doi.org/10.1117/1.JEI.27.3.033019

    Article  Google Scholar 

  16. Freitas, P.G., Akamine, W.Y.L., Farias, M.C.Q.: Using multiple spatio-temporal features to estimate video quality. Signal Process. Image Commun. 64, 1–10 (2018). https://doi.org/10.1016/j.image.2018.02.010

    Article  Google Scholar 

  17. Oppong, B.D., Mou, X.: Joint model of gradient magnitude and Gabor features via Spatio-Temporal slice. J. Vis. Commun. Image Represent. 79, 103204 (2021)

    Article  Google Scholar 

  18. Shen, L., Bai, L.: A review of Gabor wavelets for face recognition. Patt. Anal. Appl. 9, 273–292 (2006)

    Article  MathSciNet  Google Scholar 

  19. Liu, A., Lin, W., Narwaria, M.: Image quality assessment based on gradient similarity. IEEE Trans. Image Process. 21(4), 1500–1512 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  20. Xue, W., Zhang, L., Mou, X., Bovik, A.C.: Gradient magnitude similarity deviation: a highly efficient perceptual image quality index. IEEE Trans. Image Process. 23(2), 684–695 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  21. Ni, Z., Ma, L., Zeng, H., Cai, C., Ma, K.-K.: Screen content image quality assessment using edge model. In: Proceeding IEEE International Conference on Image Processing, pp. 81–85 (2016)

  22. Gujjunoori, S., Oruganti, M., Pais, A.R.: Enhanced optical flow-based full reference video quality assessment algorithm. Multimed. Tools Appl. (2022). https://doi.org/10.1007/s11042-022-12591-y

    Article  Google Scholar 

  23. Seshadrinathan, K., Sundararajan, R., Bovic, A.C., Cormack, L.K.: Study of subjective and objective quality assessment of video. IEEE Trans. Image Process. 19(6), 1427–1441 (2010). https://doi.org/10.1109/TIP.2010.2042111

    Article  MATH  MathSciNet  Google Scholar 

  24. Seshadrinathan, K., et al.: A subjective study to evaluate video quality assessment algorithms. In: Proceeding SPIE vol. 7527, 75270H (2010). Doi: https://doi.org/10.1117/12.845382

  25. Simone, D., Tagliasacchi, F., Naccari, M., Dufaux, M., Tubaro, F., Ebrahimi, T.: Subjective assessment of H.264/AVC video sequences transmitted over a noisy channel. In: 2009 International Workshop on Quality of Multimedia Experience, pp. 204–209 (2009). Doi: https://doi.org/10.1109/qomex.2009.5246952

  26. Simone, D., Naccari, F., Tagliasacchi, M., Tubaro, S., Ebrahimi, T.: H.264/AVC video database for the evaluation of quality metrics. In: Proceeding. ICASSP, pp. 2430 (2010). Doi: https://doi.org/10.1109/icassp.2010.5496296

  27. ITU: Methodology for the subjective assessment of the quality of television pictures BT Series Broadcasting service. Int. Telecommun. Union 13, 1–48 (2012)

    Google Scholar 

  28. Younghoon, L., Jungsoo, K., Chong-Min, K.: Energy-aware video encoding for image quality improvement in battery-operated surveillance camera. IEEE Trans. Very Larg. Scale Integr. Syst. 20, 310–318 (2012)

    Article  Google Scholar 

  29. Hallapuro, A., Lappalainen, V., Hamalainen, T. D.: Performance analysis of low bit rate H.26L video encoder. In: Paper Presented at the Proceedings of the Acoustics, Speech, and Signal Processing, 200. on IEEE International Conference – Vol. 02, (2001)

  30. Saponara, S., Denolf, K., Lafruit, G., Blanch, C., Bormans, J.: Performance and complexity co-evaluation of the advanced video coding standard for cost-effective multimedia communications. EURASIP J. Appl. Signal Process. 2004, 220–235 (2004)

    Google Scholar 

  31. Horowitz, M., Joch, A., Kossentini, F., Hallapuro, A.: H.264/AVC baseline profile decoder complexity analysis. IEEE Trans. Circuits Syst. Video Technol. 13, 704–716 (2003)

    Article  Google Scholar 

  32. Szu-Wei, L., Kuo, C.C.J.: Complexity modeling of spatial and temporal compensations in H.264/AVC decoding. IEEE Trans. Circuits Syst. Video Technol. 20, 706–720 (2010)

    Article  Google Scholar 

  33. Zhan, M., Hao, H., Yao, W.: On complexity modeling of H.264/AVC video decoding and its application for energy efficient decoding. IEEE Trans. Multimed. 13, 1240–1255 (2011)

    Article  Google Scholar 

  34. Lappalainen, V., Hallapuro, A., Hamalainen, T.D.: Complexity of optimized H.26L video decoder implementation. IEEE Trans. Circuits Syst. Video Technol. 13, 717–725 (2003)

    Article  Google Scholar 

  35. SO/IEC-JCT1/SC29/WG11: On software complexity, JCTVC-G757. In: Geneva, Switzerland, (2011)

  36. ISO/IEC-JCT1/SC29/WG11: HM decoder complexity assessment on ARM. In: JCTVC-G262, Geneva, Switzerland (2011)

  37. Chang-Guo, Z., et al.: MPEG video decoding with the UltraSPARC visual instruction set. In: Compcon '95.'Technologies for the Information Superhighway, Digest of Papers. pp. 470–477 (1995)

  38. Kim, W., You, J., Jeong, J.: Complexity control strategy for real-time H.264/AVC encoder. IEEE Trans. Consum. Electron. 56, 1137–1143 (2010)

    Article  Google Scholar 

  39. Kwon, D.N., Driessen, P.F., Basso, A., Agathoklis, P.: Performance and computational complexity optimization in configurable hybrid video coding system. IEEE Trans. Circuits Syst. Video Technol. 16, 31–42 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Image Processing and Pattern Recognition, Xi’an Jiaotong University, Xi’an, China

    Daniel Oppong Bediako & Xuanqin Mou

  2. School of Advance Technologies, Engineering and Science, ACCRA Institute of Technology, Accra, Ghana

    Maxwell Suobogbiree

Authors
  1. Daniel Oppong Bediako
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuanqin Mou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maxwell Suobogbiree
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bediako, D.O., Mou, X. & Suobogbiree, M. A Gabor feature-based full reference video quality assessment model based on spatiotemporal slice of videos. SIViP 17, 1621–1630 (2023). https://doi.org/10.1007/s11760-022-02372-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-022-02372-3

Keywords

Search

Navigation