Skip to main content
SpringerLink
Log in

Combining audio and video metrics to assess audio-visual quality

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this work, we studied the use of combination models to integrate audio and video quality estimates to predict the overall audio-visual quality. More specifically, an overall quality prediction for an audio-visual signal is obtained by combining the outputs of individual audio and video quality metrics with either a linear, a Minkowski, or a power function. A total of 7 different video quality metrics are considered, from which 3 are Full-Reference and 4 are No-Reference. Similarly, a total of 4 audio quality metrics are tested, 2 of which are Full-Reference and 2 are No-Reference. In total, we tested 18 Full-Reference audio-visual combination metrics and 24 No-Reference audio-visual combination metrics. The performance of all combination metrics are tested on two different audio-visual databases. Therefore, besides analysing the performance of a set of individual audio and video quality metrics, we analyzed the performance of the models that combine these audio and video quality metrics. This work gives an important contribution to the area of audio-visual quality assessment, since previous works either tested combination models only on subjective quality scores or used linear models to combine the outputs of a limited number of audio and video quality metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. http://www.cdvl.org.

  2. http://www.ene.unb.br/mylene/databases.html.

References

  1. Bong DBL, Khoo BE (2015) Objective blur assessment based on contraction errors of local contrast maps. Multimed Tools Appl 74.17:7355–7378

    Article  Google Scholar 

  2. Chikkerur S, Sundaram V, Reisslein M, Karam LJ (2011) Objective video quality assessment methods: a classification, review, and performance comparison. IEEE Trans Broadcast 57(2):165–182

    Article  Google Scholar 

  3. Engelke U, Zepernick H-J (2007) Perceptual-based quality metrics for image and video services: a survey. In: 3rd EuroNGI Conference on next generation internet networks, pp 190–197

  4. Farias MCQ, Mitra SK (2005) No-reference video quality metric based on artifact measurements. IEEE Int Conf Image Process ICIP 2005 3(2):III–141–4

    Article  Google Scholar 

  5. Garcia MN, Schleicher R, Raake A (2011) Impairment-factor-based audiovisual quality model for iptv Influence of video resolution, degradation type, and content type. EURASIP J Image Vid Process 1–14

  6. Hands DS (2004) A basic multimedia quality model. IEEE Trans Multimed 6 (6):806–816

    Article  Google Scholar 

  7. Hines A, Skoglund J, Kokaram A, Harte N (2012) Visqol: the virtual speech quality objective listener. In: International workshop on acoustic signal enhancement. Proceedings of IWAENC 2012. VDE, pp 1–4

  8. Hines A, Skoglund J, Kokaram A, Harte N (2013) Robustness of speech quality metrics to background noise and network degradations: comparing visqol, pesq and polqa. In: 2013 IEEE International conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 3697–3701

  9. Hines A, Gillen E, Harte N (2015) Measuring and monitoring speech quality for voice over IP with POLQA, ViSQOL and P. 563. In: Interspeech conference. Dresden, pp 6–10

  10. ITU Recommendation BT.500-8 (1998) Methodology for subjective assessment of the quality of television pictures

  11. Lin W, Kuo C-CJ (2011) Perceptual visual quality metrics: a survey. J Vis Commun Image Represent 22(4):297–312

    Article  Google Scholar 

  12. Malfait L, Berger J, Kastner M (2006) P.563 The itu-t standard for single-ended speech quality assessment. IEEE Trans Audio Speech Lang Process 14(6):1924–1934

    Article  Google Scholar 

  13. Martinez HB, Farias MCQ (2014) A no-reference audio-visual video quality metric. In 2014 Proceedings of the 22nd European on signal processing conference (EUSIPCO), pp 2125–2129

  14. Martinez HB, Farias MCQ (2014) Full-reference audio-visual video quality metric. J Electron Imag 23(6):061108–061108

    Article  Google Scholar 

  15. Mittal A, Moorthy AK, Bovik AC (2011) Blind/referenceless image spatial quality evaluator. In: Conference record of the forty fifth asilomar conference on signals, systems and computers (ASILOMAR). IEEE, pp 723–727

  16. Mittal A, Soundararajan R, Bovik AC (2013) Making a “completely blind” image quality analyzer. IEEE Signal Process Lett 20(3):209–212

    Article  Google Scholar 

  17. Moorthy AK, Bovik AC (2010) A two-step framework for constructing blind image quality indices. IEEE Signal Process Lett 17(5):513–516

    Article  Google Scholar 

  18. Moorthy AK, Bovik A C (2011) Visual quality assessment algorithms: what does the future hold? Multimed Tools Appl 51(2):675–696

    Article  Google Scholar 

  19. Pinson MH, Wolf S (2004) A new standardized method for objectively measuring video quality. IEEE Trans Broadcast 50(3):312–322

    Article  Google Scholar 

  20. Pinson MH, Ingram W, Webster A (2011) Audiovisual quality components. Signal Process Mag IEEE 28(6):60–67

    Article  Google Scholar 

  21. Pinson MH, Schmidmer C, Janowski L, Pépion R, Huynh-Thu Q, Corriveau P, Younkin A, Le Callet P, Barkowsky M, Ingram W (2013) Subjective objective evaluation of an audiovisual subjective dataset for research and development. In: Fifth International workshop on quality of multimedia experience (QoMEX). IEEE, pp 30–31

  22. Soh K, Iah S (2001) Subjectively assessing method for audiovisual quality using equivalent signal-to-noise ratio conversion. Trans Inst Electron Inform Commun Eng A 11:1305–1313

    Google Scholar 

  23. Thiede T, Treurniet WC, Bitto R, Schmidmer C, Sporer T, Beerends JG, Colomes C (2000) Peaq-the itu standard for objective measurement of perceived audio quality. J Audio Eng Soc 48(1/2):3–29

    Google Scholar 

  24. Wang Z, Sheikh HR, Bovik A (2002) No-reference perceptual quality assessment of jpeg compressed images. IEEE Int Conf Proc 1:I–477–I–480

    Google Scholar 

  25. Wang Z, Sheikh HR, Bovik AC (2002) No-reference perceptual quality assessment of jpeg compressed images. In: 2002 International conference on image processing. 2002 Proceedings, vol 1, pp I–477–I–480

  26. Wang Z, Lu L, Bovik A (2004) Video quality assessment based on structural distortion measurement. Signal Process Image Comm 19:121–132

    Article  Google Scholar 

  27. Winkler S, Faller C (2006) Perceived audiovisual quality of low-bitrate multimedia content. IEEE Trans Multimed 8(5):973–980

    Article  Google Scholar 

  28. Yamagishi K, Gao S (2013) Light-weight audiovisual quality assessment of mobile video Itu-t rec. p.1201.1. In: IEEE 15th International workshop on multimedia signal processing (MMSP), pp 464–469

Download references

Acknowledgments

This work was supported in part by Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) Brazil, in part by Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) - Brazil, and in part by the University of Brasília.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Brasília (UnB), Campus Universitário Darcy Ribeiro, 70919-970, Brasília, DF, Brazil

    Helard A. Becerra Martinez

  2. Department of Electrical Engineering, University of Brasília (UnB), Campus Universitário Darcy Ribeiro, 70919-970, Brasília, DF, Brazil

    Mylène C. Q. Farias

Authors
  1. Helard A. Becerra Martinez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mylène C. Q. Farias
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Helard A. Becerra Martinez.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Martinez, H.A.B., Farias, M.C.Q. Combining audio and video metrics to assess audio-visual quality. Multimed Tools Appl 77, 23993–24012 (2018). https://doi.org/10.1007/s11042-018-5656-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-5656-7

Keywords

Search

Navigation