Skip to main content
SpringerLink
Log in

Framework for the integrated video quality assessment

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Through years of development Content Networks (CN) have become more sophisticated and more technically diverse. Modern CN are designed to be more adaptive to communication environment, devices and user requirements. However, one open issue is the still fluctuating quality of service provision. As a result user experience can be negatively affected. In order to maintain a satisfactory level of user experience it is crucial to develop a feasible solution to measure the extent to which video services meet users’ expectation. Assessing video quality with respect to users’ subjective opinions is a complex task. In this paper we address challenges of this task and design an integrated framework using a number of comprehensive functional modules. Our framework integrates objective quality assessment models of Artifacts Measurement (AM) and Quality of Delivery (QoD) approaches. Only the fittest models are activated by the framework considering requirements of individual evaluation tasks. We also introduce our recent work of realising key functional modules of the framework. Joint subjective experiments between two institutes have also been carried out for the purpose of model implementation and evaluation. Results from experiments verify the concept of an integrated framework and show the effectiveness of its key modules in estimating the quality level of video services.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. In the presented research the scale has 11 values but some different scales are possible, too.

  2. http://www.p2p-next.org/

References

  1. Addis B, Leyffer S (2006) A trust-region algorithm for global optimization. Comput Optim Appl 35(3):287–304. doi:10.1007/s10589-006-8716-2

    Article  MathSciNet  MATH  Google Scholar 

  2. Botta A, Pescape A (2011) Ip packet interleaving: bridging the gap between theory and practice. In: Proc. IEEE symp. computers and communications (ISCC), pp 1022–1029. doi:10.1109/ISCC.2011.5983976

  3. Botta A, Pescapè A, Ventre G (2008) Quality of service statistics over heterogeneous networks: analysis and applications. Elesevier EJOR

  4. Brand ando T, Queluz M (2010) No-reference quality assessment of h.264/avc encoded video. IEEE Trans Circuits Syst Video Technol 20(11):1437–1447. doi:10.1109/TCSVT.2010.2077474

    Article  Google Scholar 

  5. Chan A, Zeng K, Mohapatra P, Lee SJ, Banerjee S (2010) Metrics for evaluating video streaming quality in lossy ieee 802.11 wireless networks. In: Proceedings of the 29th conference on information communications, INFOCOM’10. IEEE Press, Piscataway, NJ, pp 1613–1621. http://portal.acm.org/citation.cfm?id=1833515.1833741

    Google Scholar 

  6. Claypool M, Zhu Y (2003) Using interleaving to ameliorate the effects of packet loss in a video stream. In: Proc. 23rd int distributed computing systems workshops conf, pp 508–513. doi:10.1109/ICDCSW.2003.1203604

  7. CMP Media LLC (2009) Measuring IPTV QoS performance at the box. http://www.digitaltvdesignline.com/

  8. Dosselmann R, Yang XD (2007) A prototype no-reference video quality system. In: Fourth Canadian conference on computer and robot vision, CRV ’07, vol 2007, pp 411–417

  9. Eskicioglu AM (2000) Quality measurement for monochrome compressed images in the past 25 years. In: Proceedings of the international conference on acoustics, speech and signal processing (ICASSP). IEEE, Istanbul, Turkey, pp 1907–1910

    Google Scholar 

  10. Eskicioglu AM, Fisher PS (1995) Image quality measures and their performance. IEEE Trans Commun 43(12):2959–2965

    Article  Google Scholar 

  11. Farias MCQ, Mitra SK (2005) No-reference video quality metric based on artifact measurements. In: IEEE international conference on image processing, ICIP 2005, vol 3, pp III–141–4

  12. Fukumoto K (2004) Taking bounded variables seriously: extended beta binomial, asymmetric logit, and time series. In: Research workshop in applied statistics

  13. Geerts D, De Moor K, Ketyko I, Jacobs A, Van den Bergh J, Joseph W, Martens L, De Marez L (2010) Linking an integrated framework with appropriate methods for measuring qoe. In: Second international workshop on quality of multimedia experience (QoMEX)

  14. ITU-R (1998) ITU-R BT.1129 subjective assessment of standard definition digital television (SDTV) systems. ITU-R

  15. ITU-R (1998) ITU-R BT.500 methodology for the subjective assessment of the quality of television pictures, Geneva, Switzerland

  16. ITU-T (1996) ITU-T P.800 methods for subjective determination of transmission quality. ITU-T, Geneva, Switzerland

  17. ITU-T (1999) ITU-T P.910 subjective video quality assessment methods for multimedia applications. ITU-T

  18. Jain R (2004) Quality of experience. IEEE Multimed 11(1):96–95

    Article  Google Scholar 

  19. Kangasharju J, Mu M, Colussi GD (2008) Application-level fairness. In: Proc. international conference on information networking 2008 (ICOIN 2008), Busan, Korea

  20. Kwon JC, Jang SH, Chin Y, Oh SJ (2010) A novel video quality impairment monitoring scheme over an ipty service with packet loss. In: 2010 second international workshop on quality of multimedia experience (QoMEX), pp 224–229. doi:10.1109/QOMEX.2010.5516127

  21. Lee J, Hoppel K (1989) Noise modeling and estimation of remotely-sensed images. In: Proc. international geoscience and remote sensing, Vancouver, Canada, vol 2, pp 1005–1008

  22. Liang YJ, Apostolopoulos JG, Girod B (2003) Analysis of packet loss for compressed video: does burst-length matter? In: Proc. IEEE int. conf. acoustics, speech, and signal processing (ICASSP ’03), vol 5. doi:10.1109/ICASSP.2003.1200063

  23. Liu T, Wang Y, Boyce JM, Yang H, Wu Z (2009) A novel video quality metric for low bit-rate video considering both coding and packet-loss artifacts. IEEE J Select Top Signal Process 3(2):280–293. doi:10.1109/JSTSP.2009.2015069

    Article  Google Scholar 

  24. MSU Graphics & Media Lab (Video Group) (2009) MSU video quality measurement tool. http://compression.ru/video/quality_measure/video_measurement_tool_en.html/

  25. Mu M, Mauthe A, Garcia F (2008) A utility-based qos model for emerging multimedia applications. In: Proc. first IEEE future multimedia networking (FMN 08) workshop, Cardiff, UK

  26. Mu M, Gostner R, Mauthe A, Garcia F, Tyson G (2009) Visibility of individual packet loss on h.264 encoded video stream—a user study on the impact of packet loss on perceived video quality. In: Proc. sixteenth annual multimedia computing and networking (MMCN’09), San Jose, California, USA

  27. Mu M, Mauthe A, Haley R, Garcia F (2011) Discrete quality assessment in IPTV content distribution networks. Signal Process Image Commun 26(7):339–357, Advances in IPTV Technologies, Elsevier

    Article  Google Scholar 

  28. Ohm J-R (1999) Bildsignalverarbeitung für Multimedia-Systeme, Berlin

  29. OPTICOM (2007) Perceptual evaluation of video quality. http://www.opticom.de/technology/pevq.html

  30. Pinson M, Wolf S, Gallagher M (2004) The impact of monitor resolution and type on subjective video quality testing

  31. van den Branden Lambrecht CJ, Verscheure O (1996) Perceptual quality measure using a spatio-temporal model of the human visual system. In: Proc. SPIE, vol 2668, pp 450–461

  32. VQEG (2004) Index VQEG test sequences. ftp://vqeg.its.bldrdoc.gov/SDTV/VQEG_PhaseI/TestSequences/

  33. VQEG (2008) Final report from the video quality experts group on the validation of objective models of multimedia quality assessment, phase I. ftp://vqeg.its.bldrdoc.gov/Documents/Projects/multimedia/MM_Final_Report/VQEG_Multimedia_PhaseI_Final_Report.zip

  34. VQEG (2008) Final report from the video quality experts group on the validation of objective models of video quality assessment, phase II. VQEG final report of FR-TV phase II validation test

  35. VQEG (2009) The video quality experts group. http://www.vqeg.org/

  36. Wang Y (2006) Survey of objective video quality measurements. EMC Corporation Hopkinton. ftp://ftp.cs.wpi.edu/pub/techreports/pdf/06-02.pdf

  37. Wang Z (1998) Objective image/video quality measurement—a literature survey. In: EE 381k: multidimensional digital signal processing

  38. Wang Z, Lu L, Bovik AC (2004) Video quality assessment based on structural distortion measurement. Signal Process Image Commun 19(2):121–213

    Article  Google Scholar 

  39. Winkler S (1999) A perceptual distortion metric for digital color video. In: Proc. SPIE, vol 3644, pp 175–184

  40. Winkler S (2005) Digital video quality—vision models and metrics. John Wiley & Sons, Ltd

  41. Winkler S (2007) Video quality and beyond. In: Proc. European signal processing conference, Poznan, Poland

  42. Wolf S, Pinson MH (1999) Spatial-temporal distortion metrics for in-service quality monitoring of any digital video system. In: Proc. SPIE, vol 3845, pp 266–277

  43. Yang F, Wan S, Xie Q, Wu HR (2010) No-reference quality assessment for networked video via primary analysis of bit stream. IEEE Trans Circuits Syst Video Technol 20(11):1544–1554. doi:10.1109/TCSVT.2010.2087433

    Article  Google Scholar 

  44. Yim C, Bovik AC (2011) Evaluation of temporal variation of video quality in packet loss networks. Signal Process Image Commun 26(1):24–38. doi:10.1016/j.image.2010.11.002. http://www.sciencedirect.com/science/article/B6V08-51FGSXF-1/2/b8af320f14b4e2c56a73dcb4ff042ca1

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

Application of Piotr Romaniak’s, Mikolaj Leszczuk’s and Lucjan Janowski’s research leading to these results has received funding from the European Community’s Seventh Framework Program (FP7/2007-2013) under grant agreement n°218086 (INDECT). Mu Mu’s work is also supported by Agilent Laboratories UK, European Commission within the FP7 Project: P2P-Next and Framework for Innovation and Research in MediaCityUK (FIRM). Eduardo Cerqueira was supported by The National Council for Scientific and Technological Development (CNPq—Brazil) and Fundacao de Amparo a Pesquisa do Estado do Para (FAPESPA). The authors thank Piotr Borkowski from AGH University of Science and Technology, Roswitha Gostner from Lancaster University and Francisco Garcia from Agilent Laboratories for their input.

Author information

Authors and Affiliations

  1. The School of Computing and Communications, Lancaster University, Lancaster, UK

    Mu Mu & Andreas Mauthe

  2. The Department of Telecommunications, AGH University of Science and Technology, Krakow, Poland

    Piotr Romaniak, Mikołaj Leszczuk & Lucjan Janowski

  3. The Federal University of Para, Belem, Brazil

    Eduardo Cerqueira

Authors
  1. Mu Mu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Piotr Romaniak
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andreas Mauthe
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mikołaj Leszczuk
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lucjan Janowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Eduardo Cerqueira
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Piotr Romaniak.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mu, M., Romaniak, P., Mauthe, A. et al. Framework for the integrated video quality assessment. Multimed Tools Appl 61, 787–817 (2012). https://doi.org/10.1007/s11042-011-0946-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-011-0946-3

Keywords

Search

Navigation