Skip to main content
SpringerLink
Log in

A cost-effective and real-time QoE evaluation method for multimedia streaming services

  • Published:
Telecommunication Systems Aims and scope Submit manuscript

Abstract

Traditionally, subjective tests are conducted to obtain the Mean Opinion Score (MOS) or Quality of Experience (QoE) of a video streaming service. However, subjective tests are time-consuming and rather costly. Thus, a cost-effective and real-time QoE evaluation method for video streaming services is presented in this paper. The proposed scheme first adopts subjective tests to find the relationships between the QoE and individual QoS parameters. Next, based on subjective test results, the corresponding QoS–QoE mapping functions for individual QoS metrics are derived using the regression approach. Finally, the integrated QoE function, which is a multi-variate function of multiple QoS metrics, for assessing the overall QoE of video streaming services is proposed. Hence, network/service providers only need to measure related objective QoS parameters while the corresponding MOS/QoE can be easily derived in real time based on the integrated QoE function. Since subjective tests only need to be conducted occasionally for updating the parameters in the QoS–QoE mapping functions, the cost of QoE assessment decreases significantly.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Yuang, M. C., Liang, S. T., & Chen, Y. G. (1998). Dynamic video playout smoothing method for multimedia applications. Multimedia Tools and Applications, 6(1), 47–60.

    Article  Google Scholar 

  2. Laoutaris, N., & Stavrakakis, I. (2001). Adaptive playout strategies for packet video receivers with finite buffer capacity. In Proc. IEEE ICC’01, 3 (pp. 969–973), 2001.

  3. Kalman, M., Steinbach, E., & Girod, B. (2004). Adaptive media playout for low-delay video streaming over error-prone channels. IEEE Transaction on Circuit & Systems for Video Technology, 14(6), 841–851.

    Article  Google Scholar 

  4. Li, M., Lin, T.-W., & Cheng, S.-H. (2012). Arrival process-controlled adaptive media playout with multiple thresholds for video streaming. Multimedia Systems, 18(5), 391–407.

  5. ITU-T Rec. P.910. (2008). Subjective video quality assessment methods for multimedia applications.

  6. ITU-R Rec. BT.500-7. (1995). Methodology for the subjective assessment of the quality of television pictures.

  7. ITU-R Rec. BT.500-11. (2002). Methodology for the subjective assessment of the quality of television pictures.

  8. Takahashi, A., Hands, D., & Barriac, V. (2008). Standardization activities in the ITU for a QoE assessment of IPTV. IEEE Communications Magazine, 46(2), 78–84.

    Article  Google Scholar 

  9. Wang, B., Wen, X., Yong, S., & Wei, Z. (2009). A new approach measuring users’ QoE in the IPTV. In Proceedings of Pacific-Asia Conference on Circuits, Communications and Systems, (PACCS ’09) (pp. 453–456), 2009.

  10. Serral-Gracia, R., Cerqueira, E., Curado, M., Yannuzzi, M., Monteiro, E., & Masip-Bruin, X. (2010). An overview of quality of experience measurement challenges for video applications in IP networks. In Proceedings of 8th International Conference on Wired/Wireless Internet Communications, (WWIC 2010), LNCS 6074 (pp. 252–263), Lulea, Sweden, 2010.

  11. Piamrat, K., Viho, C., Bonnin, J.-M., & Ksentini, A. (2009). Quality of experience measurements for video streaming over wireless networks. In Proceedings of 6th International Conference on Information Technology: New Generations, ITNG ’09 (pp. 1184–1189), 2009.

  12. ITU-T Rec. P.800. (2003). Mean Opinion Score (MOS) terminology.

  13. Netravali, A. N., & Haskell, B. G. (1995). Digital pictures: Representation, compression, and standards (2nd ed.). New York: Plenum Press.

    Book  Google Scholar 

  14. Huynh-Thu, Q., & Ghanbari, M. (2012). The accuracy of PSNR in predicting video quality for different video scenes and frame rates. Telecommunication Systems, 49(1), 35–48.

    Article  Google Scholar 

  15. Pinson, M., & Wolf, S. (2004). A new standardized method for objectively measuring video quality. IEEE Transactions on Broadcasting, 50(3), 312–322.

    Article  Google Scholar 

  16. Wang, Z., Bovik, A., Sheikh, H., & Simoncelli, E. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.

    Article  Google Scholar 

  17. Venkataraman, M., & Chatterjee, M. (2011). Inferring video QoE in real time. IEEE Network, 25(1), 4–13.

    Article  Google Scholar 

  18. Khan, A., Sun, L., & Ifeachor, E. (2012). QoE prediction model and its application in video quality adaptation over UMTS networks. IEEE Transactions on Multimedia, 14(2), 431–442.

    Article  Google Scholar 

  19. Kim, H. J., Lee, D. H., Lee, J. M., Lee, K. H., Lyu, W., & Choi, S. G. (2008). The QoE evaluation method through the QoS–QoE correlation model. In Proc. NCM’08, 2 (pp. 719–725), 2008.

  20. Kim, H. L., & Choi, S. G. (2010). A study on a QoS/QoE correlation model for QoE evaluation on IPTV service. In Proceedings of the 12th International Conference on Advanced Communication Technology (ICACT 2010) (pp. 1377–1382), 2010.

  21. Romaniak, P. (2009). Towards realization of a framework for integrated video quality of experience assessment. In Proceedings of IEEE INFOCOM 2009 (pp. 1–2).

  22. Mok, R. K. P., Chan, E. W. W., & Chang, R. K. C. (2011). Measuring the quality of experience of HTTP video streaming. In Proceedings of IFIP/IEEE International Symposium on Integrated Network Management (IM) (pp. 485–492).

  23. Romaniak, P., Mu, M., Mauthe, A., D’Antonio, S., & Leszcuk, M. (2008). A framework for integrated video quality assessment. In Proceedings of 18th ITC Specialist Seminar on Quality of Experience (pp. 29–30).

  24. Joskowicz, J., & Ardao, J. C. L. (2012). A parametric model for perceptual video quality estimation. Telecommunication Systems, 49(1), 49–62.

    Article  Google Scholar 

  25. Kanumuri, S., Cosman, P., & Reibman, A. (2004). A generalized linear model for MPEG-2 packet loss visibility. In Proceedings of Packet Video Workshop, (PV2004), Irvine, December 2004.

  26. Cognet, Y. (2006). Measuring IPTV QoS performance at the box. http://www.embedded.com/print/4013017.

  27. Pinson, M., & Wolf, S. (2003). Comparing subjective video quality testing methodologies. In Proceedings of SPIE Video Communications and Image Processing Conference, 5150 (pp. 573–582).

  28. Rouse, D. M., Pepion, R., Callet, P. L., & Hemami, S. S. (2010). Tradeoffs in subjective testing methods for image and video quality assessment. In Proceeedings of SPIE, Human Vision and Electronic Imaging XV, 2010.

  29. Fiedler, M., Hossfeld, T., & Tran-Gia, P. (2010). A generic quantitative relationship between quality of experience and quality of service. IEEE Network, 24(2), 36–41.

    Article  Google Scholar 

  30. Ickin, S., Wac, K., Fiedler, M., Janowski, L., Hong, J.-H., & Dey, A. K. (2012). Factors influencing quality of experience of commonly used mobile applications. IEEE Communications Magazine, 50(4), 48–56.

    Article  Google Scholar 

  31. Gardikis, G., Xilouris, G., Pallis, E., & Kourtis, A. (2012). Joint assessment of network- and perceived-QoS in video delivery networks. Telecommunication Systems, 49(1), 75–84.

    Article  Google Scholar 

  32. RFC 4445. (2006). A Proposed Media Delivery Index (MDI). http://tools.ietf.org/html/rfc4445.

  33. Jaroslav, K. (2008). MDI measurement in the IPTV. In Proceedings of IWSSIP 2008 (pp. 49–51), Bratislava, Slovakia, 2008.

  34. Greengrass, J., Evans, J., & Begen, A. C. (2009). Not all packets are equal, part 1: Streaming video coding and SLA requirements. IEEE Internet Computing Magazine, 13(1), 70–75.

    Article  Google Scholar 

  35. Greengrass, J., Evans, J., & Begen, A. C. (2009). Not all packets are equal, part 2: The impact of network packet loss on video quality. IEEE Internet Computing Magazine, 13(2), 74–82.

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by Ministry of Science and Technology of Republic of China under Grant NSC101-2221-E-182-004. The authors would also like to thank all reviewers for their valuable comments and suggestions.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, School of Electrical and Computer Engineering, College of Engineering, Chang Gung University, 259 Wen-Hwa 1st Road, Kwei-Shan, 33302, Tao-Yuan, Taiwan

    Mingfu Li & Chen-Yu Lee

Authors
  1. Mingfu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen-Yu Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mingfu Li.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, M., Lee, CY. A cost-effective and real-time QoE evaluation method for multimedia streaming services. Telecommun Syst 59, 317–327 (2015). https://doi.org/10.1007/s11235-014-9938-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11235-014-9938-8

Keywords

Search

Navigation