Skip to main content
SpringerLink
Log in

Learning from users: a data-driven method of QoE evaluation for Internet video

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Improving quality of experience (QoE) is increasingly significant for Internet video content providers. The essential issue is how to evaluate QoE under the complex circumstance of Internet video. Based on the massive user data extracted from a large scale Video-on-Demand (VoD) provider, we present a data-driven, comprehensive and extendible study on the problems of QoE evaluation. The main works of this paper include obtaining QoE-associated features via feature engineering and building an evaluation model on features of different aspects for Internet video QoE. Firstly, for feature engineering, we propose to introduce pattern features of user viewing behaviors that interact with user-perceived video quality. A new method of frequent time series pattern mining is proposed to find typical patterns. We correlate user experience with user-perceived quality features and user behavior pattern features, and consider the impact of confounding factors by applying them as context features into modeling. Secondly, interdependency among features is challenging for QoE evaluation modeling. And the high dimension of feature vector should be considered. To address these challenges, we develop an ensemble method to model the interactions between features and their intricate relationships to user experience. Experiments demonstrate that our approach could achieve sound results in comparison with other related works.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. The support level is the frequency of a pattern that occurs in our dataset.

  2. Because of the confidential agreement, we should not mention the name of the content provider.

  3. We conduct K-means algorithm getting rid of values that is too large according to the distribution, since K-means is sensitive to singular value.

  4. In this section, we temporarily set the threshold of DVC to 0.7, labeling video session instance to good or bad DVC. We make a further analysis of DVC threshold in Section 5.4

References

  1. Agrawal R, Srikant R (1995) Mining sequential patterns. In: Proceedings of the Eleventh international conference on data engineering, 1995. IEEE, pp 3–14

  2. Agrawal R, Mannila H, Srikant R, Toivonen H, Verkamo AI (1996) Fast discovery of association rules. In: Advances in knowledge discovery and data mining, pp 307–328

  3. Agarwal RC, Aggarwal CC, Prasad VVV (2002) Depth first generation of long patterns. In: ACM SIGKDD International conference on knowledge discovery and data mining, pp 108–118

  4. Agboma F, Liotta A (2012) Quality of experience management in mobile content delivery systems. Telecommun Syst 49(1):85–98

    Article  Google Scholar 

  5. Ayres J (2002) Sequential pattern mining using a bitmap representation. In: Eighth ACM SIGKDD International conference on knowledge discovery and data mining, pp 429–435

  6. Balachandran A, Sekar V, Akella A, Seshan S (2013) Analyzing the potential benefits of CDN augmentation strategies for internet video workloads. In: Conference on internet measurement conference, pp 43–56

  7. Balachandran A, Sekar V, Akella A, Seshan S (2013) Understanding internet video viewing behavior in the wild. In: ACM SIGMETRICS/International conference on measurement and modeling of computer systems, pp 379–380

  8. Balachandran A, Sekar V, Akella A, Seshan S, Stoica I, Zhang H (2013) Developing a predictive model of quality of experience for internet video. Comput Commun Rev 43(4):339–350

    Article  Google Scholar 

  9. Bayardo RJ (1998) Efficiently mining long patterns from databases. ACM SIGMOD Record 27(2):85–93

    Article  Google Scholar 

  10. Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140

    MATH  Google Scholar 

  11. Bryll R, Gutierrez-Osuna R, Quek F (2003) Attribute bagging: improving accuracy of classifier ensembles by using random feature subsets. Pattern Recogn 36 (6):1291–1302

    Article  Google Scholar 

  12. Casas P, Schatz R, Hoßfeld T (2013) Monitoring YouTube QoE: is your mobile network delivering the right experience to your customers? In: IEEE Wireless communications and networking conference, pp 1609–1614

  13. Chen L, Zhou Y, Chiu DM (2014) A study of user behavior in online VoD services. Comput Commun 46(6):66–75

    Article  Google Scholar 

  14. Cheng H, Yan X, Han J, Hsu CW (2007) Discriminative frequent pattern analysis for effective classification. In: IEEE International conference on data engineering, pp 716–725

  15. Choi J, Reaz A, Mukherjee B (2012) A survey of user behavior in VoD service and bandwidth-saving multicast streaming schemes. Commun Surv Tutor IEEE 14(1):156–169

    Article  Google Scholar 

  16. Diewert E (2005) Weighted country product dummy variable regressions and index number formulae. Rev Income Wealth 51(4):561–570

    Article  Google Scholar 

  17. Dobrian F, Awan A, Joseph DA, Ganjam A, Zhan J, Sekar V, Stoica I, Zhang H (2013) Understanding the impact of video quality on user engagement. Commun ACM, 56(3):91–99

    Article  Google Scholar 

  18. Domingos P (2012) A few useful things to know about machine learning. Commun ACM 55(10):78–87

    Article  Google Scholar 

  19. Freudenthaler C, Schmidt-Thieme L, Rendle S (2013) Bayesian factorization machines

  20. Gopalakrishnan V, Jana R, Ramakrishnan KK, Swayne DF, Vaishampayan VA (2011) Understanding couch potatoes: measurement and modeling of interactive usage of IPTV at large scale. In: Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference, pp 225–242

  21. Greenacre MJ (2007) Correspondence analysis in practice, 2nd edn. Crc Press

  22. Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3(6):1157–1182

    MATH  Google Scholar 

  23. Han J, Pei J, Yin Y (2000) Mining frequent patterns without candidate generation. ACM Sigmod Record 29(2):1–12

    Article  Google Scholar 

  24. Jana S, Chan A, Pande A, Mohapatra P (2016) QoE prediction model for mobile video telephony. Multimed Tools Appl 75(13):7957–7980

    Article  Google Scholar 

  25. Joumblatt D, Chandrashekar J, Kveton B, Taft N (2013) Predicting user dissatisfaction with Internet application performance at end-hosts. In: INFOCOM, 2013 Proceedings IEEE, pp 235–239

  26. Klir GJ, Yuan B (1994) Fuzzy sets and fuzzy logic: theory and applications. Prentice-Hall, Inc

  27. Krishnan SS, Sitaraman RK (2013) Understanding the effectiveness of video ads: a measurement study. In: Conference on internet measurement conference, pp 149–162

  28. Krishnan SS, Sitaraman RK (2013) Video stream quality impacts viewer behavior: inferring causality using quasi-experimental designs. IEEE/ACM Trans Network 21(6):2001–2014

    Article  Google Scholar 

  29. Li Z, Lin J, Akodjenou MI, Xie G, Kaafar MA, Jin Y, Peng G (2012) Watching videos from everywhere: a study of the PPTV mobile VoD system. In: ACM Conference on internet measurement conference, pp 185–198

  30. Mok RKP, Chan EWW, Chang RKC (2011) Measuring the quality of experience of HTTP video streaming. In: IFIP/IEEE International symposium on integrated network management, pp 485–492

  31. Mok RKP, Chan EWW, Luo X, Chang RKC (2011) Inferring the QoE of HTTP video streaming from user-viewing activities. In: ACM SIGCOMM Workshop on measurements up the stack, pp 31–36

  32. Mongy S, Djeraba C, Dan AS (2007) On clustering users’ behaviors in video sessions. In: International conference on data mining, Dmin 2007. Las Vegas, pp 99–103

  33. Ng A (2016) Machine learning coursera

  34. Reiter U, Brunnström K, Moor KD, Larabi M, Pereira M, Pinheiro A, You J, Zgank A (2014) Factors influencing quality of experience, pp 55–72

    Chapter  Google Scholar 

  35. Rendle S (2012) Factorization machines with libFM. ACM Trans Intell Syst Technol 3(3):57

    Article  Google Scholar 

  36. Rish I (2001) An empirical study of the naive bayes classifier. J Univ Comput Sci 1(2):127

    Google Scholar 

  37. Robitza W, Raake A (2016) (Re-)actions speak louder than words? A novel test method for tracking user behavior in web video services. In: Eighth International conference on quality of multimedia experience, pp 1–6

  38. Shenoy P, Haritsa JR, Sudarshan S, Bhalotia G, Bawa M, Shah D (2000) Turbo-charging vertical mining of large databases. ACM Sigmod Record 29(2):22–33

    Article  Google Scholar 

  39. Shewhart WA, Wilks SS (2005) Applied logistic regression, 2nd edn

  40. Søgaard J, Shahid M, Pokhrel J, Brunnstrȯm K (2017) On subjective quality assessment of adaptive video streaming via crowdsourcing and laboratory based experiments. Multimed Tools Appl 76(15):16,727–16,748

    Article  Google Scholar 

  41. Song HH, Ge Z, Mahimkar A, Wang J, Yates J, Zhang Y, Basso A, Chen M (2011) Q-score: proactive service quality assessment in a large IPTV system. In: ACM SIGCOMM Conference on internet measurement conference, pp 195–208

  42. Trouleau W, Ashkan A, Ding W, Eriksson B (2016) Just one more: modeling binge watching behavior. In: ACM SIGKDD International conference on knowledge discovery and data mining, pp 1215–1224

  43. Xiao Z, Xu Y, Feng H, Yang T, Hu B, Zhou Y (2015) Modeling streaming QoE in wireless networks with large-scale measurement of user behavior. In: 2015 IEEE Global communications conference (GLOBECOM), pp 1–6

  44. Yu H, Zheng D, Zhao BY, Zheng W (2006) Understanding user behavior in large-scale video-on-demand systems. In: Eurosys conference, pp 333–344

  45. Yue T, Wei AM, Wang HB, Deng XD, Cheng SD (2016) A comprehensive data-driven approach to evaluating quality of experience on large-scale Internet video service. In: 2016 12th International Conference on natural computation, fuzzy systems and knowledge discovery (ICNC-FSKD). IEEE, pp 1479–1486

  46. Zaki MJ, Hsiao CJ (2002) Charm: an efficient algorithm for closed itemset mining. In: Siam International conference on data mining. Arlington, pp 457–473

  47. Zaki MJ, Parthasarathy S, Ogihara M, Li W et al (1997) New algorithms for fast discovery of association rules. In: KDD, vol 97, pp 283–286

  48. Zhang D, Zhou D, Jin X (2017) A content-adaptive video quality assessment method for online media service. Multimed Tools Appl 76(5):7175–7195

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China

    Ting Yue, Hongbo Wang & Shiduan Cheng

Authors
  1. Ting Yue
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongbo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shiduan Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongbo Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yue, T., Wang, H. & Cheng, S. Learning from users: a data-driven method of QoE evaluation for Internet video. Multimed Tools Appl 77, 27269–27300 (2018). https://doi.org/10.1007/s11042-018-5918-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-5918-4

Keywords

Search

Navigation