skip to main content
research-article

Measuring Individual Video QoE: A Survey, and Proposal for Future Directions Using Social Media

Published: 01 May 2018 Publication History

Abstract

The next generation of multimedia services have to be optimized in a personalized way, taking user factors into account for the evaluation of individual experience. Previous works have investigated the influence of user factors mostly in a controlled laboratory environment which often includes a limited number of users and fails to reflect real-life environment. Social media, especially Facebook, provide an interesting alternative for Internet-based subjective evaluation. In this article, we develop (and open-source) a Facebook application, named YouQ1, as an experimental platform for studying individual experience for videos. Our results show that subjective experiments based on YouQ can produce reliable results as compared to a controlled laboratory experiment. Additionally, YouQ has the ability to collect user information automatically from Facebook, which can be used for modeling individual experience.

References

[1]
Cisco Visual Networking Index. 2013. The zettabyte era--trends and analysis. Cisco white paper (2013).
[2]
Conviva. 2015. Experience report. Retrieved from http://www.conviva.com/conviva-viewer-experience-report/vxr-2015/.
[3]
P. Le Callet, S. Möller, and A. Perkis. 2012. Qualinet White Paper on definitions of quality of experience. ENQEMSS (COST Action IC 1003) (2012).
[4]
Y. Zhu, A. Hanjalic, and J. A. Redi. 2016. QoE prediction for enriched assessment of individual video viewing experience. ACM Mutimedia, Amsterdam.
[5]
J. A. Redi, Y. Zhu, H. de Ridder, and I. Heynderickx. 2015. How passive image viewers became active multimedia users. Springer.
[6]
ITU-R BT.500-11, Methodology for the subjective assessment of the quality of television pictures. International Telecommunication Union, Geneva, Switzerland, 4 (2002), 2.
[7]
W. Lin and C.-C. Jay Kuo. 2011. Perceptual visual quality metrics: A survey. JVCI 22, 4 (2011), 297--312.
[8]
R. Hu and P. Pu. 2011. Enhancing collaborative filtering systems with personality information. ACM Recsys.
[9]
W. Wu, L. Chen, and L. He. 2013. Using personality to adjust diversity in recommender systems. ACM Hypertext.
[10]
M. Swan. 2012. Health 2050: The realization of personalized medicine through crowdsourcing, the quantified self, and the participatory biocitizen. J. Personalized Medicine 2, 3 (2012), 93--118.
[11]
Y. Zhu, I. Heynderickx, and J. A. Redi. 2015. Understanding the role of social context and user factors in video quality of experience. Computers in Human Behavior 49 (2015), 412--426.
[12]
M. J. Scott, S. C. Guntuku, Y. Huan, W. Lin, and G. Ghinea. 2015. October. Modelling human factors in perceptual multimedia quality: On the role of personality and culture. ACM Multimedia.
[13]
S. Möller and A. Raake. 2014. Quality of experience - advanced concepts, applications and methods. Springer International Publishing Switzerland 2014.
[14]
T. Hoßfeld, C. Keimel, M. Hirth, B. Gardlo, J. Habigt, K. Diepold, and P. Tran-Gia. 2014. Best practices for QoE crowdtesting: QoE assessment with crowdsourcing. IEEE TMM 16, 2 (2014), 541--558.
[15]
S. D. Gosling and W. Mason. 2015. Internet research in psychology. AR Psychology 66 (2015), 877--902.
[16]
L. J. Skitka and E. G. Sargis. 2006. The internet as psychological laboratory. AR Psychology 57 (2006), 529--555.
[17]
M. Kosinski, S. C. Matz, S. D. Gosling, V. Popov, and D. Stillwell. 2015. Facebook as a research tool for the social sciences: Opportunities, challenges, ethical considerations, and practical guidelines. American Psychologist 70, 6 (2015), 543.
[18]
A. Doan, R. Ramakrishnan, and A. Y. Halevy. 2011. Crowdsourcing systems on the world-wide web. Communications of the ACM 54, 4 (2011), 86--96.
[19]
J. Redi, E. Siahaan, P. Korshunov, J. Habigt, and T. Hossfeld. 2015. When the crowd challenges the lab: lessons learnt from subjective studies on image aesthetic appeal. CrowdMM 2015.
[20]
K. Casler, L. Bickel, and E. Hackett. 2013. Separate but equal? A comparison of participants and data gathered via Amazon's MTurk, social media, and face-to-face behavioral testing. CHB 29, 6 (2013), 2156--2160.
[21]
R. E. Wilson, S. D. Gosling, and L. T. Graham. 2012. A review of Facebook research in the social sciences. Perspectives on Psychological Science 7, 3 (2012), 203--220.
[22]
F. Delogu, M. Franetovic, and L. Shamir. 2015. Keep me posted!: Human and machine learning analysis of facebook updates. Mobile MM Communications, 2015.
[23]
M. D. Back, J. M. Stopfer, S. Vazire, S. Gaddis, S. C. Schmukle, B. Egloff, and S. D. Gosling. 2010. Facebook profiles reflect actual personality, not self-idealization. Psychological Science (2010).
[24]
L. Gou, M. X. Zhou, and H. Yang. 2014. KnowMe and shareme: Understanding automatically discovered personality traits from social media and user sharing preferences. ACM CHI, 2014.
[25]
N. Seitz. 2003. ITU-T QoS standards for IP-based networks. IEEE Communications Magazine (2003), 82--89.
[26]
P. Brooks and B. Hestnes. 2010. User measures of quality of experience: Why being objective and quantitative is important. Network, IEEE 24, 2 (2010), 8--13.
[27]
F. Dobrian, V. Sekar, A. Awan, I. Stoica, D. Joseph, A. Ganjam, J. Zhan, and H. Zhang. 2011. Understanding the impact of video quality on user engagement. ACM Computer Communication Review (2011), 362--373.
[28]
G. Ghinea and J. Thomas. 2005. Quality of perception: user quality of service in multimedia presentations. IEEE Transactions on Multimedia 7, 4 (2005), 786--789.
[29]
S. Ickin, K. Wac, M. Fiedler, L. Janowski, J.-H. Hong, and A. K. Dey. 2012. Factors influencing quality of experience of commonly used mobile applications. IEEE Communications Magazine 50, 4 (2012), 48--56.
[30]
A. Balachandran, V. Sekar, A. Akella, S. Seshan, I. Stoica, and H. Zhang. 2012. A quest for an internet video quality-of-experience metric. In Proceedings of the 11th ACM Workshop on Hot Topics in Networks. 2012.
[31]
Q. Huynh-Thu and M. Ghanbari. 2008. Temporal aspect of perceived quality in mobile video broadcasting. IEEE Transactions on Broadcasting 54, 3 (2008), 641--651.
[32]
J. H. Westerink and J. A. Roufs. 1989. Subjective image quality as a function of viewing distance, resolution, and picture size. SMPTE Journal 98, 2 (1989), 113--119.
[33]
N. Staelens, S. Moens, W. Van den Broeck, I. Marien, B. Vermeulen, P. Lambert, R. Van de Walle, and P. Demeester. 2010. Assessing quality of experience of IPTV and video on demand services in real-life environments. IEEE Transactions on Broadcasting 56, 4 (2010), 458--466.
[34]
K. Yamori and Y. Tanaka. 2004. Relation between willingness to pay and guaranteed minimum bandwidth in multiple-priority service. Communications, 2004.
[35]
C. Owsley, R. Sekuler, and D. Siemsen. 1983. Contrast sensitivity throughout adulthood. Vision Research, 1983.
[36]
M. S. El-Nasr and S. Yan. 2006. 3dVisual attention in 3D video games. Workshop on CST, 2006.
[37]
P. Orero and A. Remael. 2007. Media for all: Subtitling for the deaf, audio description, and sign language. Rodopi, 2007.
[38]
R. E. Nisbett and Y. Miyamoto. 2005. The influence of culture: holistic versus analytic perception. Trends in Cognitive Sciences 9, 10 (2005), 467--473.
[39]
E. Balcetis and G. D. Lassiter. 2010. Social Psychology of Visual Perception. Psychology Press, 2010.
[40]
A. Acar, T. Taura, E. Yamamoto, and N. F. M. Yusof. 2011. Object vs. relation: Understanding the link between culture and cognition with the help of wordnet. Int. J. of Asian Lang. Proc 21, 4 (2011), 199--208.
[41]
I. Wechsung, M. Schulz, K.-P. Engelbrecht, J. Niemann, and S. Möller. 2011. All users are (not) equal-the influence of user characteristics on perceived quality, modality choice and performance. In Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop. Springer, New York, NY.
[42]
A. B. Naumann, I. Wechsung, and J. Hurtienne. 2010. Multimodal interaction: A suitable strategy for including older users? Interacting with Computers 22, 6 (2010), 465--474.
[43]
P. J. Silvia. 2008. Interest—The curious emotion. Curr. Dir. Psychological Science 17, 1 (2008), 57--60.
[44]
H. L. O'Brien and E. G. Toms. 2008. What is user engagement? A conceptual framework for defining user engagement with technology. JAIST 59, 6 (2008), 938--955.
[45]
P. Kortum and M. Sullivan. 2010. The effect of content desirability on subjective video quality ratings. Human Factors: The Journal of the Human Factors and Ergonomics Society 52, 1 (2010), 105--118.
[46]
J. Palhais, R. S. Cruz, and M. S. Nunes. 2011. Quality of experience assessment in internet TV. In International Conference on Mobile Networks and Management. Springer, Berlin, Heidelberg, 261--274.
[47]
A. Sackl, R. Schatz, and A. Raake. 2017. More than I ever wanted or just good enough? User expectations and subjective quality perception in the context of networked multimedia services. QUEx 2, 1 (2017), 3.
[48]
A. Sackl and R. Schatz. 2014. Got what you want? Modeling expectations to enhance web QoE prediction. Quality of Multimedia Experience (QoMEX), IEEE, 2014.
[49]
A. Sackl and R. Schatz. 2014. Evaluating the influence of expectations, price and content selection on video quality perception. Quality of Multimedia Experience (QoMEX), IEEE, 2014.
[50]
P. A. Kara, L. Bokor, A. Sackl, and M. Mourão. 2015. What your phone makes you see: Investigation of the effect of end-user devices on the assessment of perceived multimedia quality. Quality of Multimedia Experience (QoMEX), IEEE, 2015.
[51]
P. G. Engeldrum. 2000. Psychometric Scaling: A Toolkit for Imaging Systems Development. Imcotek Press, 2000.
[52]
A. G. Greenwald, D. E. McGhee, and J. L. Schwartz. 1998. Measuring individual differences in implicit cognition: the implicit association test. Journal of Personality and Social Psychology 74, 6 (1998), 1464.
[53]
C. Keimel, J. Habigt, and K. Diepold. 2012. Challenges in crowd-based video quality assessment. Quality of Multimedia Experience (QoMEX), IEEE, 2012.
[54]
T. Hoßfeld, M. Hirth, J. Redi, F. Mazza, P. Korshunov, B. Naderi, M. Seufert, B. Gardlo, S. Egger, and C. Keimel. 2014. Best practices and recommendations for crowdsourced QoE-lessons learned from the qualinet task force. Crowdsourcing (2014).
[55]
M. Weisbuch, Z. Ivcevic, and N. Ambady. 2009. On being liked on the web and in the “real world”: Consistency in first impressions across personal webpages and spontaneous behavior. JESP 45, 3 (2009), 573--576.
[56]
M. Burke, C. Marlow, and T. Lento. 2009. Feed me: Motivating newcomer contribution in social network sites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2009.
[57]
H. Li, H. Wang, J. Liu, and K. Xu. 2012. Video sharing in online social networks: measurement and analysis. In Proceedings of the International Workshop on Network and Operating System Support for Digital Audio and Video. ACM, 2012.
[58]
L. T. Graham, C. J. Sandy, and S. D. Gosling. 2011. Manifestations of individual differences in physical and virtual environments. The Wiley-Blackwell Handbook of Individual Differences (2011), 773--800.
[59]
D. Garcia and S. Sikström. 2014. The dark side of facebook: Semantic representations of status updates predict the dark triad of personality. Personality and Individual Differences 67 (2014), 92--96.
[60]
Y. Bachrach, M. Kosinski, T. Graepel, P. Kohli, and D. Stillwell. 2012. Personality and patterns of Facebook usage. In Proceedings of the 4th Annual ACM Web Science Conference. ACM, 2012.
[61]
A. Eftekhar, C. Fullwood, and N. Morris. 2014. Capturing personality from facebook photos and photo-related activities: How much exposure do you need? Computers in Human Behavior 37 (2014), 162--170.
[62]
M. Kosinski, D. Stillwell, and T. Graepel. 2013. Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences 110, 15 (2013), 5802--5805.
[63]
M. Park, M. Naaman, and J. Berger. 2016. A data-driven study of view duration on Youtube. arXiv:1603.08308 (2016).
[64]
X. Yi, L. Hong, E. Zhong, N. N. Liu, and S. Rajan. 2014. Beyond clicks: Dwell time for personalization. In Proceedings of the 8th ACM Conference on Recommender Systems. ACM, 2014.
[65]
B. Gardlo, M. Ries, T. Hossfeld, and R. Schatz. 2012. Microworkers vs. facebook: The impact of crowdsourcing platform choice on experimental results. Quality of Multimedia Experience (QoMEX), IEEE.
[66]
D. J. Stillwell and M. Kosinski. 2004. Mypersonality project: Example of successful utilization of online social networks for large-scale social research. American Psychologist 59, 2 (2004), 93--104.
[67]
H. A. Schwartz, J. Eichstaedt, and M. E. Seligman. 2013. Personality, gender, and age in the language of social media: The open-vocabulary approach. PloS one 8, 9 (2013), e73791.
[68]
D. Preoţiuc-Pietro, S. Volkova, V. Lampos, Y. Bachrach, and N. Aletras. 2015. Studying user income through language, behaviour and affect in social media. PloS one 10, 9 (2015), e0138717.
[69]
L. Zwarun and A. Hall. 2014. What's going on? Age, distraction, and multitasking during online survey taking. Computers in Human Behavior 41 (2014), 236--244.
[70]
S. R. Gulliver and G. Ghinea. 2006. Defining user perception of distributed multimedia quality. ACM Transactions on Multimedia Computing, Communications, and Applications 2, 4 (2006), 241--257.
[71]
S. D. Gosling, P. J. Rentfrow, and W. B. Swann. 2003. A very brief measure of the Big-Five personality domains. Journal of Research in Personality 37, 6 (2003), 504--528.
[72]
W. Robitza, M. N. Garcia, and A. Raake. 2015. At home in the lab: Assessing audiovisual quality of HTTP-based adaptive streaming with an immersive test paradigm. Quality of Multimedia Experience (QoMEX) IEEE, 2015.
[73]
A. Hanjalic and L.-Q. Xu. 2005. Affective video content representation and modeling. TMM, IEEE (2005).
[74]
D. Rao, D. Yarowsky, A. Shreevats, and M. Gupta. 2010. Classifying latent user attributes in Twitter. In Proceedings of the 2nd International Workshop on Search and Mining User-Generated Contents. ACM, 2010.
[75]
J. D. Burger, J. Henderson, G. Kim, and G. Zarrella. 2011. Discriminating gender on Twitter. Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2011.
[76]
S. C. Guntuku, D. B. Yaden, M. L. Kern, L. H. Ungar, and J. C. Eichstaedt. 2017. Detecting depression and mental illness on social media: an integrative review. CoBeHa 18, 43--49.
[77]
S. C. Guntuku, W. Lin, M. J. Scott, and G. Ghinea. 2015. Modelling the influence of personality and culture on affect and enjoyment in multimedia. In Proceedings of the 2015 International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 236--242.
[78]
S. C. Guntuku, M. J. Scott, H. Yang, G. Ghinea, and W. Lin. 2015. The CP-QAE-I: A video dataset for exploring the effect of personality and culture on perceived quality and affect in multimedia. In Proceedings of the 2015 7th International Workshop on Quality of Multimedia Experience (QoMEX). IEEE, 1--7.
[79]
S. C. Guntuku, J. R. Ramsay, R. M. Merchant, and L. H. Ungar. 2017. Language of ADHD in adults on social media. Journal of Attention Disorders, 1087054717738083.
[80]
S. C. Guntuku, W. Lin, J. Carpenter, W. K. Ng, L. H. Ungar, and D. Preoţiuc-Pietro. 2017. Studying personality through the content of posted and liked images on Twitter. In Proceedings of the 2017 ACM on Web Science Conference. ACM, 223--227.
[81]
S. C. Guntuku, J. T. Zhou, S. Roy, L. Weisi, and I. W. Tsang. 2016. Who likes what, and why? Insights into personality modeling based on imagelikes. IEEE Transactions on Affective Computing.
[82]
S. C. Guntuku, S. Roy, W. Lin, K. Ng, N. W. Keong, and V. Jakhetiya. 2016. Personalizing user interfaces for improving quality of experience in VoD recommender systems. In Proceedings of the 2016 8th International Conference on Quality of Multimedia Experience (QoMEX). IEEE, 1--6.
[83]
S. C. Guntuku, L. Qiu, S. Roy, W. Lin, and V. Jakhetiya. 2015. Do others perceive you as you want them to?: Modeling personality based on selfies. In Proceedings of the 1st International Workshop on Affect 8 Sentiment In Multimedia. ACM, 21--26.
[84]
M. J. Scott, S. C. Guntuku, W. Lin, and G. Ghinea. 2016. Do personality and culture influence perceived video quality and enjoyment? IEEE Transactions on Multimedia 18, 9, 1796--1807.
[85]
G. Ghinea and S. Y. Chen. 2006. Perceived quality of multimedia educational content: A cognitive style approach. Multimedia Systems 11, 3, 271--279.
[86]
S. R. Gulliver and G. Ghinea. 2010. Cognitive style and personality: Impact on multimedia perception. Online Information Review 34, 1, 39--58.
[87]
S. C. Guntuku, M. J. Scott, G. Ghinea, and W. Lin. 2016. Personality, culture, and system factors - Impact on affective response to multimedia. arXiv Preprint arXiv:1606.06873.

Cited By

View all
  • (2024)Context-Aware QoE for Mobility-Driven Applications Through Dynamic SurveysInformation10.3390/info1512079715:12(797)Online publication date: 11-Dec-2024
  • (2024)Eye Tracking and Human Influence Factors’ Impact on Quality of Experience of Mobile GamingFuture Internet10.3390/fi1611042016:11(420)Online publication date: 13-Nov-2024
  • (2024)Multiple Image Distortion DNN Modeling Individual Subject Quality AssessmentACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366419820:8(1-27)Online publication date: 29-Jun-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications
ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 14, Issue 2s
April 2018
287 pages
ISSN:1551-6857
EISSN:1551-6865
DOI:10.1145/3210485
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 May 2018
Accepted: 01 January 2018
Revised: 01 November 2017
Received: 01 June 2017
Published in TOMM Volume 14, Issue 2s

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Facebook
  2. Individual Quality of Experience
  3. user factors

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

  • China Scholarship Council (CSC)
  • Singapore MoE Tier 1 Project
  • NWO Veni

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)30
  • Downloads (Last 6 weeks)3
Reflects downloads up to 10 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Context-Aware QoE for Mobility-Driven Applications Through Dynamic SurveysInformation10.3390/info1512079715:12(797)Online publication date: 11-Dec-2024
  • (2024)Eye Tracking and Human Influence Factors’ Impact on Quality of Experience of Mobile GamingFuture Internet10.3390/fi1611042016:11(420)Online publication date: 13-Nov-2024
  • (2024)Multiple Image Distortion DNN Modeling Individual Subject Quality AssessmentACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366419820:8(1-27)Online publication date: 29-Jun-2024
  • (2024)A Theoretical Framework for Provider’s QoE Assessment using Individual and Objective QoE Monitoring2024 16th International Conference on Quality of Multimedia Experience (QoMEX)10.1109/QoMEX61742.2024.10598265(235-241)Online publication date: 18-Jun-2024
  • (2024)A Critical Review on Quality of Experience for Videos and User Engagement on Social Media Platforms2024 3rd International Conference on Digital Transformation and Applications (ICDXA)10.1109/ICDXA61007.2024.10470509(80-85)Online publication date: 29-Jan-2024
  • (2024)MRAM: Multi-scale Regional Attribute-weighting via Meta-learning for Personalized Image Aesthetics AssessmentKnowledge-Based Systems10.1016/j.knosys.2024.112546304(112546)Online publication date: Nov-2024
  • (2024)Model for Subjective Evaluation of Video Quality Considering User Perception of 5G TechnologyHuman Interface and the Management of Information10.1007/978-3-031-60114-9_22(304-319)Online publication date: 1-Jun-2024
  • (2023)2BiVQA: Double Bi-LSTM based Video Quality Assessment of UGC VideosACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3632178Online publication date: 8-Nov-2023
  • (2023)No-reference Quality Assessment for Contrast-distorted Images Based on Gray and Color-gray-difference SpaceACM Transactions on Multimedia Computing, Communications, and Applications10.1145/355535519:2(1-20)Online publication date: 6-Feb-2023
  • (2023)Assessing objective video quality in multi-screen video deliveryApplications of Digital Image Processing XLVI10.1117/12.2677614(32)Online publication date: 4-Oct-2023
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media