skip to main content
research-article

Users' (Dis)satisfaction with the personalTV application: Combining objective and subjective data

Published: 14 November 2011 Publication History

Abstract

The overabundance of content on online video platforms has made intelligent recommender systems that assist users in finding content matching their personal preferences indispensable. This article reports on a study in which “PersonalTV,” an online video recommendation application that has been developed for research purposes, was evaluated by a panel of test users for the first time. In view of this, objective implicit and subjective explicit user feedback were triangulated. The “PersonalTV” application enables its users to explore and watch videos from the YouTube library. It builds up a personal viewing profile in order to give personalized content suggestions. We investigated the relation between the recommended content and the consumption percentage (RQ 1), between the recommended content and the reported satisfaction (RQ 2), and explored whether these objective and subjective measures converge (RQ 3). Additional user feedback that may help to improve the application was collected.

References

[1]
AFP. 2010. 35 hours of video a minute uploaded to YouTube. http://www.google.com/hostednews/afp/article/ALeqM5hL4UMqXBKBTfJ2PjHINPGpWZe82w?docId=CNG.7a039cc7305a51102e864beb3aa51545.181.
[2]
Agrawal, R., Imienlinski, T., and Swami, A. 1993. Mining association rules between sets of items in large databases. SIGMOD Rec. 22, 207--216.
[3]
Ali, K. and Stam, W. V. 2004. TiVo: Making show recommendations using a distributed collaborative filtering architecture. In Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
[4]
Breese, J., Heckerman, D., and Kadie, C. 1998. Empirical analysis of predictive algorithms for collaborative filtering. In Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence. 43--52.
[5]
Burgess, J. and Green, J. 2009. YouTube: Online Video and Participatory Culture. Polity Press, Cambridge.
[6]
Burke, R. 2007. Hybrid web recommender systems. In The Adaptive Web, P. Brusilovsky, A. Kobsa and W. Nejdl, Eds. Springer, 377--408.
[7]
Castagnos, S., Jones, N., and Pu, P. 2009. Recommenders' influence on buyers' decision process. In Proceedings of the 3rd ACM Conference on Recommender Systems.
[8]
Cha, M., Kwak, H., Rodriguez, P., Ahn, Y.-Y., and Moon, S. 2007. I tube, you tube, everybody tubes: analyzing the world's largest user generated content video system. In Proceedings of the Internet Measurement Conference. 1--14.
[9]
Chatzopoulou, G., Sheng, C., and Faloutsos, M. 2010. A first step towards understanding popularity in YouTube. In Proceedings of the INFOCOM IEEE Conference on Computer Communications Workshops. 1--6.
[10]
Cheng, X., Dale, C., and Liu, J. 2007. Understanding the characteristics of internet short video sharing: Youtube as a case study. arxiv.org/pdf/0707.31670.
[11]
Chorianopoulos, K. 2008. Personalized and mobile digital TV applications. Multimed. Tools Appl. 36, 1--10.
[12]
Claypool, M., Gokhale, A., Miranda, R., Murnikov, P., Netes, D., and Sartin, M. 1999. Combining content-based and collaborative filters in an online newspaper. In Proceedings of ACM SIGIR Workshop on Recommender Systems.
[13]
Courtois, C., Mechant, P., and De Marez, L. 2011. Teenage uploaders on YouTube: Networked public expectancies, online feedback preference and received on-platform feedback. Cyberpsychol. Behav. Soc. Netw.
[14]
Davidson, J., Liebald, B., Liu, J., Nandy, P., Vleet, T. V., Gargi, U., Gupta, S., He, Y., Lambert, M., Livingston, B., and Sampath, D. 2010. The YouTube video recommendation system. In Proceedings of the 4th ACM Conference on Recommender Systems.
[15]
De Marez, L. and Verleye, G. 2004. Innovation diffusion: The need for more accurate consumer insight. Illustration of the PSAP scale as a segmentation instrument. J. Target. Measur. Anal. Market. 13, 32--49.
[16]
De Pessemier, T., De Moor, K., Ketykó, I., Joseph, W., De Marez, L., and Martens, L. 2011. Investigating the influence of QoS on personal evaluation behaviour in a mobile context. Multimed. Tools Appl. 1--24.
[17]
De Pessemier, T., Deryckere, T., and Martens, L. 2009. Context-Aware recommendations for user-generated content on a social network site. In Proceedings of the 7th European Conference on European Interactive Television Conference. ACM, New York.
[18]
Dieberger, A., Dourish, P., Hk, K., Resnick, P., and Wexelblat, A. 2000. Social navigation: Techniques for building more usable systems. Interact. 7, 36--45.
[19]
Gill, P., Arlitt, M., Li, Z., and Mahanti, A. 2007. Youtube traffic characterization: A view from the edge. In Proceedings of the 7th ACM SIGCOMM Conference on Internet Measurement. ACM, 15--28.
[20]
Gill, P., Arlitt, M., Li, Z., and Mahanti, A. 2008. Characterizing user sessions on YouTube. In Proceedings of the Conference on Multimedia Computing and Networking.
[21]
Good, N., Schafer, J. B., Konstan, J. A., Borchers, A., Sarwar, B., Herlocker, J., and Riedl, J. 1999. Combining collaborative filtering with personal agents for better recommendations. In Proceedings of the 16th National Conference on Artificial Intelligence and the 11th Innovative Applications of Artificial Intelligence Conference Innovative Applications of Artificial Intelligence. American Association for Artificial Intelligence.
[22]
Greenaway, S., Thelwall, M., and Ding, Y. 2009. Tagging YouTube: A classification of tagging practice on YouTube. In Proceedings of the 12th International Conference on Scientometrics and Infometrics. 660--664.
[23]
Haridakis, P. and Hanson, G. 2009. Social interaction and co-viewing with YouTube: Blending mass communication reception and social connection. J. Broadcast. Electron. Media 53, 317--335.
[24]
Heckerman, D., Chickering, D. M., Meek, C., Rounthwaite, R., and Kadie, C. 2001. Dependency networks for inference, collaborative filtering, and data visualization. J. Mach. Learn. Res. 1, 49--75.
[25]
Herlocker, J. L., Konstan, J. A., Terveen, L. G., and Riedl, J. T. 2004. Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. 22, 5--53.
[26]
Huffman, S. 2008. Search evaluation at Google. http://googleblog.blogspot.com/2008/09/search-evaluation-at-google.html.
[27]
Jones, J. 2010. The me in media: A functionalist approach to examining motives to produce within the public space of YouTube. Tech. rep. University of Minnesota, Minnesota.
[28]
Jones, Q., David, G., and Rafaeli, S. 2004. Information overload and the message dynamics of online interaction spaces. Inf. Syst. Res. 15, 194--210.
[29]
Krishnan, V., Narayanashetty, P. K., Nathan, M., Davies, R. T., and Konstan, J. A. 2008. Who predicts better?: Results from an online study comparing humans and an online recommender system. In Proceedings of the ACM Conference on Recommender Systems. ACM.
[30]
Kruitbosch, G. and Nack, F. 2008. Broadcast yourself on YouTube: Really? In Proceedings of the 3rd ACM International Workshop on Human-Centered Computing. ACM.
[31]
Lange, P. G. 2007. Publicly private and privately public: Social networking on YouTube. J. Comput.-Mediat. Comm. 13, 361--380.
[32]
Lerman, K. and Jones, L. 2007. Social browsing on Flickr. In Proceedings of the 1st International Conference on Weblogs and Social Media (ICWSM-07).
[33]
Maia, M., Almeida, J., and Almeida, V. 2008. Identifying user behavior in online social networks. In Proceedings of the Eurosys '08 1st Workshop on Social Network Systems. ACM, 1--6.
[34]
Martinez, A., Arias, J., Vilas, A., Garcia Duque, J., and Lopez Nores, M. 2009. What's on TV tonight? An efficient and effective personalized recommender system of TV programs. IEEE Trans. Consum. Electron. 55, 286--294.
[35]
Masthoff, J. 2004. Group modeling: Selecting a sequence of television items to suit a group of viewers. User Model. User-Adapted Interact. 14, 37--85.
[36]
Milliken, M., Gibson, K., and O'Donnell, S. 2008. User-Generated video and the online public sphere: Will YouTube facilitate digital freedom of expression in Atlantic Canada. Amer. Comm. J. 10.
[37]
Mooney, R. J. and Roy, L. 2000. Content-Based book recommending using learning for text categorization. In Proceedings of the 5th ACM Conference on Digital Libraries. ACM.
[38]
O'Donovan, J. and Smyth, B. 2005. Trust in recommender systems. In Proceedings of the 10th International Conference on Intelligent User Interfaces.
[39]
O'Neill, M. 2010. Vimeo users, plays and uploads doubled in 2010. In Social Times: Your Social Media Source.
[40]
Paolillo, J. C. 2008. Structure and network in the YouTube core. In 41st Annual Hawaii International Conference on System Sciences. 156--158.
[41]
Papagelis, M., Plexousakis, D., and Kutsuras, T. 2005. Alleviating the sparsity problem of collaborative filtering using trust inferences. In Trust Management. Springer, 224--239.
[42]
Pazzani, M. 1999. A framework for collaborative, content-based and demographic filtering. Artif. Intell. Rev. 13, 393--408.
[43]
Popescul, A., Ungar, L. H., Pennock, D. M., and Lawrence, S. 2001. Probabilistic models for unified collaborative and content-based recommendation in sparse-data environments. In Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence. Morgan Kaufmann Publishers.
[44]
Pu, P. and Chen, L. 2010. A user-centric evaluation framework of recommender systems. In Proceedings of UCERSTI Workshop of RecSys'10. 14--21.
[45]
Resnick, P. and Varian, H. R. 1997. Recommender systems. Comm. ACM 40, 56--58.
[46]
Salton, G. and McGill, M. G. 1986. Introduction to Modern Information Retrieval. McGraw-Hill, New York.
[47]
Sarwar, B., Karypis, G., Konstan, J., and Riedl, J. 2000. Application of dimensionality reduction in recommender systems--A case study. www.siteyoulike.org/user/bj/article/430333.
[48]
Siersdorfer, S., Pedro, J. S., Chelaru, S., and Nejdl, W. 2010. How useful are your comments? Analyzing and predicting YouTube comments and comment ratings. In 10th International World Wide Web Conference (WWW). 891--900.
[49]
Swearingen, K. and Sinha, R. 2002. Interaction design for recommender systems. In Proceedings of DIS2002 Conference.
[50]
Ungar, L. H. and Foster, D. P. 1998. Clustering methods for collaborative filtering. In Proceedings of the Workshop on Recommendation Systems. AAAI Press.
[51]
Warr, W. A. 2008. Social software: fun and games, or business tools. J. Inf. Sci. 34, 591--604.
[52]
Weng, J., Miao, C., Goh, A., Shen, Z., and Gay, R. 2006. Trust-Based agent community for collaborative recommendation. In Proceedings of the 5th International Joint Conference oN Autonomous Agents and Multiagent Systems.
[53]
YOUTUBE. 2010. YouTube statistics. http://www.youtube.com/t/press_statistics.

Cited By

View all
  • (2015)Lightweight Adaptation of Classifiers to Users and Contexts: Trends of the Emerging DomainThe Scientific World Journal10.1155/2015/4348262015:1Online publication date: 10-Sep-2015

Index Terms

  1. Users' (Dis)satisfaction with the personalTV application: Combining objective and subjective data

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image Computers in Entertainment
      Computers in Entertainment   Volume 9, Issue 3
      Theoretical and Practical Computer Applications in Entertainment
      November 2011
      196 pages
      EISSN:1544-3574
      DOI:10.1145/2027456
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 14 November 2011
      Published in CIE Volume 9, Issue 3

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Facebook application
      2. PersonalTV
      3. YouTube
      4. objective and subjective user feedback
      5. online video
      6. recommender systems
      7. user evaluation

      Qualifiers

      • Research-article
      • Research
      • Refereed

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)2
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 17 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2015)Lightweight Adaptation of Classifiers to Users and Contexts: Trends of the Emerging DomainThe Scientific World Journal10.1155/2015/4348262015:1Online publication date: 10-Sep-2015

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media