Skip to main content
SpringerLink
Log in

How useful is social feedback for learning to rank YouTube videos?

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

A vast amount of social feedback expressed via ratings (i.e., likes and dislikes) and comments is available for the multimedia content shared through Web 2.0 platforms. However, the potential of such social features associated with shared content still remains unexplored in the context of information retrieval. In this paper, we first study the social features that are associated with the top-ranked videos retrieved from the YouTube video sharing site for the real user queries. Our analysis considers both raw and derived social features. Next, we investigate the effectiveness of each such feature for video retrieval and the correlation between the features. Finally, we investigate the impact of the social features on the video retrieval effectiveness using state-of-the-art learning to rank approaches. In order to identify the most effective features, we adopt a new feature selection strategy based on the Maximal Marginal Relevance (MMR) method, as well as utilizing an existing strategy. In our experiments, we treat popular and rare queries separately and annotate 4,969 and 4,949 query-video pairs from each query type, respectively. Our findings reveal that incorporating social features is a promising approach for improving the retrieval performance for both types of queries.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Alcântara, O.D.A., Pereira Jr., Á.R., de Almeida, H.M., Gonçalves, M.A., Middleton, C., Baeza-Yates, R.A.: Wcl2r: a benchmark collection for learning to rank research with clickthrough data. JIDM 1(3), 551–566 (2010)

    Google Scholar 

  2. Bar-Yossef, Z., Gurevich, M.: Mining search engine query logs via suggestion sampling. Proc. VLDB Endow. 1, 54–65 (2008)

    Article  Google Scholar 

  3. Cambazoglu, B.B., Zaragoza, H., Chapelle, O., Chen, J., Liao, C., Zheng, Z., Degenhardt, J.: Early exit optimizations for additive machine learned ranking systems. In: WSDM, pp. 411–420 (2010)

  4. Cao, Z., Qin, T., Liu, T.Y., Tsai, M.F., Li, H.: Learning to rank: from pairwise approach to listwise approach. In: ICML, pp. 129–136 (2007)

  5. Carbonell, J.G., Goldstein, J.: The use of MMR, diversity-based reranking for reordering documents and producing summaries. In: SIGIR, pp. 335–336 (1998)

  6. Cha, M., Kwak, H., Rodriguez, P., Ahn, Y.Y., Moon, S.: Analyzing the video popularity characteristics of large-scale user generated content systems. IEEE/ACM Trans. Networking 17(5), 1357–1370 (2009)

    Article  Google Scholar 

  7. Chapelle, O., Chang, Y.: Yahoo! learning to rank challenge overview. JMLR - Proceedings Track 14, 1–24 (2011)

    Google Scholar 

  8. Chelaru, S., Altingovde, I.S., Siersdorfer, S.: Analyzing the polarity of opinionated queries. In: Proc. of ECIR’12, pp. 463–467 (2012)

  9. Chelaru, S.V., Orellana-Rodriguez, C., Altingovde, I.S.: Can social features help learning to rank youtube videos? In: WISE, pp. 552–566 (2012)

  10. Cheng, X., Dale, C., Liu, J.: Statistics and social network of youtube videos. In: Proc. of IEEE IWQoS’08 (2008)

  11. Cunningham, S.J., Nichols, D.M.: How people find videos. In: JCDL, pp. 201–210 (2008)

  12. Dang, V., Croft, W.B.: Feature selection for document ranking using best first search and coordinate ascent. In: Proc. of SIGIR’10 Workshop on Feature Generation and Selection for Information Retrieval (2010)

  13. Davidson, J., Liebald, B., Liu, J., Nandy, P., Vleet, T.V., Gargi, U., Gupta, S., He, Y., Lambert, M., Livingston, B., Sampath, D.: The youtube video recommendation system. In: RecSys, pp. 293–296 (2010)

  14. Esuli, A., Sebastiani, F.: Sentiwordnet: a publicly available lexical resource for opinion mining. In: Proc. of LREC’06, pp. 417–422 (2006)

  15. Fagin, R., Kumar, R., Sivakumar, D.: Comparing top k lists. SIAM J. Discret. Math. 17(1), 134–160 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  16. Filipova, K., Hall, K.: Improved video categorization from text metadata and user comments. In: Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 835–842 (2011)

  17. Freund, Y., Iyer, R., Schapire, R.E., Singer, Y.: An efficient boosting algorithm for combining preferences. J. Mach. Learn. Res. 4, 933–969 (2003)

    MathSciNet  Google Scholar 

  18. Friedman, J.H.: Stochastic gradient boosting. Comput. Stat. Data Anal. 38(4), 367–378 (2002)

    Article  MATH  Google Scholar 

  19. Geng, X., Liu, T.Y., Qin, T., Li, H.: Feature selection for ranking. In: Proc. of SIGIR’07, pp. 407–414 (2007)

  20. Giannopoulos, G., Weber, I., Jaimes, A., Sellis, T.K.: Diversifying user comments on news articles. In: WISE, pp. 100–113 (2012)

  21. Grace, J., Gruhl, D., Haas, K., Nagarajan, M., Robson, C., Sahoo, N.: Artist ranking through analysis of online community comments. Tech. rep., IBM Research Technical Report (2008)

  22. Hsu, C.F., Khabiri, E., Caverlee, J.: Ranking comments on the social web. In: Proc. of CSE’09, pp. 90–97 (2009)

  23. Hu, M., Sun, A., Lim, E.P.: Comments-oriented document summarization: understanding documents with readers’ feedback. In: Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 291–298 (2008)

  24. Jain, V., Varma, M.: Learning to re-rank: query-dependent image re-ranking using click data. In: WWW, pp. 277–286 (2011)

  25. Joachims, T.: Training linear svms in linear time. In: KDD’06, pp. 217–226 (2006)

  26. Lee, C., Lee, G.G.: MMR-based feature selection for text categorization. In: Proceedings of HLT-NAACL (2004)

  27. Liu, T.Y.: Learning to rank for information retrieval. Found. Trends Inf. Retr. 3(3), 225–331 (2009)

    Article  Google Scholar 

  28. Macdonald, C., Santos, R.L.T., Ounis, I.: On the usefulness of query features for learning to rank. In: CIKM, pp. 2559–2562 (2012)

  29. Merler, M., Yan, R., Smith, J.R.: Imbalanced rankboost for efficiently ranking large-scale image/video collections. In: CVPR, pp. 2607–2614 (2009)

  30. Metzler, D., Bruce Croft, W.: Linear feature-based models for information retrieval. Inf. Retr. 10(3), 257–274 (2007)

    Article  Google Scholar 

  31. Mishne, G., Glance, N.: Leave a reply: an analysis of weblog comments. In: Workshop on the Weblogging Ecosystem (2006)

  32. Mohan, A., Chen, Z., Weinberger, K.: Web-search ranking with initialized gradient boosted regression trees. J. Mach. Learn. Res. 14, 77–89 (2011)

    Google Scholar 

  33. Musial, K., Kazienko, P.: Social networks on the internet. World Wide Web 16(1), 31–72 (2013)

    Article  Google Scholar 

  34. Potthast, M., Stein, B., Loose, F., Becker, S.: Information retrieval in the commentsphere. ACM Trans. Intell. Syst. Technol 3(4), 68:1–68:21 (2012). doi:10.1145/2337542.2337553

    Article  Google Scholar 

  35. San Pedro, J., Yeh, T., Oliver, N.: Leveraging user comments for aesthetic aware image search reranking. In: WWW’12, pp. 439–448 (2012)

  36. Shmueli, E., Kagian, A., Koren, Y., Lempel, R.: Care to comment?: Recommendations for commenting on news stories. In: Proceedings of the 21st World Wide Web Conference, pp. 429–438 (2012)

  37. Siersdorfer, S., Chelaru, S., Nejdl, W., San Pedro, J.: How useful are your comments?: Analyzing and predicting youtube comments and comment ratings. In: WWW’10, pp. 891–900 (2010)

  38. Silvestri, F.: Mining query logs: turning search usage data into knowledge. Found. Trends Inf. Retr. 4(1–2), 1–174 (2010)

    Article  MATH  Google Scholar 

  39. Skobeltsyn, G., Junqueira, F., Plachouras, V., Baeza-Yates, R.A.: Resin: a combination of results caching and index pruning for high-performance web search engines. In: SIGIR, pp. 131–138 (2008)

  40. Thelwall, M., Sud, P., Vis, F.: Commenting on youtube videos: from guatemalan rock to el big bang. JASIST 63(3), 616–629 (2012)

    Article  Google Scholar 

  41. Tsagkias, M., Weerkamp, W., de Rijke, M.: News comments: exploring, modeling, and online prediction. In: Proceedings of the 32nd European Conference on IR Research, pp. 191–203 (2010)

  42. Vavliakis, K.N., Gemenetzi, K., Mitkas, P.A.: A correlation analysis of web social media. In: WIMS’11, pp. 1–5 (2011)

  43. Yano, T., Smith, N.A.: What’s worthy of comment? Content and comment volume in political blogs. In: Proceedings of the Fourth International Conference on Weblogs and Social Media (2010)

  44. Yee, W.G., Yates, A., Liu, S., Frieder, O.: Are web user comments useful for search? In: Proc. of SIGIR’09 Workshop on LSDS-IR (2009)

  45. Zaragoza, H., Cambazoglu, B.B., Baeza-Yates, R.A.: Web search solved?: All result rankings the same? In: CIKM, pp. 529–538 (2010)

Download references

Author information

Authors and Affiliations

  1. L3S Research Center, Hannover, Germany

    Sergiu Chelaru & Claudia Orellana-Rodriguez

  2. Computer Engineering Department, Middle East Technical University, Ankara, Turkey

    Ismail Sengor Altingovde

Authors
  1. Sergiu Chelaru
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Claudia Orellana-Rodriguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ismail Sengor Altingovde
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ismail Sengor Altingovde.

Additional information

This work is partially funded by the European Commission FP7 under grant agreement No. 287704 for the CUBRIK project and The Scientific and Technical Research Council of Turkey (TUBITAK) under the grant no. 113E065. I. S. Altingovde acknowledges the Yahoo! Faculty Research and Engagement Program.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chelaru, S., Orellana-Rodriguez, C. & Altingovde, I.S. How useful is social feedback for learning to rank YouTube videos?. World Wide Web 17, 997–1025 (2014). https://doi.org/10.1007/s11280-013-0258-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-013-0258-9

Keywords

Search

Navigation