Abstract
Recently, a growing collection of interdisciplinary literature has been suggesting that algorithmic bias in recommender systems may cause severe consequences for sociotechnical systems. Audits of multiple social media platforms documented the implicit bias in content, products, information, or connections recommended to users, sometimes based on gender, sociocultural background, or political affiliation. Within the given context, YouTube’s video recommendation algorithm has been subject to a popular public debate in terms of its broad societal impact and alleged contribution to the spread of harmful content. However, the current literature still lacks a comprehensive understanding of, and broad agreement on, whether the video recommendations on YouTube are biased in favor of a small set of items in generalizable conditions. In addition, the characterization of potential recommendation bias is a non-trivial problem given the black-box characteristics of the system. This study addresses the given pair of problems by adopting a graphical probabilistic approach and assessing the structural properties of video recommendation networks. We adopt a stochastic approach and examine PageRank distributions over a diverse set of recommendation graphs (256,725 videos, 803,210 recommendations) built upon 8 distinct datasets categorized by languages, topics, and system entry-points. As a result, we find structural and systemic bias in video recommendations, although the level and behavior of biased recommendations vary between experiments. This study sets the stage for further research in creating comprehensive evaluation methodologies that would address the severe effects of bias in terms of fairness, diversity, coverage, exposure, security, and utility.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alstott, J., Bullmore, E., Plenz, D.: Power-law: a Python package for analysis of heavy-tailed distributions. PLoS ONE 9(1), e85777 (2014)
Buntain, C., Bonneau, R., Nagler, J., Tucker, J.A.: YouTube Recommendations and Effects on Sharing Across Online Social Platforms (2020). arXiv preprint arXiv:2003.00970
Clauset, A., Shalizi, C.R., Newman, M.E.: Power law distributions in empirical data. SIAM Rev. 51(4), 661–703 (2009)
COVID-19 Misinformation Repository. COSMOS Research Center (2020). https://cosmos.ualr.edu/covid-19
Faddoul, M., Chaslot, G., Farid, H.: A Longitudinal Analysis of YouTube's Promotion of Conspiracy Videos (2020). arXiv preprint arXiv:2003.03318
Galeano, K., Galeano, L., Mead, E., Spann, B., Kready, J., Agarwal, N.: The role of youtube during the 2019 Canadian federal election: a multi-method analysis of online discourse and information actors. J. Future Conflict. 2, 1–22 (2020)
Google Developers. YouTube Data API, Google (2020). https://developers.google.com/youtube/v3
Hussein, E., Juneja, P., Mitra, T.: Measuring misinformation in video search platforms: an audit study on youtube. In: Proceedings of the ACM on Human-Computer Interaction. (CSCW1), vol. 4, pp.1–27 (2020)
Imana, B., Korolova, A., Heidemann, J.: Auditing for discrimination in algorithms delivering job ads. In: Proceedings of the Web Conference 2021 (WWW 2021), Ljubljana, Slovenia. ACM (2021). https://doi.org/10.1145/3442381.3450077
Kready, J., Shimray, S.A., Hussain, M.N., Agarwal, N.: YouTube data collection using parallel processing. In: 2020 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), pp. 1119–1122. IEEE (2020)
Le Merrer, E., Trédan, G.: The topological face of recommendation. In: Cherifi, C., Cherifi, H., Karsai, M., Musolesi, M. (eds.) COMPLEX NETWORKS 2017 2017. SCI, vol. 689, pp. 897–908. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-72150-7_72
Ledwich, M., Zaitsev, A.: Algorithmic extremism: examining YouTube's rabbit hole of radicalization. First Monday (2020)
Marcoux, T., Agarwal, N., Obadimu, A., Hussain, M.N.: Understanding information operations using YouTubeTracker. In: 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pp. 709–712 (2019). https://doi.org/10.1145/3341161.3343704
Page, L., Brin, S., Motwani, R., Winograd, T.: The PageRank citation ranking: bringing order to the web. Stanford InfoLab (1999)
Juneja, P., Mitra, T.: Auditing e-commerce platforms for algorithmically curated vaccine misinformation. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI 2021). Association for Computing Machinery (2021). https://doi.org/10.1145/3411764.3445250
Ribeiro, M.H., Ottoni, R., West, R., Almeida, V.A., Meira Jr, W.: Auditing radicalization pathways on youtube. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 131–141 (2020)
Roose, K. (2019). The making of a YouTube radical. The New York Times
Roth, C., Mazières, A., Menezes, T.: Tubes and bubbles topological confinement of YouTube recommendations. PLoS ONE 15(4), e0231703 (2020)
Tufekci, Z.: YouTube, the great radicalizer. In: The New York Times, vol. 10, p. 2018 (2018)
Wakabayashi, D.: YouTube Moves to Make Conspiracy Videos Harder to Find. The New York Times (2019). https://www.nytimes.com/2019/01/25/technology/youtube-conspiracy-theory-videos.html
Acknowledgements
This research is funded in part by the U.S. National Science Foundation (OIA-1946391, OIA-1920920, IIS-1636933, ACI-1429160, and IIS-1110868), U.S. Office of Naval Research (N00014-10-1-0091, N00014-14-1-0489, N00014-15-P-1187, N00014-16-1-2016, N00014-16-1-2412, N00014-17-1-2675, N00014-17-1-2605, N68335-19-C-0359, N00014-19-1-2336, N68335-20-C-0540, N00014-21-1-2121), U.S. Air Force Research Lab, U.S. Army Research Office (W911NF-20-1-0262, W911NF-16-1-0189), U.S. Defense Advanced Research Projects Agency (W31P4Q-17-C-0059), Arkansas Research Alliance, the Jerry L. Maulden/Entergy Endowment at the University of Arkansas at Little Rock, and the Australian Department of Defense Strategic Policy Grants Program (SPGP) (award number: 2020-106-094). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the funding organizations. The researchers gratefully acknowledge the support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Kirdemir, B., Kready, J., Mead, E., Hussain, M.N., Agarwal, N., Adjeroh, D. (2021). Assessing Bias in YouTube’s Video Recommendation Algorithm in a Cross-lingual and Cross-topical Context. In: Thomson, R., Hussain, M.N., Dancy, C., Pyke, A. (eds) Social, Cultural, and Behavioral Modeling. SBP-BRiMS 2021. Lecture Notes in Computer Science(), vol 12720. Springer, Cham. https://doi.org/10.1007/978-3-030-80387-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-80387-2_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-80386-5
Online ISBN: 978-3-030-80387-2
eBook Packages: Computer ScienceComputer Science (R0)