skip to main content
10.1145/3640457.3688012acmconferencesArticle/Chapter ViewAbstractPublication PagesrecsysConference Proceedingsconference-collections
extended-abstract

AI-based Human-Centered Recommender Systems: Empirical Experiments and Research Infrastructure

Published: 08 October 2024 Publication History

Abstract

This is a dissertation plan built around human-centered empirical experiments evaluating recommender systems (RecSys). We see this as an important research theme since many AI-based RecSys algorithmic studies lack real human assessment. Therefore, we do not know how they work in the wild that only human experiments can tell us. We split this extended abstract into two parts – 1) A series of individual studies focusing on open questions about different human values or recommendation algorithms. Our completed works include user control over content diversity, user appreciation on DL-RecSys algorithms, and human-LLMRec interaction study. We also propose three future works to understand news recommendation depolarization, personalized news podcast, and interactive user representation; 2) An experimentation infrastructure named POPROX. As a personalized news recommendation platform, it aims to support the longitudinal study needs from the general AI and RecSys research community.

References

[1]
Ziad Al Bawab, George H Mills, and Jean-Francois Crespo. 2012. Finding trending local topics in search queries for personalization of a recommendation system. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining. 397–405.
[2]
Leoz-Aizpuru Asier and Pedrero-Esteban Luis Miguel. 2022. Audio storytelling innovation in a digital age: The case of daily news podcasts in Spain. Information 13, 4 (2022), 204.
[3]
Joeran Beel and Stefan Langer. 2015. A comparison of offline evaluations, online evaluations, and user studies in the context of research-paper recommender systems. In Research and Advanced Technology for Digital Libraries: 19th International Conference on Theory and Practice of Digital Libraries, TPDL 2015, Poznań, Poland, September 14-18, 2015, Proceedings 19. Springer, 153–168.
[4]
Jaime Carbonell and Jade Goldstein. 1998. The use of MMR, diversity-based reranking for reordering documents and producing summaries. In Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval. 335–336.
[5]
Stefano Castriota, Marco Delmastro, and Mirco Tonin. 2020. National or local? The demand for news in Italy during COVID-19. (2020).
[6]
Zheng Chai, Zhihong Chen, Chenliang Li, Rong Xiao, Houyi Li, Jiawei Wu, Jingxu Chen, and Haihong Tang. 2022. User-aware multi-interest learning for candidate matching in recommenders. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1326–1335.
[7]
Weiyu Cheng, Yanyan Shen, Linpeng Huang, and Yanmin Zhu. 2021. Dual-embedding based deep latent factor models for recommendation. ACM Transactions on Knowledge Discovery from Data (TKDD) 15, 5 (2021), 1–24.
[8]
Lisbeth Clausen. 2004. Localizing the global:‘Domestication’processes in international news production. Media, Culture & Society 26, 1 (2004), 25–44.
[9]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
[10]
Kiran Garimella, Tim Smith, Rebecca Weiss, and Robert West. 2021. Political polarization in online news consumption. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 15. 152–162.
[11]
Soyeon Caren Han, Taejun Lim, Siqu Long, Bernd Burgstaller, and Josiah Poon. 2021. Glocal-k: Global and local kernels for recommender systems. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 3063–3067.
[12]
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web. 173–182.
[13]
Lucien Heitz, Juliane A Lischka, Alena Birrer, Bibek Paudel, Suzanne Tolmeijer, Laura Laugwitz, and Abraham Bernstein. 2022. Benefits of diverse news recommendations for democracy: A user study. Digital Journalism 10, 10 (2022), 1710–1730.
[14]
Joseph Konstan and Loren Terveen. 2021. Human-centered recommender systems: Origins, advances, challenges, and opportunities. AI Magazine 42, 3 (2021), 31–42.
[15]
Joseph A Konstan and John Riedl. 2012. Recommender systems: from algorithms to user experience. User modeling and user-adapted interaction 22 (2012), 101–123.
[16]
Labeeba Kothur and Vidushi Pandey. 2023. Role of social media news consumption in cultivating opinion polarization. Information Technology & People (2023).
[17]
Philippe Laban, Elicia Ye, Srujay Korlakunta, John Canny, and Marti Hearst. 2022. Newspod: Automatic and interactive news podcasts. In 27th International Conference on Intelligent User Interfaces. 691–706.
[18]
Ro’ee Levy. 2021. Social media, news consumption, and polarization: Evidence from a field experiment. American economic review 111, 3 (2021), 831–870.
[19]
Junjie Li, Haoran Li, and Chengqing Zong. 2019. Towards personalized review summarization via user-aware sequence network. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 6690–6697.
[20]
Weiwen Liu, Jun Guo, Nasim Sonboli, Robin Burke, and Shengyu Zhang. 2019. Personalized fairness-aware re-ranking for microlending. In Proceedings of the 13th ACM conference on recommender systems. 467–471.
[21]
Yanqin Lu, Rik Ray, Louisa Ha, and Peiqin Chen. 2020. Social media news consumption and opinion polarization on China’s trade practices: Evidence from a US national survey. International Journal of Communication 14 (2020), 18.
[22]
Katharina Ludwig, Alexander Grote, Andreea Iana, Mehwish Alam, Heiko Paulheim, Harald Sack, Christof Weinhardt, and Philipp Müller. 2023. Divided by the algorithm? the (limited) effects of content-and sentiment-based news recommendation on affective, ideological, and perceived polarization. Social Science Computer Review 41, 6 (2023), 2188–2210.
[23]
Gregory J Martin and Joshua McCrain. 2019. Local news and national politics. American Political Science Review 113, 2 (2019), 372–384.
[24]
Sean M McNee, John Riedl, and Joseph A Konstan. 2006. Making recommendations better: an analytic model for human-recommender interaction. In CHI’06 extended abstracts on Human factors in computing systems. 1103–1108.
[25]
Aditya Pal, Chantat Eksombatchai, Yitong Zhou, Bo Zhao, Charles Rosenberg, and Jure Leskovec. 2020. Pinnersage: Multi-modal user embedding framework for recommendations at pinterest. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2311–2320.
[26]
Elizabeth M Perse. 1990. Media involvement and local news effects. (1990).
[27]
Payam Pourashraf and Bamshad Mobasher. 2022. Using user’s local context to support local news. In Adjunct Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization. 359–365.
[28]
Tao Qi, Fangzhao Wu, Chuhan Wu, Peiru Yang, Yang Yu, Xing Xie, and Yongfeng Huang. 2021. HieRec: Hierarchical user interest modeling for personalized news recommendation. arXiv preprint arXiv:2106.04408 (2021).
[29]
Gautam Kishore Shahi. 2023. Political Polarization in Times of Crisis: Ideological Bias and Emotions of News Coverage of the COVID-19 Pandemic on YouTube. In PLAIS EuroSymposium on Digital Transformation. Springer, 56–73.
[30]
Jeong-Woo Son, A-Yeong Kim, and Seong-Bae Park. 2013. A location-based news article recommendation with explicit localized semantic analysis. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval. 293–302.
[31]
Harald Steck. 2018. Calibrated recommendations. In Proceedings of the 12th ACM conference on recommender systems. 154–162.
[32]
Harald Steck, Linas Baltrunas, Ehtsham Elahi, Dawen Liang, Yves Raimond, and Justin Basilico. 2021. Deep learning for recommender systems: A Netflix case study. AI Magazine 42, 3 (2021), 7–18.
[33]
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management. 1441–1450.
[34]
Ruixuan Sun, Avinash Akella, Ruoyan Kong, Moyan Zhou, and Joseph A Konstan. 2023. Interactive Content Diversity and User Exploration in Online Movie Recommenders: A Field Experiment. International Journal of Human–Computer Interaction (2023), 1–15.
[35]
Ruixuan Sun, Xinyi Li, Avinash Akella, and Joseph A Konstan. 2024. Large Language Models as Conversational Movie Recommenders: A User Study. arXiv preprint arXiv:2404.19093 (2024).
[36]
Ruixuan Sun, Xinyi Wu, Avinash Akella, Ruoyan Kong, Knijnenburg Bart, and Joseph A Konstan. 2024. What Are We Optimizing For? A Human-centric Evaluation Of Deep Learning-based Movie Recommenders. arXiv preprint arXiv:2401.11632 (2024).
[37]
Jason Weston, Ron J Weiss, and Hector Yee. 2013. Nonlinear latent factorization by embedding multiple user interests. In Proceedings of the 7th ACM conference on Recommender systems. 65–68.
[38]
Kelsey Whipple, Ivy Ashe, and Lourdes M Cueva Chacón. 2023. Examining podcast listeners’ perceptions of the journalistic functions of podcasts. Electronic News 17, 1 (2023), 40–66.
[39]
Chuhan Wu, Fangzhao Wu, Suyu Ge, Tao Qi, Yongfeng Huang, and Xing Xie. 2019. Neural news recommendation with multi-head self-attention. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP). 6389–6394.
[40]
Liwei Wu, Shuqing Li, Cho-Jui Hsieh, and James Sharpnack. 2020. SSE-PT: Sequential recommendation via personalized transformer. In Proceedings of the 14th ACM conference on recommender systems. 328–337.
[41]
Yuki Yada. 2023. News Recommendation using PLMs. https://github.com/YadaYuki/news-recommendation-llm.
[42]
Rui Yan, Jian-Yun Nie, and Xiaoming Li. 2011. Summarize what you are interested in: An optimization framework for interactive personalized summarization. In Proceedings of the 2011 conference on empirical methods in natural language processing. 1342–1351.
[43]
Guangbing Yang, Dunwei Wen, Nian-Shing Chen, Erkki Sutinen, 2012. Personalized text content summarizer for mobile learning: An automatic text summarization system with relevance based language model. In 2012 IEEE Fourth International Conference on Technology for Education. IEEE, 90–97.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
RecSys '24: Proceedings of the 18th ACM Conference on Recommender Systems
October 2024
1438 pages
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 October 2024

Check for updates

Author Tags

  1. Human-Recommender Interaction
  2. Human-centered AI
  3. Real-user Evaluation

Qualifiers

  • Extended-abstract
  • Research
  • Refereed limited

Funding Sources

Conference

Acceptance Rates

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 135
    Total Downloads
  • Downloads (Last 12 months)135
  • Downloads (Last 6 weeks)8
Reflects downloads up to 18 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media