research-article

News Popularity Beyond the Click-Through-Rate for Personalized Recommendations

Authors:

Ashutosh Nayak,

Rajasekhara Reddy Duvvuru MuniAuthors Info & Claims

SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 1396 - 1405

https://doi.org/10.1145/3539618.3591741

Published: 18 July 2023 Publication History

Abstract

Popularity detection of news articles is critical for making relevant recommendations for users and drive user engagement for maximum business value. Among several well-known metrics such as likes, shares, comments, Click-Through-Rate (CTR) has evolved as a default metric of popularity. However, CTR is highly influenced by the probability of news articles getting an impression, which in turn depends on the recommendation algorithm. Furthermore, it does not consider the age of the news articles, which are highly perishable and also misses out on human contextual behavioral preferences towards news. Here, we use the MIND dataset, open sourced by Microsoft to investigate the existing metrics of popularity and propose six new metrics. Our aim is to create awareness about the different perspectives of measuring popularity while discussing the advantages and disadvantages of the proposed metrics with respect to the human click behavior. We evaluated the predictability of the proposed metrics in comparison to CTR prediction. We further evaluated the utility of the proposed metrics through different test cases. Our results indicate that by using appropriate popularity metrics, we can reduce the initial news corpus (item set) by 50% and still could achieve 99% of the total clicks as compared to unfiltered news corpus based recommender systems. Similarly, our results show that we can reduce the effective number of articles recommended per impression that could improve user experience with the news platforms. The metrics proposed in this paper can be useful in other contexts, especially in recommenders with perishable items e.g. video reels or blogs.

References

[1]

Sejal Bhatia. 2021. Application and evaluation of Machine Learning for news article popularity prediction. In 2021 International Conference on Innovative Computing, Intelligent Communication and Smart Electrical Systems (ICSES). IEEE, 1--5.

[2]

Gloria Chatzopoulou, Cheng Sheng, and Michalis Faloutsos. 2010. A First Step Towards Understanding Popularity in YouTube. In 2010 INFOCOM IEEE Conference on Computer Communications Workshops. 1--6. https://doi.org/10.1109/INFCOMW.2010.5466701

[3]

Junxuan Chen, Baigui Sun, Hao Li, Hongtao Lu, and Xian-Sheng Hua. 2016. Deep ctr prediction in display advertising. In Proceedings of the 24th ACM international conference on Multimedia. 811--820.

Digital Library

[4]

Sungmin Cho, Hongjun Lim, Keunchan Park, Sungjoo Yoo, and Eunhyeok Park. 2021. On the Overlooked Significance of Underutilized Contextual Features in Recent News Recommendation Models. arXiv preprint arXiv:2112.14370 (2021).

[5]

Seyed Pouyan Eslami, Maryam Ghasemaghaei, and Khaled Hassanein. 2022. Understanding consumer engagement in social media: The role of product lifecycle. Decision Support Systems, Vol. 162 (2022), 113707.

Digital Library

[6]

Kristen Forbes. 2016. Examining the beauty industry's use of social influencers. Elon Journal of Undergraduate Research in Communications, Vol. 7, 2 (2016), 78--87.

[7]

Mouzhi Ge, Carla Delgado-Battenfeld, and Dietmar Jannach. 2010. Beyond accuracy: evaluating recommender systems by coverage and serendipity. In Proceedings of the fourth ACM conference on Recommender systems. 257--260.

Digital Library

[8]

Sayar Ghosh Roy, Anshul Padhi, Risubh Jain, Manish Gupta, and Vasudeva Varma. 2022. Towards Proactively Forecasting Sentence-Specific Information Popularity within Online News Documents. In Proceedings of the 33rd ACM Conference on Hypertext and Social Media. 11--20.

[9]

Harry Godwin. 1962. Half-life of radiocarbon. Nature, Vol. 195 (1962), 984--984.

[10]

Jyotirmoy Gope and Sanjay Kumar Jain. 2017. A survey on solving cold start problem in recommender systems. In 2017 International Conference on Computing, Communication and Automation (ICCCA). IEEE, 133--138.

[11]

Mario Haim, Anna Sophie Kümpel, and Hans-Bernd Brosius. 2018. Popularity cues in online media: A review of conceptualizations, operationalizations, and general effects. Studies in Communication| Media, Vol. 7, 2 (2018), 186--207.

[12]

Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In proceedings of the 25th international conference on world wide web. 507--517.

Digital Library

[13]

Maximilian Jenders, Gjergji Kasneci, and Felix Naumann. 2013. Analyzing and predicting viral tweets. In Proceedings of the 22nd international conference on world wide web. 657--664.

Digital Library

[14]

Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2016. Bag of Tricks for Efficient Text Classification. arXiv preprint arXiv:1607.01759 (2016).

[15]

Yaser Keneshloo, Shuguang Wang, Eui-Hong Han, and Naren Ramakrishnan. 2016. Predicting the popularity of news articles. In Proceedings of the 2016 SIAM International Conference on Data Mining. SIAM, 441--449.

[16]

Arnd Christian König, Michael Gamon, and Qiang Wu. 2009. Click-through prediction for news queries. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval. 347--354.

Digital Library

[17]

Junghyuk Lee and Jong-Seok Lee. 2018. Music popularity: Metrics, characteristics, and audio-based prediction. IEEE Transactions on Multimedia, Vol. 20, 11 (2018), 3173--3182.

Digital Library

[18]

Kristina Lerman and Tad Hogg. 2010. Using a model of social dynamics to predict popularity of news. In Proceedings of the 19th international conference on World wide web. 621--630.

Digital Library

[19]

Jian Li, Jieming Zhu, Qiwei Bi, Guohao Cai, Lifeng Shang, Zhenhua Dong, Xin Jiang, and Qun Liu. 2022. MINER: Multi-Interest Matching Network for News Recommendation. In Findings of the Association for Computational Linguistics: ACL 2022. 343--352.

[20]

Yung-Ming Li, Jyh-Hwa Liou, and Ching-Yuan Ni. 2019. Diffusing mobile coupons with social endorsing mechanism. Decision Support Systems, Vol. 117 (2019), 87--99.

[21]

Jiahui Liu, Peter Dolan, and Elin Rønby Pedersen. 2010. Personalized news recommendation based on click behavior. In Proceedings of the 15th international conference on Intelligent user interfaces. 31--40.

Digital Library

[22]

Erik Maier and Jaap Wieringa. 2021. Acquiring customers through online marketplaces? The effect of marketplace sales on sales in a retailer's own channels. International Journal of Research in Marketing, Vol. 38, 2 (2021), 311--328.

[23]

Huanru Henry Mao, Shuyang Li, Julian McAuley, and Garrison Cottrell. 2020. Speech recognition and multi-speaker diarization of long conversations. arXiv preprint arXiv:2005.08072 (2020).

[24]

Mohammad Naseri and Hamed Zamani. 2019. Analyzing and predicting news popularity in an instant messaging service. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1053--1056.

Digital Library

[25]

Shumpei Okura, Yukihiro Tagami, Shingo Ono, and Akira Tajima. 2017. Embedding-based news recommendation for millions of users. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining. 1933--1942.

Digital Library

[26]

Fernando Benjamin Perez Maurera, Maurizio Ferrari Dacrema, and Paolo Cremonesi. 2022. Towards the Evaluation of Recommender Systems with Impressions. In Proceedings of the 16th ACM Conference on Recommender Systems. 610--615.

Digital Library

[27]

Stan Phelps. [n.,d.]. Take A Cue From In-N-Out And Chick-fil-A To Withhold And Differentiate In A Sea Of Sameness. Forbes ( [n.,d.]). https://www.forbes.com/sites/stanphelps/2018/04/18/take-a-cue-from-in-n-out-and-chick-fil-a-to-withhold-and-differentiate-in-a-sea-of-sameness/'sh=6d5591b11e14

[28]

Tao Qi, Fangzhao Wu, Chuhan Wu, and Yongfeng Huang. 2021. Pp-rec: News recommendation with personalized user interest and time-aware news popularity. arXiv preprint arXiv:2106.01300 (2021).

[29]

Shaina Raza and Chen Ding. 2021. News recommender system: a review of recent progress, challenges, and opportunities. Artificial Intelligence Review (2021), 1--52.

[30]

Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 (2019).

[31]

Munish Saini, Rohan Verma, Antarpuneet Singh, and Kuljit Kaur Chahal. 2020. Investigating diversity and impact of the popularity metrics for ranking software packages. Journal of Software: Evolution and Process, Vol. 32, 9 (2020), e2265.

Digital Library

[32]

Qingquan Song, Dehua Cheng, Hanning Zhou, Jiyan Yang, Yuandong Tian, and Xia Hu. 2020. Towards automated neural interaction discovery for click-through rate prediction. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 945--955.

Digital Library

[33]

Dinuka A Soysa, Denis Guangyin Chen, Oscar C Au, and Amine Bermak. 2013. Predicting YouTube content popularity via Facebook data: A network spread model for optimizing multimedia delivery. In 2013 IEEE Symposium on Computational Intelligence and Data Mining (CIDM). IEEE, 214--221.

[34]

Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management. 1441--1450.

Digital Library

[35]

Ilya Trofimov, Anna Kornetova, and Valery Topinskiy. 2012. Using boosted trees for click-through rate prediction for sponsored search. In Proceedings of the Sixth International Workshop on Data Mining for Online Advertising and Internet Economy. 1--6.

Digital Library

[36]

Manos Tsagkias, Wouter Weerkamp, and Maarten De Rijke. 2009. Predicting the volume of comments on online news stories. In Proceedings of the 18th ACM conference on Information and knowledge management. 1765--1768.

Digital Library

[37]

Manos Tsagkias, Wouter Weerkamp, and Maarten de Rijke. 2010. News comments: Exploring, modeling, and online prediction. In European Conference on Information Retrieval. Springer, 191--203.

[38]

T Franklin Waddell. 2018. What does the crowd think? How online comments and popularity metrics affect news credibility and issue importance. New Media & Society, Vol. 20, 8 (2018), 3068--3083.

[39]

Heyuan Wang, Fangzhao Wu, Zheng Liu, and Xing Xie. 2020. Fine-grained interest matching for neural news recommendation. In Proceedings of the 58th annual meeting of the association for computational linguistics. 836--845.

[40]

Chuhan Wu, Fangzhao Wu, Suyu Ge, Tao Qi, Yongfeng Huang, and Xing Xie. 2019. Neural news recommendation with multi-head self-attention. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP). 6389--6394.

[41]

Chuhan Wu, Fangzhao Wu, Yongfeng Huang, and Xing Xie. 2022. Personalized News Recommendation: Methods and Challenges. ACM Transactions on Information Systems (TOIS) (2022).

[42]

Chuhan Wu, Fangzhao Wu, Tao Qi, and Yongfeng Huang. 2021. Empowering news recommendation with pre-trained language models. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1652--1656.

Digital Library

[43]

Fangzhao Wu, Ying Qiao, Jiun-Hung Chen, Chuhan Wu, Tao Qi, Jianxun Lian, Danyang Liu, Xing Xie, Jianfeng Gao, Winnie Wu, et al. 2020. Mind: A large-scale dataset for news recommendation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 3597--3606.

[44]

Siqi Wu, Marian-Andrei Rizoiu, and Lexing Xie. 2018. Beyond views: Measuring and predicting engagement in online videos. In Twelfth international AAAI conference on web and social media.

[45]

Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. 2019. Deep learning based recommender system: A survey and new perspectives. ACM Computing Surveys (CSUR), Vol. 52, 1 (2019), 1--38.

Digital Library

[46]

Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Deep interest evolution network for click-through rate prediction. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 5941--5948.

Digital Library

Cited By

PEREZ MAURERA FFerrari Dacrema MCastells PCremonesi P(2025)Impression-Aware Recommender SystemsACM Transactions on Recommender Systems10.1145/3712292Online publication date: 15-Jan-2025
https://dl.acm.org/doi/10.1145/3712292
Zhang GLi DGu HLu TGu N(2024)Heterogeneous Graph Neural Network with Personalized and Adaptive Diversity for News RecommendationACM Transactions on the Web10.1145/364988618:3(1-33)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.1145/3649886
Yang ZGao HGao DYang LYang LCai XNing WZhang G(2024)MLoRA: Multi-Domain Low-Rank Adaptive Network for CTR PredictionProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688134(287-297)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688134
Show More Cited By

Index Terms

News Popularity Beyond the Click-Through-Rate for Personalized Recommendations
1. Applied computing
  1. Document management and text processing

Recommendations

News Session-Based Recommendations using Deep Neural Networks
DLRS 2018: Proceedings of the 3rd Workshop on Deep Learning for Recommender Systems

News recommender systems are aimed to personalize users experiences and help them to discover relevant articles from a large and dynamic search space. Therefore, news domain is a challenging scenario for recommendations, due to its sparse user profiling, ...
A Hybrid Multi-criteria Semantic-Enhanced Collaborative Filtering Approach for Personalized Recommendations
WI-IAT '11: Proceedings of the 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Volume 01

Recommender systems aim to assist web users to find only relevant information to their needs rather than an undifferentiated mass of information. Collaborative filtering (CF) techniques are probably the most popular and widely adopted techniques in ...
Naïve filterbots for robust cold-start recommendations
KDD '06: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining

The goal of a recommender system is to suggest items of interest to a user based on historical behavior of a community of users. Given detailed enough history, item-based collaborative filtering (CF) often performs as well or better than almost any ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2023

3567 pages

ISBN:9781450394086

DOI:10.1145/3539618

General Chairs:
Hsin-Hsi Chen
National Taiwan University
,
Wei-Jou (Edward) Duh
National Taiwan University
,
Hen-Hsen Huang
Academia Sinica
,
Program Chairs:
Makoto P. Kato
Spotify
,
Josiane Mothe
Universite de Toulouse
,
Barbara Poblete
University of Chile and Amazon Visiting Academic

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 July 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Samsung Electronics

Conference

SIGIR '23

Sponsor:

SIGIR

SIGIR '23: The 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 23 - 27, 2023

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
558
Total Downloads

Downloads (Last 12 months)206
Downloads (Last 6 weeks)10

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

PEREZ MAURERA FFerrari Dacrema MCastells PCremonesi P(2025)Impression-Aware Recommender SystemsACM Transactions on Recommender Systems10.1145/3712292Online publication date: 15-Jan-2025
https://dl.acm.org/doi/10.1145/3712292
Zhang GLi DGu HLu TGu N(2024)Heterogeneous Graph Neural Network with Personalized and Adaptive Diversity for News RecommendationACM Transactions on the Web10.1145/364988618:3(1-33)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.1145/3649886
Yang ZGao HGao DYang LYang LCai XNing WZhang G(2024)MLoRA: Multi-Domain Low-Rank Adaptive Network for CTR PredictionProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688134(287-297)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688134
Deng JWang SShen DZhao LYang FZhou GMeng G(2024)A Multimodal Transformer for Live Streaming Highlight Prediction2024 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME57554.2024.10687664(1-6)Online publication date: 15-Jul-2024
https://doi.org/10.1109/ICME57554.2024.10687664

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten