skip to main content
10.1145/3640457.3688031acmconferencesArticle/Chapter ViewAbstractPublication PagesrecsysConference Proceedingsconference-collections
extended-abstract

"More to Read" at the Los Angeles Times: Solving a Cold Start Problem with LLMs to Improve Story Discovery

Published: 08 October 2024 Publication History

Abstract

News publishers, who are seeking to grow their digital audience, face a challenge in providing relevant content recommendations for unregistered users arriving directly to article pages. In these cold start scenarios, classic techniques, like asking a user to register and select topics of interest, fall short. We present a contextual targeting approach that leverages the user’s current article choice as an implicit signal of user interests. We designed and developed an interface with recommendations to help users discover more articles. Our A/B testing showed that our models increased click-through rates by 39.4% over a popularity baseline. One of them, a large language model (LLM), generates relevant recommendations that balance immersion and novelty. We discuss the implications of using LLMs for responsibly enhancing user experiences while upholding editorial standards. We identify key opportunities in detecting nuanced user preferences and identifying and interrupting filter bubbles on news publisher sites.

Supplemental Material

MP4 File - Summary Video and Sizzle Reel of the Paper "More to Read" at the Los Angeles Times: Solving a Cold Start Problem with LLMs to Improve Story Discovery
In this two minute video, authors Franklin Horn, Aurelian Alston, David Kaufman, and Won J. You give an overview of how the Los Angeles Times leveraged Large Language Models to solve a cold start problem for unregistered users. The "More to Read" feature analyzes the user's current article and offers recommendations that help a user further immerse themselves in the news. In an a/b experiment, TF/IDF and GTE showed similar positive results over a popularity baseline. The results raised interesting questions about the relationship between CTR and user interest in recommendations. Franklin Horn, Aurelia Alston, David Kaufman, and Won J. You highlight how this experience helps mitigate filter bubbles, ensuring alignment with editorial values while delivering a personalized and engaging reading experience. They point out that "values-driven design in recommendation systems isn't a constraint, it's a catalyst for promoting trust and loyalty in digital experiences."
PDF File
Supplemental visual aid of the recommendations interface, More to Read illustrating a typical user journey interacting with the recommended articles.

References

[1]
Christine Bauer, Chandni Bagchi, Olusanmi A. Hundogan, and Karin van Es. 2024. Where Are the Values? A Systematic Literature Review on News Recommender Systems. ACM Trans. Recomm. Syst. 2, 3, Article 23 (jun 2024), 40 pages. https://doi.org/10.1145/3654805
[2]
Willard Grosvenor Bleyer. 1913. Newspaper Writing and Editing. https://www.gutenberg.org/cache/epub/65884/pg65884-images.html. Release date: July 20, 2021 [eBook #65884]. Language: English. Credits: WebRover, MFR, Quentin Campbell and the Online Distributed Proofreading Team at https://www.pgdp.net. Accessed: 2024-05-15.
[3]
Chartbeat. 2020. 2020 Reader Engagement Data. https://blog.chartbeat.com/2020/12/16/2020-reader-engagement-data/. Accessed: 2024-05-15.
[4]
Lukas Havrlant and Vladik Kreinovich. 2023. A simple probabilistic explanation of term frequency-inverse document frequency (tf-idf) heuristic (and variations motivated by this explanation). https://scholarworks.utep.edu/cgi/viewcontent.cgi?article=1852&context=cs_techrep. Accessed: 2024-05-15.
[5]
Daniel Hernandez. 2024. How L.A. reached peak taco. https://www.latimes.com/food/story/2024-07-23/why-los-angeles-is-best-taco-city-in-world Los Angeles Times.
[6]
Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. Towards General Text Embeddings with Multi-stage Contrastive Learning. https://arxiv.org/abs/2308.03281. Accessed: 2024-05-15.
[7]
Carlos De Loera. 2023. Jennifer Aniston is ‘so over’ cancel culture: ‘I just don’t understand what it means’. https://www.latimes.com/entertainment-arts/story/2023-08-22/jennifer-aniston-cancel-culture. Accessed: 2024-05-15.
[8]
Los Angeles Times. 2023. L.A. Times Tops Chartbeat’s Most Engaging Stories of 2023. https://www.latimes.com/about/pressreleases/story/2023-12-13/l-a-times-chartbeat-2023-most-engaging-stories Los Angeles Times.
[9]
Los Angeles Times. 2024. About the Los Angeles Times. https://www.latimes.com/about Accessed: 2024-05-15.
[10]
Feng Lu, Anca Dumitrache, and David Graus. 2020. Beyond Optimizing for Clicks: Incorporating Editorial Values in News Recommendation. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization (Genoa, Italy) (UMAP ’20). Association for Computing Machinery, New York, NY, USA, 145–153. https://doi.org/10.1145/3340631.3394864
[11]
Emily St. Martin. 2023. Jennifer Aniston nixed intimacy coordinator in ‘Morning Show’ sex scene with Jon Hamm. https://www.latimes.com/entertainment-arts/tv/story/2023-12-12/jennifer-aniston-jon-hamm-sex-scene-intimacy-coordinator-morning-show. Accessed: 2024-05-15.
[12]
Malia Mendez. 2024. ’Stranger Things’ star says his L.A. book event was canceled because of ’antisemitic intimidation’. https://www.latimes.com/entertainment-arts/story/2024-03-06/stranger-things-brett-gelman-book-soup-event-canceled. Accessed: 2024-05-15.
[13]
Al Mamunur Rashid, George Karypis, and John Riedl. 2008. Learning preferences of new users in recommender systems: an information theoretic approach. SIGKDD Explor. Newsl. 10, 2 (2008), 90–100. https://doi.org/10.1145/1540276.1540302
[14]
Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. Mpnet: Masked and permuted pre-training for language understanding. https://proceedings.neurips.cc/paper_files/paper/2020/file/c3a690be93aa602ee2dc0ccab5b7b67e-Paper.pdf. Accessed: 2024-05-15.
[15]
Jonathan Stray, Alon Halevy, Parisa Assar, Dylan Hadfield-Menell, Craig Boutilier, Amar Ashar, Lex Beattie, Michael Ekstrand, Claire Leibowicz, Connie Moon Sehat, Sara Johansen, Lianne Kerlin, David Vickrey, Spandana Singh, Sanne Vrijenhoek, Amy Zhang, McKane Andrus, Natali Helberger, Polina Proutskova, Tanushree Mitra, and Nina Vasan. 2022. Building Human Values into Recommender Systems: An Interdisciplinary Synthesis. arxiv:2207.10192 [cs.IR] https://arxiv.org/abs/2207.10192
[16]
Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. Text Embeddings by Weakly-Supervised Contrastive Pre-training. https://arxiv.org/html/2212.03533v2. Accessed: 2024-05-15.
[17]
Richard Winton, Matt Brennan, and Connor Sheets. 2023. ‘Friends’ star Matthew Perry dead at 54, found in hot tub at L.A. home. https://www.latimes.com/california/story/2023-10-28/freinds-star-matthew-perry-dead-at-56 Los Angeles Times.
[18]
Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. 2023. C-Pack: Packaged Resources To Advance General Chinese Embedding. https://arxiv.org/pdf/2309.07597.pdf. Accessed: 2024-05-15.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
RecSys '24: Proceedings of the 18th ACM Conference on Recommender Systems
October 2024
1438 pages
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 October 2024

Check for updates

Author Tags

  1. case study
  2. human values
  3. news recommendation
  4. user engagement

Qualifiers

  • Extended-abstract
  • Research
  • Refereed limited

Conference

Acceptance Rates

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 260
    Total Downloads
  • Downloads (Last 12 months)260
  • Downloads (Last 6 weeks)13
Reflects downloads up to 18 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media