research-article

Automatic ad format selection via contextual bandits

Authors:

Deepak AgarwalAuthors Info & Claims

CIKM '13: Proceedings of the 22nd ACM international conference on Information & Knowledge Management

Pages 1587 - 1594

https://doi.org/10.1145/2505515.2514700

Published: 27 October 2013 Publication History

Abstract

Visual design plays an important role in online display advertising: changing the layout of an online ad can increase or decrease its effectiveness, measured in terms of click-through rate (CTR) or total revenue. The decision of which lay- out to use for an ad involves a trade-off: using a layout provides feedback about its effectiveness (exploration), but collecting that feedback requires sacrificing the immediate reward of using a layout we already know is effective (exploitation). To balance exploration with exploitation, we pose automatic layout selection as a contextual bandit problem. There are many bandit algorithms, each generating a policy which must be evaluated. It is impractical to test each policy on live traffic. However, we have found that offline replay (a.k.a. exploration scavenging) can be adapted to provide an accurate estimator for the performance of ad layout policies at Linkedin, using only historical data about the effectiveness of layouts. We describe the development of our offline replayer, and benchmark a number of common bandit algorithms.

References

[1]

D. Agarwal, B.-C. Chen, and P. Elango. Spatio-temporal models for estimating click-through rate. In WWW, pages 21--30, 2009.

Digital Library

[2]

J.-Y. Audibert, R. Munos, and C. Szepesvári. Exploration-exploitation tradeoff using variance estimates in multi-armed bandits. Theoretical Computer Science, 410(19):1876--1902, 2009.

Digital Library

[3]

P. Auer, N. Cesa-Bianchi, and P. Fischer. Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47(2--3):235--256, 2002.

Digital Library

[4]

K. Bauman, A. Kornetova, V. Topinskii, and D. Khakimova. Optimization of click-through rate prediction in the Yandex search engine. Automatic Documentation and Mathematical Linguistics, 47(2):52--58, 2013.

[5]

O. Chapelle and L. Li. An empirical evaluation of Thompson sampling. In NIPS, pages 2249--2257, 2011.

Digital Library

[6]

O. Chapelle, E. Manavoglu, and R. Rosales. Simple and scalable response prediction for display advertising. Transactions on Intelligent Systems and Technology, (to appear), 2013.

[7]

H. Cheng, E. Manavoglu, Y. Cui, R. Zhang, and J. Mao. Dynamic ad layout revenue optimization for display advertising. In Workshop on Data Mining for Online Advertising, 2012.

Digital Library

[8]

D. S. Diamond. A quantitative approach to magazine advertisement format selection. Journal of Marketing Research, 5(4):376--386, Nov 1968.

[9]

B. Edelman, M. Ostrovsky, and M. Schwarz. Internet advertising and the generalized second-price auction: Selling billions of dollars worth of keywords. American Economic Review, 97(1):242--259, 2005.

[10]

B. Edelman, M. Ostrovsky, and M. Schwarz. Internet advertising and the generalized second-price auction: Selling billions of dollars worth of keywords. American Economic Review, 99(2):430--434, 2009.

[11]

T. Graepel, J. Q. Candela, T. Borchert, and R. Herbrich. Web-scale Bayesian click-through rate prediction for sponsored search advertising in Microsoft's Bing search engine. In ICML, pages 13--20, 2010.

Digital Library

[12]

J. Langford, A. L. Strehl, and J. Wortman. Exploration scavenging. In ICML, pages 528--535, 2008.

Digital Library

[13]

J. Langford and T. Zhang. The epoch-greedy algorithm for multi-armed bandits with side information. In Advances in neural information processing systems, pages 817--824, 2007.

[14]

L. Li, W. Chu, J. Langford, and X. Wang. Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms. In I. King, W. Nejdl, and H. Li, editors, WSDM, pages 297--306. ACM, 2011.

Digital Library

[15]

H. B. McMahan, G. Holt, D. Sculley, M. Young, D. Ebner, J. Grady, L. Nie, T. Phillips, E. Davydov, D. Golovin, S. Chikkerur, D. Liu, M. Wattenberg, A. M. Hrafnkelsson, T. Boulos, and J. Kubica. Ad click prediction: a view from the trenches. In KDD, 2013.

Digital Library

[16]

M. Richardson, E. Dominowska, and R. Ragno. Predicting clicks: estimating the click-through rate for new ads. In WWW, pages 521--530, 2007.

Digital Library

[17]

R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction. IEEE Transactions on Neural Networks, 9(5):1054--1054, 1998.

Digital Library

[18]

L. Tran-Thanh, A. C. Chapman, E. M. de Cote, A. Rogers, and N. R. Jennings. Epsilon-first policies for budget-limited multi-armed bandits. In AAAI, 2010.

Digital Library

[19]

J. Vermorel and M. Mohri. Multi-armed bandit algorithms and empirical evaluation. In ECML, pages 437--448, 2005.

Digital Library

Cited By

Zhao B(2025)Optimizing Click-Through Rates in Online Advertising Using Thompson SamplingITM Web of Conferences10.1051/itmconf/2025730101273(01012)Online publication date: 17-Feb-2025
https://doi.org/10.1051/itmconf/20257301012
Cheng XMaghsudi S(2024)Distributed Consensus Algorithm for Decision-Making in Multiagent Multiarmed BanditIEEE Transactions on Control of Network Systems10.1109/TCNS.2024.339585011:4(2187-2199)Online publication date: Dec-2024
https://doi.org/10.1109/TCNS.2024.3395850
Ghoorchian SKortukov EMaghsudi S(2024)Non-Stationary Linear Bandits With Dimensionality Reduction for Large-Scale Recommender SystemsIEEE Open Journal of Signal Processing10.1109/OJSP.2024.33864905(548-558)Online publication date: 2024
https://doi.org/10.1109/OJSP.2024.3386490
Show More Cited By

Index Terms

Automatic ad format selection via contextual bandits
1. Computing methodologies
  1. Modeling and simulation
    1. Simulation theory
      1. Systems theory
2. Mathematics of computing
  1. Information theory

Recommendations

Personalized Recommendation via Parameter-Free Contextual Bandits
SIGIR '15: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval

Personalized recommendation services have gained increasing popularity and attention in recent years as most useful information can be accessed online in real-time. Most online recommender systems try to address the information needs of users by virtue ...
Is Combining Contextual and Behavioral Targeting Strategies Effective in Online Advertising?

Online targeting has been increasingly used to deliver ads to consumers. But discovering how to target the most valuable web visitors and generate a high response rate is still a challenge for advertising intermediaries and advertisers. The purpose of ...
Customer Acquisition via Display Advertising Using Multi-Armed Bandit Experiments

Firms using online advertising regularly run experiments with multiple versions of their ads since they are uncertain about which ones are most effective. During a campaign, firms try to adapt to intermediate results of their tests, optimizing what they ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '13: Proceedings of the 22nd ACM international conference on Information & Knowledge Management

October 2013

2612 pages

ISBN:9781450322638

DOI:10.1145/2505515

General Chairs:
Qi He
LinkedIn, USA
,
Arun Iyengar
IBM T.J. Watson Research Center, USA
,
Program Chairs:
Wolfgang Nejdl
L3S Research Center, Germany
,
Jian Pei
Simon Fraser University, Canada
,
Rajeev Rastogi
Amazon, India

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM'13

Sponsor:

CIKM'13: 22nd ACM International Conference on Information and Knowledge Management

October 27 - November 1, 2013

California, San Francisco, USA

Acceptance Rates

CIKM '13 Paper Acceptance Rate 143 of 848 submissions, 17%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

54
Total Citations
View Citations
618
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)7

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhao B(2025)Optimizing Click-Through Rates in Online Advertising Using Thompson SamplingITM Web of Conferences10.1051/itmconf/2025730101273(01012)Online publication date: 17-Feb-2025
https://doi.org/10.1051/itmconf/20257301012
Cheng XMaghsudi S(2024)Distributed Consensus Algorithm for Decision-Making in Multiagent Multiarmed BanditIEEE Transactions on Control of Network Systems10.1109/TCNS.2024.339585011:4(2187-2199)Online publication date: Dec-2024
https://doi.org/10.1109/TCNS.2024.3395850
Ghoorchian SKortukov EMaghsudi S(2024)Non-Stationary Linear Bandits With Dimensionality Reduction for Large-Scale Recommender SystemsIEEE Open Journal of Signal Processing10.1109/OJSP.2024.33864905(548-558)Online publication date: 2024
https://doi.org/10.1109/OJSP.2024.3386490
Brahmachari SLumbreras JTomamichel M(2024)Quantum contextual bandits and recommender systems for quantum dataQuantum Machine Intelligence10.1007/s42484-024-00189-66:2Online publication date: 12-Sep-2024
https://doi.org/10.1007/s42484-024-00189-6
Singh NSingh S(2024)A systematic literature review of solutions for cold start problemInternational Journal of System Assurance Engineering and Management10.1007/s13198-024-02359-y15:7(2818-2852)Online publication date: 14-May-2024
https://doi.org/10.1007/s13198-024-02359-y
Komiyama JImaizumi MOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)High-dimensional contextual bandit problem without sparsityProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3668272(49416-49427)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3668272
Cheng XMaghsudi S(2023)Collaborative Regret Minimization for Piecewise-Stationary Multi-Armed Bandit2023 31st European Signal Processing Conference (EUSIPCO)10.23919/EUSIPCO58844.2023.10289959(1385-1389)Online publication date: 4-Sep-2023
https://doi.org/10.23919/EUSIPCO58844.2023.10289959
Wang XWei MYao TSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)Efficient Sparse Linear Bandits under High Dimensional DataProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599329(2431-2443)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599329
Mukherjee ATajer A(2023)SPRT-Based Efficient Best Arm Identification in Stochastic BanditsIEEE Journal on Selected Areas in Information Theory10.1109/JSAIT.2023.32889884(128-143)Online publication date: 2023
https://doi.org/10.1109/JSAIT.2023.3288988
Shmueli GTafti A(2023)How to “improve” prediction using behavior modificationInternational Journal of Forecasting10.1016/j.ijforecast.2022.07.00839:2(541-555)Online publication date: Apr-2023
https://doi.org/10.1016/j.ijforecast.2022.07.008
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten