skip to main content
10.1145/2623330.2623634acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article

Modeling delayed feedback in display advertising

Published: 24 August 2014 Publication History

Abstract

In performance display advertising a key metric of a campaign effectiveness is its conversion rate -- the proportion of users who take a predefined action on the advertiser website, such as a purchase. Predicting this conversion rate is thus essential for estimating the value of an impression and can be achieved via machine learning. One difficulty however is that the conversions can take place long after the impression -- up to a month -- and this delayed feedback hinders the conversion modeling. We tackle this issue by introducing an additional model that captures the conversion delay. Intuitively, this probabilistic model helps determining whether a user that has not converted should be treated as a negative sample -- when the elapsed time is larger than the predicted delay -- or should be discarded from the training set -- when it is too early to tell. We provide experimental results on real traffic logs that demonstrate the effectiveness of the proposed model.

Supplementary Material

MP4 File (p1097-sidebyside.mp4)

References

[1]
D. Agarwal, R. Agrawal, R. Khanna, and N. Kota. Estimating rates of rare events with multiple hierarchies through scalable log-linear models. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 213--222. ACM, 2010.
[2]
O. Chapelle, E. Manavoglu, and R. Rosales. Simple and scalable response prediction for display advertising. Transactions on Intelligent Systems and Technology, 2014. To appear.
[3]
F. Denis, R. Gilleron, and F. Letouzey. Learning from positive and unlabeled examples. Theoretical Computer Science, 348(1):70--83, 2005.
[4]
C. Elkan and K. Noto. Learning classifiers from only positive and unlabeled data. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 213--220. ACM, 2008.
[5]
D. Hillard, S. Schroedl, E. Manavoglu, H. Raghavan, and C. Leggetter. Improving ad relevance in sponsored search. In Proceedings of the third ACM international conference on Web search and data mining, pages 361--370, 2010.
[6]
T. Joachims. Making large-scale support vector machine learning practical. In Advances in kernel methods, pages 169--184. MIT Press, 1999. svmlight.joachims.org.
[7]
J. D. Kalbfleisch and R. L. Prentice. The statistical analysis of failure time data, volume 360 of Wiley Series in Probability and Statistics. John Wiley & Sons, 2nd edition, 2002.
[8]
K. Lange. A gradient algorithm locally equivalent to the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological), 57(2):425--437, 1995.
[9]
K.-C. Lee, B. Orten, A. Dasdan, and W. Li. Estimating conversion rate in display advertising from past erformance data. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 768--776. ACM, 2012.
[10]
R. P. McAfee. The design of advertising exchanges. Review of Industrial Organization, 39(3):169--185, 2011.
[11]
H. B. McMahan, G. Holt, D. Sculley, et al. Ad click prediction: a view from the trenches. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1222--1230. ACM, 2013.
[12]
A. K. Menon, K.-P. Chitrapura, S. Garg, D. Agarwal, and N. Kota. Response prediction using collaborative filtering with hierarchies and side-information. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, 2011.
[13]
R. G. Miller Jr. Survival analysis, volume 66. Wiley-Interscience, 2nd edition, 1998.
[14]
S. Muthukrishnan. Ad exchanges: Research issues. In Proceedings of the 5th International Workshop on Internet and Network Economics, 2009.
[15]
J. Nocedal. Updating quasi-newton matrices with limited storage. Mathematics of computation, 35(151):773--782, 1980.
[16]
R. Rosales, H. Cheng, and E. Manavoglu. Post-click conversion modeling and analysis for non-guaranteed delivery display advertising. In Proceedings of the fifth ACM international conference on Web search and data mining, pages 293--302. ACM, 2012.
[17]
K. Weinberger, A. Dasgupta, J. Langford, A. Smola, and J. Attenberg. Feature hashing for large scale multitask learning. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 1113--1120, 2009.

Cited By

View all
  • (2025)Adapting Constrained Markov Decision Process for OCPC Bidding with Delayed ConversionsACM Transactions on Information Systems10.1145/370642043:2(1-29)Online publication date: 18-Jan-2025
  • (2025)Predicting Calibrated Conversion Rate of Online Advertising Using a Multi-task Mixture-of-Experts Calibration ModelBig Data10.1007/978-981-96-1024-2_14(189-201)Online publication date: 24-Jan-2025
  • (2024)Bootstrap your conversionsProceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence10.5555/3702676.3702743(1438-1452)Online publication date: 15-Jul-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
KDD '14: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining
August 2014
2028 pages
ISBN:9781450329569
DOI:10.1145/2623330
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 August 2014

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. conversion prediction
  2. display advertising
  3. machine learning

Qualifiers

  • Research-article

Conference

KDD '14
Sponsor:

Acceptance Rates

KDD '14 Paper Acceptance Rate 151 of 1,036 submissions, 15%;
Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)119
  • Downloads (Last 6 weeks)16
Reflects downloads up to 03 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Adapting Constrained Markov Decision Process for OCPC Bidding with Delayed ConversionsACM Transactions on Information Systems10.1145/370642043:2(1-29)Online publication date: 18-Jan-2025
  • (2025)Predicting Calibrated Conversion Rate of Online Advertising Using a Multi-task Mixture-of-Experts Calibration ModelBig Data10.1007/978-981-96-1024-2_14(189-201)Online publication date: 24-Jan-2025
  • (2024)Bootstrap your conversionsProceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence10.5555/3702676.3702743(1438-1452)Online publication date: 15-Jul-2024
  • (2024)Debiasing the Conversion Rate Prediction Model in the Presence of Delayed Implicit FeedbackEntropy10.3390/e2609079226:9(792)Online publication date: 15-Sep-2024
  • (2024)Imagine and Imitate: Cost-Effective Bidding under Partially Observable Price LandscapesBig Data and Cognitive Computing10.3390/bdcc80500468:5(46)Online publication date: 28-Apr-2024
  • (2024)Online conversion rate prediction via multi-interval screening and synthesizing under delayed feedbackProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i8.28726(8796-8804)Online publication date: 20-Feb-2024
  • (2024)Learning with Asynchronous LabelsACM Transactions on Knowledge Discovery from Data10.1145/366218618:8(1-27)Online publication date: 3-May-2024
  • (2024)Utilizing Non-click Samples via Semi-supervised Learning for Conversion Rate PredictionProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688151(350-359)Online publication date: 8-Oct-2024
  • (2024)Privacy Preserving Conversion Modeling in Data Clean RoomProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688054(819-822)Online publication date: 8-Oct-2024
  • (2024)Ads Recommendation in a Collapsed and Entangled WorldProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671607(5566-5577)Online publication date: 25-Aug-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media