Skip to main content
SpringerLink
Log in

Evaluation of Uplift Models with Non-Random Assignment Bias

  • Conference paper
  • First Online:
Advances in Intelligent Data Analysis XX (IDA 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13205))

Included in the following conference series:

  • 1054 Accesses

Abstract

Uplift Modeling measures the impact of an action (marketing, medical treatment) on a person’s behavior. This allows the selection of the subgroup of persons for which the effect of the action will be most noteworthy. Uplift estimation is based on groups of people who have received different treatments. These groups are assumed to be equivalent. However, in practice, we observe biases between these groups. We propose in this paper a protocol to evaluate and study the impact of the Non-Random Assignment bias (NRA) on the performance of the main uplift methods. Then we present a weighting method to reduce the effect of the NRA bias. Experimental results show that our bias reduction method significantly improves the performance of uplift models under NRA bias.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    For a reproducible purpose, codes and experiment results are available in the supplementary material [19].

  2. 2.

    http://blog.minethatdata.com/2008/03/minethatdata-e-mail-analytics-and-data.html/.

  3. 3.

    https://ods.ai/competitions/x5-retailhero-uplift-modeling/data.

  4. 4.

    https://ods.ai/tracks/df21-megafon/competitions/megafon-df21-comp/data.

  5. 5.

    https://zenodo.org/record/3653141#.YUCYEufgoW8.

  6. 6.

    When comparison with state of the art is possible, the achieved qini values without bias (\(b=0\)) are those usually found in the literature [3].

References

  1. Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)

    MathSciNet  MATH  Google Scholar 

  2. Devriendt, F., Guns, T., Verbeke, W.: Learning to rank for uplift modeling (2020)

    Google Scholar 

  3. Diemert, E.: A large scale benchmark for uplift modeling (2018)

    Google Scholar 

  4. Gerber, A.S., Green, D.P., Larimer, C.W.: Social pressure and voter turnout: evidence from a large-scale field experiment. Am. Polit. Sci. Rev. 102(1), 33–48 (2008)

    Article  Google Scholar 

  5. Guelman, L.A.: Optimal personalized treatment learning models with insurance applications (2015)

    Google Scholar 

  6. Gutierrez, P., Gérardy, J.Y.: Causal inference and uplift modelling: a review of the literature. In: PAPIs (2016)

    Google Scholar 

  7. Jacob, D.: CATE meets ML - the conditional average treatment effect and machine learning (2021)

    Google Scholar 

  8. Jaskowski, M., Jaroszewicz, S.: Uplift modeling for clinical trial data (2012)

    Google Scholar 

  9. Kennedy, E.H.: Optimal doubly robust estimation of heterogeneous causal effects (2020)

    Google Scholar 

  10. Kouw, W.M., Loog, M.: An introduction to domain adaptation and transfer learning (2019)

    Google Scholar 

  11. Li, C., et al.: Reinforcement learning for uplift modeling (2019)

    Google Scholar 

  12. Lo, V.S.Y., Pachamanova, D.A.: From predictive uplift modeling to prescriptive uplift analytics: a practical approach to treatment optimization while accounting for estimation risk. J. Mark. Anal. 3, 79–95 (2015). https://doi.org/10.1057/jma.2015.5

    Article  Google Scholar 

  13. Lopez-Paz, D., Oquab, M.: Revisiting classifier two-sample tests (2018)

    Google Scholar 

  14. Lunceford, J.K., Davidian, M.: Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study. Stat. Med. 23(19), 2937–2960 (2004)

    Article  Google Scholar 

  15. Olaya, D., Coussement, K., Verbeke, W.: A survey and benchmarking study of multitreatment uplift modeling. Data Min. Knowl. Discov. 34(2), 273–308 (2020). https://doi.org/10.1007/s10618-019-00670-y

    Article  MathSciNet  Google Scholar 

  16. Radcliffe, N.: Using control groups to target on predicted lift: building and assessing uplift model. Direct Mark. Anal. J. 14–21 (2007)

    Google Scholar 

  17. Radcliffe, N.J., Surry, P.D.: Differential response analysis: modeling true responses by isolating the effect of a single action (1999)

    Google Scholar 

  18. Radcliffe, N.J., Surry, P.D.: Real-world uplift modelling with significance-based uplift trees (2012)

    Google Scholar 

  19. Rafla, M., Voisine, N., Crémilleux, B.: Supplementary material. https://github.com/MinaWagdi/UpliftEvaluation_NRA/

  20. Rubin, D.B.: Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 66, 688–701 (1974)

    Article  Google Scholar 

  21. Rubin, D.B.: Using propensity scores to help design observational studies: application to the tobacco litigation. Health Serv. Outcomes Res. Methodol. 2(3), 169–188 (2001). https://doi.org/10.1023/A:1020363010465

    Article  Google Scholar 

  22. Rzepakowski, P., Jaroszewicz, S.: Decision trees for uplift modeling with single and multiple treatments. Knowl. Inf. Syst. 32, 303–327 (2011). https://doi.org/10.1007/s10115-011-0434-0

    Article  Google Scholar 

  23. Wilcoxon, F.: Individual comparisons by ranking methods. Biom. Bull. 1(6), 80–83 (1945). http://www.jstor.org/stable/3001968

  24. Zhao, Y., Fang, X., Simchi-Levi, D.: Uplift modeling with multiple treatments and general response types. In: Proceedings of the 2017 SIAM International Conference on Data Mining, pp. 588–596. SIAM (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Orange Labs, 22300, Lannion, France

    Mina Rafla & Nicolas Voisine

  2. UNICAEN, ENSICAEN, CNRS - UMR GREYC, Normandie Univ., 14000, Caen, France

    Mina Rafla & Bruno Crémilleux

Authors
  1. Mina Rafla
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nicolas Voisine
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bruno Crémilleux
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mina Rafla .

Editor information

Editors and Affiliations

  1. University of Rennes, Rennes, France

    Tassadit Bouadi

  2. University of Rennes, Rennes, France

    Elisa Fromont

  3. University of Munich, LMU, Munich, Germany

    Eyke Hüllermeier

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rafla, M., Voisine, N., Crémilleux, B. (2022). Evaluation of Uplift Models with Non-Random Assignment Bias. In: Bouadi, T., Fromont, E., Hüllermeier, E. (eds) Advances in Intelligent Data Analysis XX. IDA 2022. Lecture Notes in Computer Science, vol 13205. Springer, Cham. https://doi.org/10.1007/978-3-031-01333-1_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-01333-1_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-01332-4

  • Online ISBN: 978-3-031-01333-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation