Abstract
Causal Impact (CI) measurement is broadly used across the industry to inform both short- and long-term investment decisions of various types. In this paper, we apply the double machine learning (DML) methodology to estimate average and conditional average treatment effects across 100s of customer action types for ecommerce and digital businesses and 100s of millions of customers that can be used in decisions supporting those busiensses. We operationalize DML through a causal machine learning library. It uses distributed computation on Spark and is configured via a flexible, JSON-driven model configuration approach to estimate causal impacts at scale (i.e., across hundred of actions and millions of customers). We outline the DML methodology and implementation. We show examples of average treatment effect and conditional average treatment effect (i.e., customer-level) estimates values along with confidence intervals. Our validation metrics show a \(2.2\%\) gain over the baseline methods and a 2.5X gain in the computational time. Our contribution is to advance the scalable application of CI, while also providing an interface that allows faster experimentation, ability to onboard new use cases, and improved accessibility of underlying code for partner teams.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We use placeholder X,Y,Z to maintain business confidentiality.
- 2.
We anonymize actions to preserve business confidentiality.
References
Chernozhukov, V., et al.: Double/debiased machine learning for treatment and structural parameters. Economet. J. 21(1), C1–C68 (2018). https://doi.org/10.1111/ectj.12097
Sekhon, J.: The Neyman-Rubin Model of Causal Inference and Estimation via Matching Methods. The Oxford Handbook of Political Methodology (2007)
Holland, P.W.: Statistics and causal inference. J. Am. Stat. Assoc. 81(396), 945–960 (1986). https://doi.org/10.1080/01621459.1986.10478354
Neyman, J.: Sur les applications de la theorie des probabilites aux experiences agricoles: essai des principes. Master’s thesis, excerpts reprinted in English. Stat. Sci. 5, 463–472 (1923)
Rubin, D.: Causal inference using potential outcomes. J. Am. Stat. Assoc. 81(396), 945–960 (2005). https://doi.org/10.1080/01621459.1986.10478354
Chernozhukov, V., Goldman, M., Semenova, V., Taddy, M.: Orthogonal Machine Learning for Demand Estimation: High Dimensional Causal Inference in Dynamic Panels. ArXiv:1712.09988 [Stat] (2017)
Huber, P.J.: The behavior of maximum likelihood estimates under nonstandard conditions. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, vol. 5, pp. 221–233 (1967)
White, H.: A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica 48(4), 817–838 (1980)
Horvitz, D.G., Thompson, D.J.: A generalization of sampling without replacement from a Finite Universe. J. Am. Stat. Assoc. 47(260), 663–685 (1952)
Nie, X., Wager, S.: Quasi-oracle estimation of heterogeneous treatment effects. ArXiv:1712.04912 [Econ, Math, Stat] (2017)
Kennedy, E.H.: Optimal doubly robust estimation of heterogeneous causal effects. ArXiv:2004.14497 [math.ST] (2020)
Robins, J.M., Mark, S.D.: Estimating exposure effects by modelling the expectation of exposure conditional on confounders. Biometrics 48, 479–495 (1992). MR1173493
Ruth, C., et al.: Long-Term Outcomes Of Manitoba’s Insight Mentoring Program: A Comparative Statistical Analysis. Manitoba Centre for Health Policy, Winnipeg, MB (2015)
Austin, P.C., Stuart, E.A.: Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies. Stat. Med. 34, 3661–3679 (2015)
Hirano, K., Imbens, G.W.: Estimation of causal effects using propensity score weighting: an application to data on right heart catheterization. Health Serv. Outcomes Res. Method. 2, 259–278 (2001)
Abadie, A., Imbens, G.: On the failure of bootstrap for matching estimators. Econometrica 76(6), 1537–1157 (2008)
Victor, C., Demirer, M., Duflo, E., Fernandez-Val, I.: Generic machine learning inference on heterogenous treatment effects in randomized experiments. aXiv:1712.04802v6 [stat.ML] (2022)
Knaus, M.C., Lechner, M., Strittmatter, A.: Machine learning estimation of heterogeneous causal effects: empirical monte carlo evidence. Economet. J. 24(1), 134–161 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Ethical Implication
The data presented in the paper is completely anonymized. It cannot be used for inference of personal information of any kind.
The method presented in this paper falls in the domain of observational causal inference. Observational causal inference methods are used to gauge impact of things already happened. The inference methods by itself do not aid in any wrong doing. But in the unfortunate case of bad things happening to an individual (e.g., unfair economic policy/ smoking/ abuse), the causal methods can help identify the impact and help guide the recovery methods. In that sense, work presented here can be used to seek justice for the victim.
Of course, as a society we want to make sure that we do not subject individuals to an unscrupulous treatment to extract the causal impact of that treatment. Because the impact of such treatment could be adverse in some cases. But again the work presented here is used to analyze the aftermath of an action/ treatment. The type of treatments a person can be subjected to is outside the scope of current work.
Appendices
Appendix
A Sample JSON Config
We show a snippet of JSON config in Fig. 11. We can swap the specified models in the outcome and propensity step with any ML model. Likewise we can easily configure pre/post-processing steps and hyperparameters through JSON files.
A sample JSON config where we are using Ridge regression for the outcome model and the logistic regression for the propensity model.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
More, S., Kotwal, P., Chappidi, S., Mandalapu, D., Khawand, C. (2023). Double Machine Learning at Scale to Predict Causal Impact of Customer Actions. In: De Francisci Morales, G., Perlich, C., Ruchansky, N., Kourtellis, N., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Applied Data Science and Demo Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14174. Springer, Cham. https://doi.org/10.1007/978-3-031-43427-3_31
Download citation
DOI: https://doi.org/10.1007/978-3-031-43427-3_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43426-6
Online ISBN: 978-3-031-43427-3
eBook Packages: Computer ScienceComputer Science (R0)
