Abstract
There is a growing interest in explainable machine learning methods. In our investigation, we have collected heterogeneous features from two series of YouTube video ads and seven series of Instagram picture ads to form our datasets. There are two main challenges that we found in analysing such data: i) multicollinearity and ii) infrequent common features. Due to these issues, standard estimation methods, such as OLS, Lasso, and Elastic-net, are only able to find a small number of significant features. This paper proposes a method called Significant Feature Lasso (SF-Lasso) to maximize model explainability by identifying most of the significant features that affect a target outcome (such as online video and picture ad popularity). Experiments show that SF-Lasso is able to identify much more significant features while maintaining similar prediction accuracy as what Lasso and Elastic-net can obtain. The human evaluation shows that SF-Lasso is better at identifying true features that appeal to ad viewers. We also find that the number of significant features is mainly affected by the model size (i.e., the number of active variables) and the correlations among explanatory variables.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Belloni, A., Chernozhukov, V.: Least squares after model selection in high-dimensional sparse models. Bernoulli 19, 521–547 (2013)
Chen, J., Song, L., Wainwright, M.J., Jordan, M.I.: Learning to explain: an information-theoretic perspective on model interpretation. In: ICML, vol. 80, pp. 882–891 (2018)
Doshi-Velez, F., Kim, B.: Towards a rigorous science of interpretable machine learning. arXiv, Machine Learning (2017)
Friedman, J.H., Hastie, T.J., Tibshirani, R.: Regularization paths for generalized linear models via coordinate descent. J. Stat. Softw. 33(1), 1–22 (2010)
Hara, S., Maehara, T.: Enumerate lasso solutions for feature selection. In: AAAI, pp. 1985–1991 (2017)
Harder, F., Bauer, M., Park, M.: Interpretable and differentially private predictions. In: AAAI, pp. 4083–4090 (2020)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer Series in Statistics. Springer, New York (2001)
Hastie, T., Tibshirani, R., Wainwright, M.: Statistical Learning with Sparsity: The Lasso and Generalizations (2015)
Hoerl, A.E., Kennard, R.W.: Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1), 55–67 (1970)
Kim, Y., Bum Kim, S.: Collinear groupwise feature selection via discrete fusion group regression. Pattern Recognit. 83, 1–13 (2018)
Lee, J.D., Sun, D.L., Sun, Y., Taylor, J.E.: Exact post-selection inference, with application to the lasso. Ann. Stat. 44(3), 907–927 (2016)
Lockhart, R., Taylor, J., Tibshirani, R.J., Tibshirani, R.: A significance test for the lasso. Ann. Stat. 42(2), 413–468 (2014)
Ross, A.S., Lage, I., Doshi-Velez, F.: The neural lasso: local linear sparsity for interpretable explanations. In: NIPS (2017)
Shrikumar, A., Greenside, P., Kundaje, A.: Learning important features through propagating activation differences. In: ICML, pp. 3145–3153 (2017)
Shrikumar, A., Greenside, P., Kundaje, A.: Learning important features through propagating activation differences. In: ICML, vol. 70, pp. 3145–3153 (2017)
Singh, C., Murdoch, W.J., Yu, B.: Hierarchical interpretations for neural network predictions. In: ICLR (2019)
Taylor, J., Tibshirani, R.: Post-selection inference for \(l\)1-penalized likelihood models. Can. J. Stat. 46(1), 41–61 (2018)
Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Royal Stat. Soc. Series B (Methodol.) 58(1), 267–288 (1996)
Tu, M., Huang, K., Wang, G., Huang, J., He, X., Zhou, B.: Select, answer and explain: interpretable multi-hop reading comprehension over multiple documents. In: AAAI, pp. 9073–9080 (2020)
Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. Roy. Stat. Soc. B 67, 301–320 (2005)
Acknowledgements
This research is supported by the Australian Government Research Training Program Scholarship.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Park, E., Wong, R.K., Kwon, J., Chu, V.W. (2021). Maximizing Explainability with SF-Lasso and Selective Inference for Video and Picture Ads. In: Karlapalem, K., et al. Advances in Knowledge Discovery and Data Mining. PAKDD 2021. Lecture Notes in Computer Science(), vol 12712. Springer, Cham. https://doi.org/10.1007/978-3-030-75762-5_45
Download citation
DOI: https://doi.org/10.1007/978-3-030-75762-5_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-75761-8
Online ISBN: 978-3-030-75762-5
eBook Packages: Computer ScienceComputer Science (R0)