Abstract
Deep learning models trained using massive amounts of data, tend to capture one view of the data and its associated mapping. Different deep learning models built on the same training data may capture different views of the data based on the underlying techniques used. For explaining the decisions arrived by Black box deep learning models, we argue that it is essential to reproduce that model’s view of the training data faithfully. This faithful reproduction can then be used for explanation generation. We investigate two methods for data-view extraction: Hill Climbing approach and a GAN-driven approach. We then use this synthesized data for explanation generation by using methods such as Decision-Trees and Permutation Importance. We evaluate these approaches on a Black box model trained on public datasets and show its usefulness in explanation generation.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Adadi, A., Berrada, M.: Peeking inside the black-box: a survey on explainable artificial intelligence (xai). IEEE Access 6, 52138–52160 (2018)
Bastani, O., Kim, C., Bastani, H.: Interpretability via model extraction. CoRR abs/1706.09773 (2017)
Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., Abbeel, P.: Infogan: interpretable representation learning by information maximizing generative adversarial nets. In: Proceedings of the 30th International Conference on NIPS, pp. 2180–2188 (2016)
Friedman, J.H., Popescu, B.E.: Predictive learning via rule ensembles. Ann. Appl. Stat. 2(3), 916–954 (2008)
Gilpin, L.H., Bau, D., Yuan, B.Z., Bajwa, A., Specter, M., Kagal, L.: Explaining explanations: an overview of interpretability of machine learning. In: 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), pp. 80–89 (2018)
Guo, W., Huang, S., Tao, Y., Xing, X., Lin, L.: Explaining deep learning models-a bayesian non-parametric approach. In: Neural Information Processing Systems. NIPS 2018 (2018)
Lakkaraju, H., Bach, S.H., Leskovec, J.: Interpretable decision sets: a joint framework for description and prediction. In: Proceedings of the 22Nd ACM SIGKDD. KDD’2016 (2016)
Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. In: Neural Information Processing Systems (NIPS 2017), vol. 30 (2017)
Murdoch, W.J., Singh, C., Kumbier, K., Abbasi-Asl, R., Yu, B.: Definitions, methods, and applications in interpretable machine learning. In: Proceedings of the National Academy of Sciences (2019)
Puri, N., Gupta, P., Agarwal, P., Verma, S., Krishnamurthy, B.: MAGIX: model agnostic globally interpretable explanations. CoRR abs/1706.07160 (2017)
Ribeiro, M.T., Singh, S., Guestrin, C.: “Why should I trust you?”: explaining the predictions of any classifier. In: SIGKDD (2016)
Sangroya, A., Anantaram, C., Rawat, M., Rastogi, M.: Using formal concept analysis to explain black box deep learning classification models. In: IJCAI-19 Workshop, FCA4AI, pp. 19–26 (2019)
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: visual explanations from deep networks via gradient-based localization. In: IJCV (2019)
Shokri, R., Stronati, M., Song, C., Shmatikov, V.: Membership inference attacks against machine learning models. In: 2017 IEEE Symposium on Security and Privacy (SP) (2017)
Shrikumar, A., Greenside, P., Kundaje, A.: Learning important features through propagating activation differences. CoRR abs/1704.02685 (2017)
Smilkov, D., Thorat, N., Kim, B., Viégas, F.B., Wattenberg, M.: Smoothgrad: removing noise by adding noise. CoRR abs/1706.03825 (2017)
Wang, F., Rudin, C.: Falling rule lists. In: AISTATS (2015)
Acknowledgements
We thank the Infosys Center of Artificial Intelligence, IIIT-Delhi for support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Patir, R., Singhal, S., Anantaram, C., Goyal, V. (2020). Interpretability of Black Box Models Through Data-View Extraction and Shadow Model Creation. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Communications in Computer and Information Science, vol 1333. Springer, Cham. https://doi.org/10.1007/978-3-030-63823-8_44
Download citation
DOI: https://doi.org/10.1007/978-3-030-63823-8_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63822-1
Online ISBN: 978-3-030-63823-8
eBook Packages: Computer ScienceComputer Science (R0)