Skip to main content
SpringerLink
Log in

Explanation of Black Box AI for GDPR Related Privacy Using Isabelle

  • Conference paper
  • First Online:
Data Privacy Management, Cryptocurrencies and Blockchain Technology (DPM 2022, CBT 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13619))

  • 371 Accesses

Abstract

In this paper, we present a methodology for constructing explanations for AI classification algorithms. The methodology consists of constructing a model of the context of the application in the Isabelle Infrastructure framework (IIIf) and an algorithm that allows to extract a precise logical rule that specifies the behaviour of the black box algorithm thus allowing to explain it. The explanation is given within the rich logical model of the IIIf. It is thus suitable for human audiences. We illustrate this and validate the methodology on the application example of credit card scoring with special relation to the right of explanation as given by the GDPR.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The latter two type definitions are omitted for brevity.

  2. 2.

    Note, that the interleaving of the CTL-operators AG and EF with logical operators, like implication \(\longrightarrow \) is only possible since we use a Higher Order logic embedding of CTL.

  3. 3.

    It is important to note that we request really only input output pairs and not a mathematical description of the black box. This is in contrast to the stronger assumptions made in the literature, for example [25] (see also the discussion in Sect. 5.1).

References

  1. Arrieta, A.B., et al.: Explainable artificial intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Inf. Fusion 58, 82–115 (2020)

    Article  Google Scholar 

  2. Belle, V., Papantonis, I.: Principles and practice of explainable machine learning. CoRR, abs/2009.11698 (2020)

    Google Scholar 

  3. Bull, S.: London boroughs mapped and ranked by residents’ credit scores - how money-savvy is your area? Accessed 22 July 2022

    Google Scholar 

  4. CHIST-ERA. Success: Secure accessibility for the internet of things (2016). http://www.chistera.eu/projects/success

  5. Gilpin, L.H., Bau, D., Yuan, B.Z., Bajwa, A., Specter, M.A., Kagal, L.: Explaining explanations: an approach to evaluating interpretability of machine learning. CoRR, abs/1806.00069 (2018)

    Google Scholar 

  6. T. is Money. How well do your neighbours manage their money? Interactive map reveals average credit scores by postcode. https://www.thisismoney.co.uk/money/cardsloans/article-3273996/How-neighbours-manage-money-Interactive-map-reveals-average-credit-scores-postcode.html. Accessed 22 July 2022

  7. Kammüller, F.: A proof calculus for attack trees in Isabelle. In: Garcia-Alfaro, J., Navarro-Arribas, G., Hartenstein, H., Herrera-Joancomartí, J. (eds.) ESORICS/DPM/CBT 2017. LNCS, vol. 10436, pp. 3–18. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67816-0_1

    Chapter  Google Scholar 

  8. Kammüller, F.: Attack trees in Isabelle. In: Naccache, D., et al. (eds.) ICICS 2018. LNCS, vol. 11149, pp. 611–628. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01950-1_36

    Chapter  Google Scholar 

  9. Kammüller, F.: Formal modeling and analysis of data protection for GDPR compliance of IoT healthcare systems. In: IEEE Systems, Man and Cybernetics, SMC 2018. IEEE (2018)

    Google Scholar 

  10. Kammüller, F.: Attack trees in Isabelle extended with probabilities for quantum cryptography. Comput. Secur. 87 (2019)

    Google Scholar 

  11. Kammüller, F.: Combining secure system design with risk assessment for IoT healthcare systems. In: Workshop on Security, Privacy, and Trust in the IoT, SPTIoT 2019, co-located with IEEE PerCom. IEEE (2019)

    Google Scholar 

  12. Kammüller, F.: Dependability engineering in Isabelle (2021). arXiv preprint, http://arxiv.org/abs/2112.04374

  13. Kammüller, F.: Explanation by automated reasoning using the Isabelle infrastructure framework (2021). arXiv preprint, http://arxiv.org/abs/2112.14809

  14. Kammüller, F.: Isabelle Insider and Infrastructure framework with Kripke strutures, CTL, attack trees, security refinement, and examples including IoT, GDPR, QKD, social networks, and credit scoring (2022). https://github.com/flokam/IsabelleAT

  15. Kammüller, F., Alvarado, C.M.: Exploring rationality of self awareness in social networking for logical modeling of unintentional insiders (2021). arXiv preprint, http://arxiv.org/abs/2111.15425

  16. Kammüller, F., Kerber, M.: Applying the Isabelle insider framework to airplane security. Sci. Comput. Program. 206 (2021)

    Google Scholar 

  17. Kammüller, F., Kerber, M., Probst, C.: Insider threats for auctions: Formal modeling, proof, and certified code. J. Wirel. Mob. Netw. Ubiquit. Comput. Dependable Appl. (JoWUA) 8(1) (2017)

    Google Scholar 

  18. Kammüller, F., Lutz, B.: Modeling and analyzing the corona-virus warning app with the Isabelle infrastructure framework. In: Garcia-Alfaro, J., Navarro-Arribas, G., Herrera-Joancomarti, J. (eds.) DPM/CBT 2020. LNCS, vol. 12484, pp. 128–144. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-66172-4_8

    Chapter  Google Scholar 

  19. Kammüller, F., Nestmann, U.: Inter-blockchain protocols with the Isabelle infrastructure framework. In: Formal Methods for Blockchain, 2nd International Workshop, co-located with CAV 2020, Open Access Series in Informatics. Dagstuhl Publishing (2020, to appear)

    Google Scholar 

  20. Kammüller, F., Wenzel, M., Paulson, L.C.: Locales a sectioning concept for Isabelle. In: Bertot, Y., Dowek, G., Théry, L., Hirschowitz, A., Paulin, C. (eds.) TPHOLs 1999. LNCS, vol. 1690, pp. 149–165. Springer, Heidelberg (1999). https://doi.org/10.1007/3-540-48256-3_11

    Chapter  Google Scholar 

  21. Myers, A.C., Liskov, B.: Complete, safe information flow with decentralized labels. In: Proceedings of the IEEE Symposium on Security and Privacy. IEEE (1999)

    Google Scholar 

  22. Nipkow, T., Wenzel, M., Paulson, L.C. (eds.): Isabelle/HOL – A Proof Assistant for Higher-Order Logic. LNCS, vol. 2283. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45949-9

    Book  MATH  Google Scholar 

  23. Pieters, W.: Explanation and trust: what to tell the user in security and AI? Ethics Inf. Technol. 13(1), 53–64 (2011)

    Article  Google Scholar 

  24. Viganó, L., Magazzeni, D.: Explainable security. In: IEEE European Symposium on Security and Privacy Workshops, EuroS &PW. IEEE (2020)

    Google Scholar 

  25. Wachter, S., Mittelstadt, B., Russell, C.: Counterfactual explanations without opening the black box: automated decisions and the GDPR. Harvard J. Law Technol. 31(2) (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Middlesex University London, London, UK

    Florian Kammüller

  2. Technische Universität Berlin, Berlin, Germany

    Florian Kammüller

Authors
  1. Florian Kammüller
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Florian Kammüller .

Editor information

Editors and Affiliations

  1. Samovar, Télécom SudParis, Institut Polytechnique de Paris, Palaiseau, France

    Joaquin Garcia-Alfaro

  2. Universitat Autonoma de Barcelona, Bellaterra, Spain

    Guillermo Navarro-Arribas

  3. Technical University of Denmark, Lyngby, Denmark

    Nicola Dragoni

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kammüller, F. (2023). Explanation of Black Box AI for GDPR Related Privacy Using Isabelle. In: Garcia-Alfaro, J., Navarro-Arribas, G., Dragoni, N. (eds) Data Privacy Management, Cryptocurrencies and Blockchain Technology. DPM CBT 2022 2022. Lecture Notes in Computer Science, vol 13619. Springer, Cham. https://doi.org/10.1007/978-3-031-25734-6_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-25734-6_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-25733-9

  • Online ISBN: 978-3-031-25734-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation