Skip to main content
SpringerLink
Log in

On Learning Sparse Boolean Formulae for Explaining AI Decisions

  • Conference paper
  • First Online:
NASA Formal Methods (NFM 2017)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 10227))

Included in the following conference series:

Abstract

In this paper, we consider the problem of learning Boolean formulae from examples obtained by actively querying an oracle that can label these examplesz as either positive or negative. This problem has received attention in both machine learning as well as formal methods communities, and it has been shown to have exponential worst-case complexity in the general case as well as for many restrictions. In this paper, we focus on learning sparse Boolean formulae which depend on only a small (but unknown) subset of the overall vocabulary of atomic propositions. We propose an efficient algorithm to learn these sparse Boolean formulae with a given confidence. This assumption of sparsity is motivated by the problem of mining explanations for decisions made by artificially intelligent (AI) algorithms, where the explanation of individual decisions may depend on a small but unknown subset of all the inputs to the algorithm. We demonstrate the use of our algorithm in automatically generating explanations of these decisions. These explanations will make intelligent systems more understandable and accountable to human users, facilitate easier audits and provide diagnostic information in the case of failure. The proposed approach treats the AI algorithm as a black-box oracle; hence, it is broadly applicable and agnostic to the specific AI algorithm. We illustrate the practical effectiveness of our approach on a diverse set of case studies.

S. Jha—The author is currently at SRI International.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abouzied, A., Angluin, D., Papadimitriou, C., Hellerstein, J.M., Silberschatz, A.: Learning and verifying quantified boolean queries by example. In: ACM Symposium on Principles of Database Systems, pp. 49–60. ACM (2013)

    Google Scholar 

  2. Angluin, D., Computational learning theory: survey and selected bibliography. In: ACM Symposium on Theory of Computing, pp. 351–369. ACM (1992)

    Google Scholar 

  3. Angluin, D., Kharitonov, M.: When won’t membership queries help? In: ACM Symposium on Theory of Computing, pp. 444–454. ACM (1991)

    Google Scholar 

  4. Bittner, B., Bozzano, M., Cimatti, A., Gario, M., Griggio, A.: Towards pareto-optimal parameter synthesis for monotonie cost functions. In: FMCAD, pp. 23–30, October 2014

    Google Scholar 

  5. Boigelot, B., Godefroid, P.: Automatic synthesis of specifications from the dynamic observation of reactive programs. In: Brinksma, E. (ed.) TACAS 1997. LNCS, vol. 1217, pp. 321–333. Springer, Heidelberg (1997). doi:10.1007/BFb0035397

    Chapter  Google Scholar 

  6. Botinčan, M., Babić, D., Sigma*: Symbolic learning of input-output specifications. In: POPL, pp. 443–456 (2013)

    Google Scholar 

  7. Cook, B., Kroening, D., Rümmer, P., Wintersteiger, C.M.: Ranking function synthesis for bit-vector relations. FMSD 43(1), 93–120 (2013)

    MATH  Google Scholar 

  8. Ehrenfeucht, A., Haussler, D., Kearns, M., Valiant, L.: A general lower bound on the number of examples needed for learning. Inf. Comput. 82(3), 247–261 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  9. Elizalde, F., Sucar, E., Noguez, J., Reyes, A.: Generating explanations based on Markov decision processes. In: Aguirre, A.H., Borja, R.M., Garciá, C.A.R. (eds.) MICAI 2009. LNCS (LNAI), vol. 5845, pp. 51–62. Springer, Heidelberg (2009). doi:10.1007/978-3-642-05258-3_5

    Chapter  Google Scholar 

  10. Feng, C., Muggleton, S.: Towards inductive generalisation in higher order logic. In: 9th International Workshop on Machine learning, pp. 154–162 D (2014)

    Google Scholar 

  11. Godefroid, P., Taly, A.: Automated synthesis of symbolic instruction encodings from i/o samples. SIGPLAN Not. 47(6), 441–452 (2012)

    Article  Google Scholar 

  12. Goldsmith, J., Sloan, R.H., Szörényi, B., Turán, G.: Theory revision with queries: Horn, read-once, and parity formulas. Artif. Intell. 156(2), 139–176 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  13. Gurfinkel, A., Belov, A., Marques-Silva, J.: Synthesizing safe bit-precise invariants. In: Ábrahám, E., Havelund, K. (eds.) TACAS 2014. LNCS, vol. 8413, pp. 93–108. Springer, Heidelberg (2014). doi:10.1007/978-3-642-54862-8_7

    Chapter  Google Scholar 

  14. Harbers, M., Meyer, J.-J., van den Bosch, K.: Explaining simulations through self explaining agents. J. Artif. Soc. Soc. Simul. 13, 10 (2010)

    Google Scholar 

  15. Hellerstein, L., Servedio, R.A.: On PAC learning algorithms for rich boolean function classes. Theoret. Comput. Sci. 384(1), 66–76 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  16. Jha, S., Seshia, S.A.: A theory of formal synthesis via inductive learning. Acta Informatica, pp. 1–34 (2017)

    Google Scholar 

  17. Jha, S., A. Seshia, and A. Tiwari. Synthesis of optimal switching logic for hybrid systems. In: EMSOFT, pp. 107–116. ACM (2011)

    Google Scholar 

  18. Kearns, M., Li, M., Valiant, L.: Learning boolean formulas. J. ACM 41(6), 1298–1328 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  19. Kearns, M., Valiant, L.: Cryptographic limitations on learning boolean formulae and finite automata. Journal of the ACM (JACM) 41(1), 67–95 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  20. LaValle, S.M.: Planning Algorithms. Cambridge University Press, Cambridge (2006)

    Book  MATH  Google Scholar 

  21. Lecun, Y., Cortes, C.: The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/

  22. Lee, J., Moray, N.: Trust, control strategies and allocation of function in human-machine systems. Ergonomics 35(10), 1243–1270 (1992)

    Article  Google Scholar 

  23. Mansour, Y.: Learning boolean functions via the fourier transform. In: Theoretical Advances in Neural Computation and Learning, pp. 391–424 (1994)

    Google Scholar 

  24. Nau, D., Ghallab, M., Traverso, P.: Automated Planning: Theory and Practice. Morgan Kaufmann Publishers Inc., San Francisco (2004)

    MATH  Google Scholar 

  25. Pitt, L., Valiant, L.G.: Computational limitations on learning from examples. J. ACM (JACM) 35(4), 965–984 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  26. Raman, V.: Reactive switching protocols for multi-robot high-level tasks. In: IEEE/RSJ, pp. 336–341 (2014)

    Google Scholar 

  27. Raman, V., Lignos, C., Finucane, C., Lee, K.C.T., Marcus, M.P., Kress-Gazit, H.: Sorry Dave, I’m afraid i can’t do that: explaining unachievable robot tasks using natural language. In: Robotics: Science and Systems (2013)

    Google Scholar 

  28. Reynolds, A., Deters, M., Kuncak, V., Tinelli, C., Barrett, C.: Counterexample-guided quantifier instantiation for synthesis in SMT. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015. LNCS, vol. 9207, pp. 198–216. Springer, Cham (2015). doi:10.1007/978-3-319-21668-3_12

    Chapter  Google Scholar 

  29. Ribeiro, M.T., Singh, S., Guestrin, C.: Why Should I Trust You?: Explaining the predictions of any classifier. In: KDD, pp. 1135–1144 (2016)

    Google Scholar 

  30. Russell, J., Cohn, R.: OODA Loop. Book on Demand, Norderstedt (2012)

    Google Scholar 

  31. Sankaranarayanan, S.: Automatic invariant generation for hybrid systems using ideal fixed points. In: HSCC, pp. 221–230 (2010)

    Google Scholar 

  32. Sankaranarayanan, S., Miller, C., Raghunathan, R., Ravanbakhsh, H., Fainekos, G.: A model-based approach to synthesizing insulin infusion pump usage parameters for diabetic patients. In: Annual Allerton Conference on Communication, Control, and Computing, pp. 1610–1617, October 2012

    Google Scholar 

  33. Sankaranarayanan, S., Sipma, H.B., Manna, Z.: Constructing invariants for hybrid systems. FMSD 32(1), 25–55 (2008)

    MATH  Google Scholar 

  34. Štrumbelj, E., Kononenko, I.: Explaining prediction models and individual predictions with feature contributions. KIS 41(3), 647–665 (2014)

    Google Scholar 

  35. Urban, C., Gurfinkel, A., Kahsai, T.: Synthesizing ranking functions from bits and pieces. In: Chechik, M., Raskin, J.-F. (eds.) TACAS 2016. LNCS, vol. 9636, pp. 54–70. Springer, Heidelberg (2016). doi:10.1007/978-3-662-49674-9_4

    Chapter  Google Scholar 

  36. Yuan, C., Lim, H., Lu, T.-C.: Most relevant explanation in bayesian networks. J. Artif. Intell. Res. (JAIR) 42, 309–352 (2011)

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. United Technologies Research Center, Berkeley, USA

    Susmit Jha, Vasumathi Raman, Alessandro Pinto, Tuhin Sahai & Michael Francis

Authors
  1. Susmit Jha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vasumathi Raman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alessandro Pinto
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tuhin Sahai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michael Francis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Susmit Jha .

Editor information

Editors and Affiliations

  1. Stanford University, Palo Alto, California, USA

    Clark Barrett

  2. NASA Ames Research Center, Moffett Field, California, USA

    Misty Davies

  3. NASA Ames Research Center, Moffett Field, California, USA

    Temesghen Kahsai

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Jha, S., Raman, V., Pinto, A., Sahai, T., Francis, M. (2017). On Learning Sparse Boolean Formulae for Explaining AI Decisions. In: Barrett, C., Davies, M., Kahsai, T. (eds) NASA Formal Methods. NFM 2017. Lecture Notes in Computer Science(), vol 10227. Springer, Cham. https://doi.org/10.1007/978-3-319-57288-8_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-57288-8_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-57287-1

  • Online ISBN: 978-3-319-57288-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation