Skip to main content
SpringerLink
Log in

Proving that Programs Are Differentially Private

  • Conference paper
  • First Online:
Book cover Programming Languages and Systems (APLAS 2019)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 11893))

Included in the following conference series:

Abstract

We extend recent work in Quantitative Information Flow (QIF) to provide tools for the analysis of programs that aim to implement differentially private mechanisms. We demonstrate how differential privacy can be expressed using loss functions, and how to use this idea in conjunction with a QIF-enabled program semantics to verify differentially private guarantees. Finally we describe how to use this approach experimentally using Kuifje, a recently developed tool for analysing information-flow properties of programs.

This research was supported by the Australian Research Council Grant DP140101119.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Alvim et al. explained this as a gain to benefit the adversary; couching the interpretation as losses is mathematically equivalent but the formulation as losses turns out to be more convenient for reasoning about programs [17].

  2. 2.

    Stochastic means that the rows sum to 1.

  3. 3.

    We use \(p_y\) and \(\pi |^y\) for typographical convenience. Notation suited for calculation would need to incorporate C and \(\pi \).

  4. 4.

    The revised definition would change the second line of Definition 5 to be

    $$ dp_{\epsilon }(w, x) ~~~=~~~ e^{\epsilon \times d({\mathop {w}\limits ^{\leftarrow }},{\mathop {w}\limits ^{\rightarrow }})}~,~~~ \textit{if}~~ w \ne \star ~~~\wedge ~~~ {\mathop {w}\limits ^{\rightarrow }}=x. $$
  5. 5.

    We use logs base e throughout.

  6. 6.

    This is essentially a Markov update of the state.

  7. 7.

    Strictly speaking the state is determined by the values of all the program variables. However the only secret that we worry about for this example is the value of resp. These details can all be handled by adjusting the definition of \(dp_{\epsilon }\).

References

  1. Alvim, M.S., Andrés, M.E., Chatzikokolakis, K., Degano, P., Palamidessi, C.: On the information leakage of differentially-private mechanisms. J. Comput. Secur. 23(4), 427–469 (2015)

    Article  Google Scholar 

  2. Alvim, M.S., Andrés, M.E., Chatzikokolakis, K., Palamidessi, C.: On the relation between differential privacy and quantitative information flow. In: Aceto, L., Henzinger, M., Sgall, J. (eds.) ICALP 2011. LNCS, vol. 6756, pp. 60–76. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22012-8_4

    Chapter  MATH  Google Scholar 

  3. Alvim, M.S., Chatzikokolakis, K., Degano, P., Palamidessi, C.: Differential privacy versus quantitative information flow. CoRR, abs/1012.4250 (2010)

    Google Scholar 

  4. Alvim, M.S., Chatzikokolakis, K., McIver, A., Morgan, C., Palamidessi, C., Smith, G.: Additive and multiplicative notions of leakage, and their capacities. In: IEEE 27th Computer Security Foundations Symposium, CSF 2014, Vienna, Austria, 19–22 July 2014, pp. 308–322. IEEE (2014)

    Google Scholar 

  5. Alvim, M.S., Chatzikokolakis, K., Palamidessi, C., Smith, G.: Measuring information leakage using generalized gain functions. In: Proceedings 25th IEEE Computer Security Foundations Symposium, CSF 2012, pp. 265–279, June 2012

    Google Scholar 

  6. Alvim, M.S., Scedrov, A., Schneider, F.B.: When not all bits are equal: worth-based information flow. In: Abadi, M., Kremer, S. (eds.) POST 2014. LNCS, vol. 8414, pp. 120–139. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-54792-8_7

    Chapter  Google Scholar 

  7. Barthe, G., Gaboardi, M., Arias, E.J.G., Hsu, J., Kunz, C., Strub, P.-Y.: Proving differential privacy in Hoare logic. In: IEEE 27th Computer Security Foundations Symposium, CSF 2014, Vienna, Austria, 19–22 July 2014, pp. 411–424 (2014)

    Google Scholar 

  8. Barthe, G., Gaboardi, M., Grégoire, B., Hsu, J., Strub, P.-Y.: Proving differential privacy via probabilistic couplings. In: Proceedings of the 31st Annual ACM/IEEE Symposium on Logic in Computer Science, pp. 749–758 (2016)

    Google Scholar 

  9. Barthe, G., Gaboardi, M., Hsu, J., Pierce, B.: Programming language techniques for differential privacy. ACM SIGLOG News 3(1), 34–53 (2016)

    Google Scholar 

  10. Bognar, M., Schrijvers, T.. Kuifje: a prototype for a quantitative information flow aware programming language. https://github.com/martonbognar/kuifje

  11. Chatzikokolakis, K., Andrés, M.E., Bordenabe, N.E., Palamidessi, C.: Broadening the scope of differential privacy using metrics. In: De Cristofaro, E., Wright, M. (eds.) PETS 2013. LNCS, vol. 7981, pp. 82–102. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39077-7_5

    Chapter  Google Scholar 

  12. Dwork, C.: Differential privacy. In: Bugliesi, M., Preneel, B., Sassone, V., Wegener, I. (eds.) ICALP 2006. LNCS, vol. 4052, pp. 1–12. Springer, Heidelberg (2006). https://doi.org/10.1007/11787006_1

    Chapter  Google Scholar 

  13. Ebadi, H., Sands, D.: Featherweight PINQ. J. Priv. Secur. 7(2) (2017)

    Google Scholar 

  14. Giry, M.: A categorical approach to probability theory. In: Banaschewski, B. (ed.) Categorical Aspects of Topology and Analysis. LNM, vol. 915, pp. 68–85. Springer, Heidelberg (1982). https://doi.org/10.1007/BFb0092872

    Chapter  Google Scholar 

  15. Gibbons, C.M.J., Mciver, A., Schrijvers, T.: Quantitative information flow with monads in haskell. In: Foundations of Probabilistic Programming. CUP (2019, to appear)

    Google Scholar 

  16. McIver, A., Meinicke, L., Morgan, C.: Compositional closure for Bayes risk in probabilistic noninterference. In: Abramsky, S., Gavoille, C., Kirchner, C., Meyer auf der Heide, F., Spirakis, P.G. (eds.) ICALP 2010. LNCS, vol. 6199, pp. 223–235. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14162-1_19

    Chapter  Google Scholar 

  17. McIver, A., Morgan, C., Rabehaja, T.: Abstract hidden Markov models: a monadic account of quantitative information flow. In: Proceedings LiCS 2015 (2015)

    Google Scholar 

  18. Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27(379–423), 623–656 (1948)

    Article  MathSciNet  Google Scholar 

  19. Smith, G.: On the foundations of quantitative information flow. In: de Alfaro, L. (ed.) FoSSaCS 2009. LNCS, vol. 5504, pp. 288–302. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-00596-1_21

    Chapter  Google Scholar 

  20. Wang, Y., Ding, Z., Wang, G., Kifer, D., Zhang, D.: Proving differential privacy with shadow execution. In: Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2019, New York, NY, USA, pp. 655–669. ACM (2019)

    Google Scholar 

  21. Warner, S.L.: Randomized response: a survey technique for eliminating evasive answer Bias. J. Am. Stat. Assoc. 60, 63–69 (1965)

    Article  Google Scholar 

  22. Zhang, D., Kifer, D.: Lightdp: towards automating differential privacy proofs. In: Proceedings of Principles of Programming Languages, pp. 1–17 (2017)

    Google Scholar 

Download references

Acknowledgements

I thank Tom Schrijvers for having the idea of embedding these ideas in Haskell, based on Carroll Morgan’s talk at IFIP WG2.1 in Vermont, and for carrying it out to produce the tool Kuifje. Together with Jeremy Gibbons all four of us wrote the first paper devoted to it [15]. (It was Jeremy who suggested the name “Kuifje”, the Dutch name for TinTin — and hence his “QIF”.)

Author information

Authors and Affiliations

  1. Department Computing, Macquarie University, Sydney, Australia

    Annabelle McIver

  2. University of New South Wales and Data61, Sydney, Australia

    Carroll Morgan

Authors
  1. Annabelle McIver
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carroll Morgan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Annabelle McIver .

Editor information

Editors and Affiliations

  1. University of Kaiserslautern, Kaiserslautern, Germany

    Anthony Widjaja Lin

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

McIver, A., Morgan, C. (2019). Proving that Programs Are Differentially Private. In: Lin, A. (eds) Programming Languages and Systems. APLAS 2019. Lecture Notes in Computer Science(), vol 11893. Springer, Cham. https://doi.org/10.1007/978-3-030-34175-6_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-34175-6_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-34174-9

  • Online ISBN: 978-3-030-34175-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation