Abstract
Differential privacy has become a golden standard for designing privacy-preserving randomized algorithms. However, such algorithms are subtle to design, as many of them are found to have incorrect privacy claim. To help identify this problem, one approach is designing disprovers to search for counterexamples that demonstrate high violation of claimed privacy level. In this paper, we present DP-Opt(mizer), a disprover that tries to search for counterexamples whose lower bounds on differential privacy exceed the claimed level of privacy guaranteed by the algorithm. We leverage the insights of counterexample construction proposed by the latest work, meanwhile resolve their limitations. We transform the search task into an improved optimization objective which takes into account the empirical error, then solve it with various off-the-shelf optimizers. An evaluation on a variety of both correct and incorrect algorithms illustrates that DP-Opt almost always produces stronger guarantees than the latest work up to a factor of 9.42, with runtime reduced by an average of \(19.2\%\).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Available at https://github.com/barryZZJ/dp-opt.
References
Albarghouthi, A., Hsu, J.: Synthesizing coupling proofs of differential privacy. Proc. ACM Program. Lang. 2(POPL), 1–30 (2017)
Askin, Ö., Kutta, T., Dette, H.: Statistical quantification of differential privacy: a local approach. arXiv preprint arXiv:2108.09528 (2021)
Barthe, G., Chadha, R., Jagannath, V., Sistla, A.P., Viswanathan, M.: Deciding differential privacy for programs with finite inputs and outputs. In: Proceedings of the 35th Annual ACM/IEEE Symposium on Logic in Computer Science, pp. 141–154 (2020)
Barthe, G., Gaboardi, M., Grégoire, B., Hsu, J., Strub, P.Y.: Proving differential privacy via probabilistic couplings. In: Proceedings of the 31st Annual ACM/IEEE Symposium on Logic in Computer Science, pp. 749–758 (2016)
Barthe, G., Köpf, B., Olmedo, F., Zanella Beguelin, S.: Probabilistic relational reasoning for differential privacy. In: Proceedings of the 39th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pp. 97–110 (2012)
Bichsel, B., Gehr, T., Drachsler-Cohen, D., Tsankov, P., Vechev, M.: DP-Finder: finding differential privacy violations by sampling and optimization. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp. 508–524 (2018)
Bichsel, B., Steffen, S., Bogunovic, I., Vechev, M.: DP-Sniper: black-box discovery of differential privacy violations using classifiers. In: 2021 IEEE Symposium on Security and Privacy (SP), pp. 391–409 (2021)
Chen, Y., Machanavajjhala, A.: On the privacy properties of variants on the sparse vector technique. arXiv preprint arXiv:1508.07306 (2015)
Clopper, C.J., Pearson, E.S.: The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 26(4), 404–413 (1934)
Ding, Z., Wang, Y., Wang, G., Zhang, D., Kifer, D.: Detecting violations of differential privacy. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp. 475–489 (2018)
Dwork, C., McSherry, F., Nissim, K., Smith, A.: Calibrating noise to sensitivity in private data analysis. In: Halevi, S., Rabin, T. (eds.) TCC 2006. LNCS, vol. 3876, pp. 265–284. Springer, Heidelberg (2006). https://doi.org/10.1007/11681878_14
Erlingsson, Ú., Pihur, V., Korolova, A.: RAPPOR: randomized aggregatable privacy-preserving ordinal response. In: Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, pp. 1054–1067 (2014)
Li, F., Li, H., Niu, B., Chen, J.: Privacy computing: concept, computing framework, and future development trends. Engineering 5(6), 1179–1192 (2019)
Liu, D., Wang, B.Y., Zhang, L.: Verifying pufferfish privacy in hidden Markov models. In: International Conference on Verification, Model Checking, and Abstract Interpretation, pp. 174–196 (2022)
Lyu, M., Su, D., Li, N.: Understanding the sparse vector technique for differential privacy. arXiv preprint arXiv:1603.01699 (2016)
Thulin, M.: The cost of using exact confidence intervals for a binomial proportion. Electron. J. Stat. 8(1), 817–840 (2014)
Wang, Y., Ding, Z., Kifer, D., Zhang, D.: CheckDP: an automated and integrated approach for proving differential privacy or finding precise counterexamples. In: Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, pp. 919–938 (2020)
Wang, Y., Ding, Z., Wang, G., Kifer, D., Zhang, D.: Proving differential privacy with shadow execution. In: Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, pp. 655–669 (2019)
Wilson, R.J., Zhang, C.Y., Lam, W., Desfontaines, D., Simmons-Marengo, D., Gipson, B.: Differentially private SQL with bounded user contribution. arXiv preprint arXiv:1909.01917 (2019)
Zhang, D., Kifer, D.: LightDP: towards automating differential privacy proofs. In: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages, pp. 888–901 (2017)
Acknowledgements
This work is supported by the National Key R &D Program of China (2021YFB3100300), the National Natural Science Foundation of China (61872441, 61932015), and the Youth Innovation Promotion Association, Chinese Academy of Sciences (2019160, 2021154).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Niu, B., Zhou, Z., Chen, Y., Cao, J., Li, F. (2022). DP-Opt: Identify High Differential Privacy Violation by Optimization. In: Wang, L., Segal, M., Chen, J., Qiu, T. (eds) Wireless Algorithms, Systems, and Applications. WASA 2022. Lecture Notes in Computer Science, vol 13472. Springer, Cham. https://doi.org/10.1007/978-3-031-19214-2_34
Download citation
DOI: https://doi.org/10.1007/978-3-031-19214-2_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-19213-5
Online ISBN: 978-3-031-19214-2
eBook Packages: Computer ScienceComputer Science (R0)