Abstract
Differential privacy is among the most prominent techniques for preserving privacy of sensitive data, oweing to its robust mathematical guarantees and general applicability to a vast array of computations on data, including statistical analysis and machine learning. Previous work demonstrated that concrete implementations of differential privacy mechanisms are vulnerable to statistical attacks. This vulnerability is caused by the approximation of real values to floating point numbers. This paper presents a practical solution to the finite-precision floating point vulnerability, where the inverse transform sampling of the Laplace distribution can itself be inverted, thus enabling an attack where the original value can be retrieved with non-negligible advantage.
The proposed solution has the advantages of being (i) mathematically sound, (ii) generalisable to any infinitely divisible probability distribution, and (iii) of simple implementation in modern architectures. Finally, the solution has been designed to make side channel attack infeasible, because of inherently exponential, in the size of the domain, brute force attacks.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
ANSI/IEEE Std 754–2019 http://754r.ucbtest.org.
- 2.
Since \(\cos (\theta ) + \sin (\theta ) = \sqrt{2} \cos \left( \theta - \frac{\pi }{4}\right) \).
- 3.
We need only use n samples for sampling procedures that do not share uniform variates between executions, for example the normalvariate method in Python’s random library, which uses the Kinderman-Monahan sampling procedure [19].
- 4.
References
Ahrens, J.H., Dieter, U.: Computer methods for sampling from the exponential and normal distributions. Commun. ACM 15(10), 873–882 (1972)
Ahrens, J.H., Dieter, U.: Efficient table-free sampling methods for the exponential, Cauchy, and normal distributions. Commun. ACM 31(11), 1330–1337 (1988)
Bell, J.R.: Algorithm 334: normal random deviates. Commun. ACM 11(7), 498 (1968)
Box, G.E.P., Muller, M.E.: A note on the generation of random normal deviates. Ann. Math. Stat. 29(2), 610–611 (1958)
Brent, R.P.: Fast normal random number generators on vector processors. Technical report TR-CS-93-04, Department of Computer Science, The Australian National University, Canberra, 0200 ACT, Australia (1993)
Canonne, C.L., Kamath, G., Steinke, T.: The discrete Gaussian for differential privacy. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 15676–15688. Curran Associates, Inc. (2020). https://proceedings.neurips.cc/paper/2020/file/b53b3a3d6ab90ce0268229151c9bde11-Paper.pdf
Cheu, A., Smith, A., Ullman, J.: Manipulation attacks in local differential privacy. arXiv preprint arXiv:1909.09630 (2019)
Devroye, L.: Non-uniform Random Variate Generation. Springer, New York (1986). https://doi.org/10.1007/978-1-4613-8643-8
Dodis, Y., López-Alt, A., Mironov, I., Vadhan, S.: Differential privacy with imperfect randomness. In: Safavi-Naini, R., Canetti, R. (eds.) CRYPTO 2012. LNCS, vol. 7417, pp. 497–516. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32009-5_29
Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman and Company, New York (1979)
Garfinkel, S.L., Leclerc, P.: Randomness concerns when deploying differential privacy. In: Proceedings of the 19th Workshop on Privacy in the Electronic Society. WPES 2020, pp. 73–86. Association for Computing Machinery, New York (2020)
Gazeau, I., Miller, D., Palamidessi, C.: Preserving differential privacy under finite-precision semantics. Theoret. Comput. Sci. 655, 92–108 (2016). Quantitative Aspects of Programming Languages and Systems (2013–14)
Goldberg, D.: What every computer scientist should know about floating-point arithmetic. ACM Comput. Surv. (CSUR) 23(1), 5–48 (1991)
Google Differential Privacy Team: Secure noise generation. github.com/google 1(1), 1–14 (2020)
Haeberlen, A., Pierce, B.C., Narayan, A.: Differential privacy under fire. In: USENIX Security Symposium, vol. 33 (2011)
Holohan, N., Braghin, S., Mac Aonghusa, P., Levacher, K.: Diffprivlib: the IBM differential privacy library. arXiv e-prints 1907.02444 [cs.CR], July 2019
Ilvento, C.: Implementing the exponential mechanism with base-2 differential privacy. In: Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, pp. 717–742 (2020)
Karney, C.F.F.: Sampling exactly from the normal distribution. ACM Trans. Math. Softw. 42(1), 1–14 (2016)
Kinderman, A.J., Monahan, J.F.: Computer generation of random variables using the ratio of uniform deviates. ACM Trans. Math. Softw. 3(3), 257–260 (1977)
Kotz, S., Kozubowski, T., Podgorski, K.: The Laplace Distribution and Generalizations: A Revisit with Applications to Communications, Economics, Engineering, and Finance. Springer, Heidelberg (2012)
McSherry, F., Talwar, K.: Mechanism design via differential privacy. In: 2007 48th Annual IEEE Symposium on Foundations of Computer Science. FOCS 2007, pp. 94–103. IEEE (2007)
Mironov, I.: On significance of the least significant bits for differential privacy. In: Proceedings of the 2012 ACM Conference on Computer and Communications Security. CCS 2012, pp. 650–661. Association for Computing Machinery, New York (2012)
Mironov, I., Pandey, O., Reingold, O., Vadhan, S.: Computational differential privacy. In: Halevi, S. (ed.) CRYPTO 2009. LNCS, vol. 5677, pp. 126–142. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03356-8_8
Thomas, D.B., Luk, W., Leong, P.H., Villasenor, J.D.: Gaussian random number generators. ACM Comput. Surv. 39(4), 11-es (2007)
van der Walt, S., Colbert, S.C., Varoquaux, G.: The NumPy array: a structure for efficient numerical computation. Comput. Sci. Eng. 13(2), 22–30 (2011)
Acknowledgments
The authors wish to thank David Malone (Hamilton Institute, Maynooth University) for useful discussions at the beginning of this project.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendices
Appendix
A Probability Density Functions
The following probability distributions are referenced in Sect. 4.
1.1 A.1 Uniform Distribution
The uniform distribution on the interval \([a, b] \subset \mathbb {R}\), \(-\infty< a< b < \infty \), is given by the Probability Density Function (PDF)
We make use of the uniform distribution \(\mathcal {U}(0, 1)\) on the unit interval [0, 1].
1.2 A.2 Gaussian Distribution
The Gaussian distribution with mean \(\mu \) and variance \(\sigma ^2\) is given by the PDF
We refer to the case when \(\mu = 0\) and \(\sigma = 1\) as the standard Gaussian distribution. If \(N \sim \mathcal {N}(0, 1)\), then \(\sigma N + \mu \sim \mathcal {N}(\mu , \sigma )\).
1.3 A.3 Laplace Distribution
The Laplace distribution with mean \(\mu \) and variance \(2b^2\) is given by the PDF
We refer to the case when \(\mu = 0\) and \(b = 1\) as the standard Laplace distribution. If \(L \sim \mathrm {Lap}(0, 1)\), then \(b L + \mu \sim \mathrm {Lap}(\mu , b)\).
1.4 A.4 Exponential Distribution
The exponential distribution with mean \(\frac{1}{\lambda }\) and variance \(\frac{1}{\lambda ^2}\) is given by the PDF
We refer to the case when \(\lambda = 1\) as the standard exponential distribution. If \(E \sim \mathrm {Exp}(1)\), then \(\frac{E}{\lambda } \sim \mathrm {Exp}(\lambda )\).
1.5 A.5 Gamma Distribution
The gamma distribution with mean \(k \theta \) and variance \(k \theta ^2\) is given by the PDF
If \(G \sim \varGamma (k, \theta )\), then \(c G \sim \varGamma (k, c \theta )\) for any \(c > 0\).
1.6 A.6 Chi-Squared Distribution
The chi-squared distribution with \(k \in \mathbb {N}\) degrees of freedom is given by the PDF
B Code Samples
The following code samples were used in estimating execution time for different implementations. This code was run using Python 3.8.6.
1.1 B.1 Naïve Sampling
The naïve standard Laplace sampling given by (2) was implemented using:

1.2 B.2 Theorem 1 Sampling
The implementation of Theorem 1 was given by:

1.3 B.3 Sampling with math and random
We combine the Gaussian and Laplace sampling procedures from (7) and (2) to generate standard Laplace samples from 8n uniform variates using the math and random libraries as follows:

1.4 B.4 Sampling with Numpy
Finally, we present an implementation of the same procedure using the popular Numpy package, leveraging its C-based code for faster computations with larger n:

Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Holohan, N., Braghin, S. (2021). Secure Random Sampling in Differential Privacy. In: Bertino, E., Shulman, H., Waidner, M. (eds) Computer Security – ESORICS 2021. ESORICS 2021. Lecture Notes in Computer Science(), vol 12973. Springer, Cham. https://doi.org/10.1007/978-3-030-88428-4_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-88428-4_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88427-7
Online ISBN: 978-3-030-88428-4
eBook Packages: Computer ScienceComputer Science (R0)