Skip to main content
Log in

Exact MCMC with differentially private moves

Revisiting the penalty algorithm in a data privacy framework

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

We view the penalty algorithm of Ceperley and Dewing (J Chem Phys 110(20):9812–9820, 1999), a Markov chain Monte Carlo algorithm for Bayesian inference, in the context of data privacy. Specifically, we studied differential privacy of the penalty algorithm and advocate its use for data privacy. The algorithm can be made differentially private while remaining exact in the sense that its target distribution is the true posterior distribution conditioned on the private data. We also show that in a model with independent observations the algorithm has desirable convergence and privacy properties that scale with data size. Two special cases are also investigated and privacy-preserving schemes are proposed for those cases: (i) Data are distributed among several users who are interested in the inference of a common parameter while preserving their data privacy. (ii) The data likelihood belongs to an exponential family. The results of our numerical experiments on the Beta-Bernoulli and the logistic regression models agree with the theoretical results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. For a sequence of real-valued random variables \(\{ X_{n} \}_{n \ge 1}\) with joint distribution P, and a sequence of real numbers \(\{ a_{n} \}_{n \ge 1}\), we say \(X_{n}\) is \({\mathcal {O}}_{P}(a_{n})\) if for any \(\epsilon > 0\) there exists an \(M > 0\) such that \({\mathbb {P}}(|X_{n}/a_{n}| > M) < \epsilon \) for all \(n \ge 1\).

  2. In fact, since ESS is an analytical quantity associated to an MCMC chain and it is approximated from the samples generated by the chain, we let the algorithms run for longer for better accuracy of our approximate ESS computations, and then perform the appropriate normalisation.

References

  • Abadi, M., Chu, A., Goodfellow, I., McMahan, H.B., Mironov, I., Talwar, K., Zhang, L.: Deep learning with differential privacy. In: Conference on Computer and Communications Security, pp. 308–318. ACM SIGSAC (2016)

  • Andrieu, C., Doucet, A., Yıldırım, S., Chopin, N.: On the utility of Metropolis–Hastings with asymmetric acceptance ratio. Technical report (2018). arXiv:1803.09527

  • Atchadé, Y.F., Perron, F.: On the geometric ergodicity of Metropolis–Hastings algorithms. Statistics 41(1), 77–84 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  • Bierkens, J.: Non-reversible Metropolis–Hastings. Stat. Comput. 26(6), 1213–1228 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  • Bun, M., Steinke, T.: Concentrated differential privacy: simplifications, extensions, and lower bounds. In: Proceedings, Part I, of the 14th International Conference on Theory of Cryptography, vol. 9985, pp. 635–658. Springer, New York (2016)

  • Ceperley, D.M., Dewing, M.: The penalty method for random walks with uncertain energies. J. Chem. Phys. 110(20), 9812–9820 (1999)

    Article  Google Scholar 

  • Dwork, C., McSherry, F., Nissim, K., Smith, A.: Calibrating noise to sensitivity in private data analysis. In: Halevi, S., Rabin, T. (eds.) Theory of Cryptography. TCC 2006. Lecture Notes in Computer Science, vol. 3876, pp. 265–284. Springer, Berlin, Heidelberg (2006)

  • Dwork, C., Roth, A.: The algorithmic foundations of differential privacy. Theor. Comput. Sci. 9(3–4), 211–407 (2013)

    MathSciNet  MATH  Google Scholar 

  • Dwork, C., Rothblum, G.N.: Concentrated differential privacy. Technical report (2016). arXiv:1603.01887v2

  • Dwork, C., Rothblum, G.N., Vadhan, S.: Boosting and differential privacy. In: 2010 51st Annual IEEE Symposium on Foundations of Computer Science (FOCS), pp. 51–60 (2010)

  • Foulds, J., Geumlek, J., an Kamalika Chaudhuri, M.W.: On the theory and practice of privacy-preserving Bayesian data analysis. Technical report (2016). arxiv:1603.07294

  • Geumlek, J., Song, S., Chaudhuri, K.: Renyi differential privacy mechanisms for posterior sampling. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 30, pp. 5289–5298. Curran Associates Inc, Red Hook (2017)

    Google Scholar 

  • Gustafson, P.: A guided walk Metropolis algorithm. Stat. Comput. 8(4), 357–364 (1998)

    Article  Google Scholar 

  • Hastings, W.K.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 52(1), 97–109 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  • Heikkilä, M., Lagerspetz, E., Kaski, S., Shimizu, K., Tarkoma, S., Honkela, A.: Differentially private Bayesian learning on distributed data. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 30, pp. 3226–3235. Curran Associates Inc, Red Hook (2017)

    Google Scholar 

  • Minami, K., Arai, H., Sato, I., Nakagawa, H.: Differential privacy without sensitivity. In: Lee, D.D., Sugiyama, M., Luxburg, U.V., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 29, pp. 956–964. Curran Associates Inc, Red Hook (2016)

    Google Scholar 

  • Mironov, I.: Rényi differential privacy. In: 2017 IEEE 30th Computer Security Foundations Symposium (CSF), pp. 263–275 (2017)

  • Nicholls, G., Fox, C., Watt, A.: Coupled MCMC with a randomised acceptance probability. Technical report (2012). arXiv:1205.6857

  • Park, M., Foulds, J.R., Chaudhuri, K., Welling, M.: Variational Bayes in private settings (VIPS). Technical report (2016). arXiv:1611.00340v3

  • Robert, C.P., Casella, G.: Monte Carlo Statistical Methods, 2nd edn. Springer, New York (2004)

    Book  MATH  Google Scholar 

  • Roberts, G., Tweedie, R.: Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms. Biometrika 83, 95–110 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  • Wang, Y.-X., Fienberg, S., Smola, A.: Privacy for free: posterior sampling and stochastic gradient Monte Carlo. In: Blei, D., Bach, F. (eds), Proceedings of the 32nd International Conference on Machine Learning (ICML-15), Workshop and Conference Proceedings, pp. 2493–2502. JMLR (2015)

  • Welling, M., Teh, Y.W.: Bayesian learning via stochastic gradient langevin dynamics. In: Getoor, L., Scheffer, T. (eds), Proceedings of 28th International Conference on Machine Learning (ICML 2011), ICML ’11, pp. 681–688. ACM (2011)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sinan Yıldırım.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Proofs for Section 2.1

Proof of Proposition 1

Dropping \(\theta , \theta '\) from the notation, we will show that the derivative of \(\alpha _{\sigma }\) w.r.t. \(\sigma \) is strictly negative.

$$\begin{aligned} \alpha _{\sigma }&= \int _{-\infty }^{\infty } \min \{ 1, \exp ( r + \sigma x - \sigma ^{2}/2 ) \} \phi (x)\mathrm{d}x \\&= \int _{\frac{\sigma }{2} - \frac{r}{\sigma }}^{\infty } \phi (x)\mathrm{d}x + \int _{-\infty }^{\frac{\sigma }{2} - \frac{r}{\sigma }} \exp (r + \sigma x - \sigma ^{2}/2) \phi (x)\mathrm{d}x \\&= \int _{\frac{\sigma }{2} - \frac{r}{\sigma }}^{\infty } \phi (x)\mathrm{d}x + \exp (r) \int _{-\infty }^{-\frac{\sigma }{2} - \frac{r}{\sigma }} \phi (x)\mathrm{d}x \end{aligned}$$

where \(\phi \) is the probability density of the standard normal distribution. Now,

$$\begin{aligned} \frac{\mathrm{d}\alpha _{\sigma }}{\mathrm{d}\sigma }&= -\phi \left( \frac{\sigma }{2} - \frac{r}{\sigma }\right) \left( \frac{1}{2} + \frac{r}{\sigma }^{2}\right) \\&\quad + e^{r} \phi \left( \frac{\sigma }{2} + \frac{r}{\sigma }\right) \left( -\frac{1}{2} + \frac{r}{\sigma }^{2}\right) \\&= -\phi \left( \frac{\sigma }{2} - \frac{r}{\sigma }\right) \left( \frac{1}{2} + \frac{r}{\sigma }^{2}\right) \\&\quad + \phi \left( \frac{\sigma }{2} - \frac{r}{\sigma }\right) \left( -\frac{1}{2} + \frac{r}{\sigma }^{2}\right) \\&= -\phi \left( \frac{\sigma }{2} - \frac{r}{\sigma }\right) \end{aligned}$$

which is negative. \(\square \)

Proof of Proposition 2

We need to show that the expected acceptance probability of the penalty algorithm satisfies

$$\begin{aligned} \alpha _{\sigma }(\theta , \theta ') \ge \kappa \alpha (\theta , \theta ') \end{aligned}$$
(10)

for some \(\kappa > 0\) for all \(\theta , \theta '\). Then we can follow the same steps starting from Equation (6.3) of Nicholls et al. (2012, Appendix A) to conclude that for any \(\theta \in \varTheta \) and measurable set \(E \subseteq \varTheta \) we have \(T_{\sigma }(\theta , E) \ge \kappa T(\theta , E)\) and that since by hypothesis T satisfies a minorisation condition so does \(T_{\sigma }\), which leads to uniform ergodicity. To show (10), let \(\kappa = 1 - \varPhi (B/2)\), where \(\varPhi \) is the cumulative distribution function of \({\mathcal {N}}(0, 1)\), and observe that, for any \(\theta , \theta ' \in \varTheta \), and \(V \sim {\mathcal {N}}(0, 1)\), we have

$$\begin{aligned} {\mathbb {P}} \left( {\hat{\lambda }}_{\sigma }(\theta , \theta ') \ge \lambda (\theta , \theta ') \right)&= {\mathbb {P}} \left( \sigma (\theta , \theta ') V - \sigma ^{2}(\theta , \theta ')/2 \right) \\&={\mathbb {P}} \left( V \ge \sigma (\theta , \theta ')/2 \right) \\&\ge {\mathbb {P}} \left( V \ge B/2 \right) \\&= 1 - \varPhi (B/2) = \kappa . \end{aligned}$$

Therefore, by using a decomposition for the expected value, we have

$$\begin{aligned} \alpha _{\sigma }(\theta , \theta ')&= {\mathbb {E}}\left[ {\hat{\alpha }}_{\sigma }(\theta , \theta ') \right] \\&\ge {\mathbb {E}}\left[ {\hat{\alpha }}_{\sigma }(\theta , \theta ') \left| {\hat{\lambda }}_{\sigma }(\theta , \theta ') \ge \lambda (\theta , \theta ') \right. \right] {\mathbb {P}} \\&\quad \times \left( {\hat{\lambda }}_{\sigma }(\theta , \theta ') \ge \lambda (\theta , \theta ') \right) \\&\ge \alpha (\theta , \theta ') \kappa \end{aligned}$$

where the second inequality holds since \({\hat{\alpha }}_{\sigma }(\theta , \theta ') \ge \alpha (\theta , \theta ')\) in the event \({\hat{\lambda }}_{\sigma }(\theta , \theta ') \ge \lambda (\theta , \theta ')\). \(\square \)

Proof of Lemma 1

We have shown in the proof of Proposition 2 that if \(\sup _{\theta , \theta '} \sigma ^{2}(\theta , \theta ') = B < \infty \) then \(\alpha _{\sigma }(\theta , \theta ') \ge \kappa \alpha (\theta , \theta ')\) with \(\kappa = 1 - \varPhi (B/2)\).

$$\begin{aligned} \rho _{\sigma }(\theta )&= 1 - \int \alpha _{\sigma }(\theta , \theta ') q(\mathrm{d}\theta '|\theta ) \\&\le 1 - \kappa \int \alpha (\theta , \theta ') q(\mathrm{d}\theta '|\theta ) \\&= 1 - \kappa (1 - \rho (\theta )) \\&= (1 - \kappa ) + \kappa \rho (\theta ) \end{aligned}$$

Therefore, \({{\,\mathrm{ess\,sup}\,}}\rho _{\sigma } \le (1 - \kappa ) + \kappa {{\,\mathrm{ess\,sup}\,}}\rho < 1\). \(\square \)

Proof of Proposition 3

The geometric ergodicity of the MH algorithm implies \({{\,\mathrm{ess\,sup}\,}}\rho < 1\) by Roberts and Tweedie (1996, Proposition 5.1.). The boundedness of \(\sigma ^{2}(\theta , \theta ')\) gives \({{\,\mathrm{ess\,sup}\,}}\rho _{\sigma } < 1\) by Lemma 1. Moreover, we observe that \(T_{\sigma }\) is self-adjoint since the penalty algorithm is reversible. These facts can be used to establish the geometric ergodicity by verifying the steps of Atchadé and Perron (2007, Theorem 2.1) for the kernel of the penalty algorithm. \(\square \)

Proofs for Section 3

We cite the definition of zCDP and important results needed for Theorem 1. First we define the zCDP (Bun and Steinke 2016).

Definition 4

(zCDP) Given \(\chi > 0\) and \(\rho > 0\), an algorithm \({\mathcal {A}}\) with domain \({\mathcal {X}}\) and range \({\mathcal {S}}\) is \((\chi , \rho )\)-zCDP if for all \(X, Y \in {\mathcal {X}}\) such that \(h(X, Y) \le 1\), and for all \(\alpha \in [1, \infty )\), we have

$$\begin{aligned} D_{\alpha }({\mathcal {A}}(X) || {\mathcal {A}}(Y)) \le \chi + \rho \alpha \end{aligned}$$

where \(D_{\alpha }\) is the \(\alpha \)-Réyni divergence between the distribution of \({\mathcal {A}}(X)\) and \({\mathcal {A}}(Y)\). We also define \(\rho \)-zCDP to be \((0, \rho )\)-zCDP.

While zCDP regards on all positive moments of a privacy loss variable, a recent related definition of differential privacy, the Réyni differential privacy (Mironov 2017) handles one moment at a time and therefore can be more flexible to use in general. That being said, it has been shown in Bun and Steinke (2016) and mentioned in Park et al. (2016) that zCDP is tight for the Gaussian mechanism, that is why we prefer to use the following result on the zCDP of the Gaussian mechanism to analyse the privacy of our algorithm.

Lemma 2

The Gaussian mechanism for \(f: {\mathcal {X}} \rightarrow {\mathbb {R}}\) with \(\sigma ^{2} > 0\) with is \(((\varDelta _{1}f)^{2}/ 2 \sigma ^{2})\)-zCDP.

The function f relevant to the penalty algorithm is the \(d_{n}(\cdot ; \theta , \theta '): {\mathcal {Y}}^{n} \rightarrow {\mathbb {R}}\)’s, i.e. the difference in log-likelihood functions, when considered as functions of data samples.

Next, we show the crucial relation between zCDP and DP, which allows us to have tighter bounds on the DP of each iteration of the penalty algorithms hence more allowed number of iterations.

Lemma 3

If an algorithm \({\mathcal {A}}\) is \((\chi , \rho )\)-zCDP, then it is \((\epsilon , \delta )\)-DP for all \((\epsilon , \delta )\) jointly satisfying \(\delta > 0\) and \(\epsilon = \chi + \rho + \sqrt{4 \rho \log (1 /\delta )}\).

Our final result on zCDP holds that zCDP admits the following basic composition property.

Lemma 4

Assume \({\mathcal {A}}_{1}\) and \({\mathcal {A}}_{2}\) are independent randomised algorithms that are \(\rho _{1}\)-zCDP and \(\rho _{2}\)-zCDP, respectively, with the same domain \({\mathcal {X}}\) and range \({\mathcal {S}}\). Then the composition \({\mathcal {A}} = ({\mathcal {A}}_{1}, {\mathcal {A}}_{2})\) such that \({\mathcal {A}}(X) = ({\mathcal {A}}_{1}(X), {\mathcal {A}}_{2}(X))\) is \((\rho _{1} + \rho _{2})\)-zCDP.

Lemma 4 can be trivially generalised to \(k > 1\) mechanisms, leading to the corollary below.

Corollary 4

The composition of \(k > 1\)\(\rho \)-zCDP algorithms is a \(k \rho \)-zCDP algorithm.

Now we can proceed to the proof of Theorem 1.

Proof of Theorem 1

Using Lemma 3 with \(\chi = 0\) and solving for \(\rho \), we have that \(\rho _{n}\)-zCDP satisfies \((\epsilon , n^{-\beta })\)-DP, where \(\rho _{n}\) is defined in (3). Next, by Lemma 2, each iteration of the penalty algorithm with \(\sigma _{n}^{2}(\theta , \theta ') = \tau ^{2} n^{2 \alpha } c^{2}(\theta , \theta ')\) is \(\rho _{\text {iter}} = n^{-2 \alpha }/ (2\tau ^{2})\)-zCDP. Using Corollary 4, we can have at most \(\lfloor \rho _{n}/\rho _{\text {iter}} \rfloor = \lfloor 2 \rho _{n} n^{2 \alpha } \tau ^{2} \rfloor \) iterations to achieve \((\epsilon , n^{-\beta })\)-DP. The order of the variance \(\sigma _{n}^{2}(\theta _{n, t}, \theta _{n, t+1}')\) is obvious from Assumption A1 and the choice for \(\sigma _{n}^{2}(\theta , \theta ')\). \(\square \)

Proof of Proposition 4

By a zeroth-order Taylor polynomial approximation, we have for any \(\theta , \theta ' \in \varTheta \)

$$\begin{aligned} |\ell (y, \theta ') - \ell (y, \theta )| \le \sum _{i = 1}^{d_{\theta }} |\theta '(i) - \theta (i)| M \end{aligned}$$

by Assumption A2. Combining this by Assumption A3, we have the desired order to hold \(P_{n, \sigma , \theta _{0}}\)-a.s. \(\square \)

Proof of Corollary 3

We will skip a detailed proof to avoid repetition, except note that, under Assumption A6, \(\xi _{n}(i)\) chosen as in Corollary 3 leads to a bounded the penalty variance

$$\begin{aligned} \sigma _{n}^{2}(\theta , \theta ') \le C \tau ^{2} \varDelta _{2} S_{n}. \end{aligned}$$

which proves the claim the in the second part of the corollary. \(\square \)

Advanced composition

Here, we establish the differential privacy of the penalty algorithm using the advanced composition technique. This technique provides us a result that suggests fewer number of iterations, as shown in Sect. 3.2.1.

Assumption A7

For all \(\theta \in \varTheta \), there exists an \(\alpha > 0\) such that \(\nabla _{2} d_{n}(\cdot , \theta , \theta ') \le c n^{-\alpha }\)\(P_{n, \sigma , \theta _{0}}\)-a.s., where the \(\ell _{2}\) sensitivity \(\nabla _{2} f\) of the function \(f: {\mathcal {X}} \rightarrow {\mathbb {R}}^{d}\) for some \(d \ge 1\) is defined as

$$\begin{aligned} \nabla _{2} f = \sup _{x, y \in {\mathcal {X}} : h(x, y) \le 1} \vert \vert f(x) - f(y) \vert \vert _{2}. \end{aligned}$$

Theorem 2

Assume A7 with some \(c > 0\) and \(\alpha > 0\). For every \(\beta > 0\), \(k_{0} > 0\) and \(\theta _{0} \in \varTheta \), there exists some \(\sigma ^{2} > 0\) such that it holds P-a.s. that the \(k(n) = \lfloor k_{0} n^{2\alpha }/\log (n) \rfloor \) iterations of the penalty algorithm targeting \(\pi _{n}\) and using \(\sigma _{n}^{2}(\theta , \theta ') = \sigma ^{2}\) is \((\epsilon _{n, k(n)}, \delta _{n, k(n)})\)-differentially private where

$$\begin{aligned} \epsilon _{n, k(n)}&\le 2 \sqrt{k_{0} (2\alpha + \beta ) \beta } \frac{c}{\sigma } + 4 k_{0} \frac{c^{2}}{\sigma ^{2}} (2\alpha + \beta ) \end{aligned}$$
(11)
$$\begin{aligned} \delta _{n, k(n)}&\le 1.25 k_{0} n^{-\beta }/\log (n) + n^{-\beta }. \end{aligned}$$
(12)

In particular, for \(\beta > 1\), we have \(\epsilon _{n, k(n)} = {\mathcal {O}}(1)\) and \(\delta _{n, k(n)} = o(1/n)\).

The following theorem by Dwork and Roth (2013) on the Gaussian mechanism plays a significant role in establishing the proof of Theorem 2.

Theorem 3

Let \(\epsilon \in (0, 1)\) be arbitrary. The Gaussian mechanism for function f with \(\sigma > \nabla _{2}f \sqrt{2 \log (1.25/\delta )}/\epsilon \) is \((\epsilon , \delta )\)-differentially private.

We need to cite one more theorem about differential privacy, regarding what happens when a sequence of database access mechanisms with a certain differential privacy is applied (Dwork et al. 2010).

Theorem 4

For all \(\epsilon , \delta , \delta ' \ge 0\), the class of \((\epsilon , \delta )\)-differentially private mechanisms satisfies \((\epsilon ', k \delta + \delta ')\)-differential privacy under k-fold adaptive composition for:

$$\begin{aligned} \epsilon ' = \sqrt{2k \log (1/\delta ')} \epsilon + k \epsilon (e^{\epsilon } - 1). \end{aligned}$$

The structure of Algorithm 1 suggests that the penalty algorithm can be seen as a k-fold adaptive composition, where a database access mechanism with a certain differential privacy is called. This, combined with Theorems 3 and 4, facilitates the proof of Theorem 2.

Proof of Theorem 2

Let \(\beta ' = 2 \alpha + \beta \) and pick a finite \(\sigma > 0\) that satisfies

$$\begin{aligned} \sigma > \max _{n \ge 1} c n^{-\alpha } \sqrt{2 \beta ' \log n}. \end{aligned}$$
(13)

which is possible since the right hand side is finite for each \(\alpha > 0\). Next, let \({\mathcal {D}}_{c, \alpha , n, \sigma } = \{ {\mathcal {M}}_{\theta , \theta ', n, \sigma }: \nabla _{2} d_{n}(\cdot , \theta , \theta ') < c n^{-\alpha } \}\) be a family of the Gaussian mechanisms where \({\mathcal {M}}_{\theta , \theta ', n, \sigma }\) was defined earlier. Then, Theorem 3 implies that each mechanism in \({\mathcal {D}}_{c, \alpha , n, \sigma }\) is \((\epsilon , \delta )\)-DP where

$$\begin{aligned} \epsilon = \frac{c}{\sigma } n^{-\alpha } \sqrt{2 \beta ' \log n}, \quad \delta = 1.25 n^{-\beta '} \end{aligned}$$

since \(\sigma > c n^{-\alpha } \sqrt{2 \beta ' \log n}\) by (13), hence satisfying the condition of Theorem 3.

Furthermore, by Theorem 4, we see that for \(\delta '_{n} > 0\) and \(k \ge 1\), \({\mathcal {D}}_{c, \alpha , n, \sigma }\) satisfies \((\epsilon _{n, k}, \delta _{n, k})\)-DP under k-fold adaptive composition

$$\begin{aligned}&\epsilon _{n, k} \le \sqrt{4k \beta ' \log n \log (1/\delta '_{n})} \frac{c}{\sigma } n^{-\alpha } + 2 k \frac{c^{2}}{\sigma ^{2}} n^{-2\alpha } 2 \beta ' \log n, \\&\delta _{n, k} = 1.25 k n^{-\beta '} + \delta '_{n}. \end{aligned}$$

where we used \(e^{\epsilon } -1 \le 2 \epsilon \) since \(\epsilon < 1\) (as noted by Wang et al. (2015) also). Now, substituting \(\delta '_{n} = n^{-\beta }\) and \(k = k(n) = \lfloor k_{0} n^{2\alpha }/\log (n) \rfloor \), we have

$$\begin{aligned} \epsilon _{n, k(n)}&\le 2 \sqrt{k_{0} \beta ' \beta } \frac{c}{\sigma } + 4 k_{0} \frac{c^{2}}{\sigma ^{2}} \beta ' \\ \delta _{n, k(n)}&\le 1.25 k_{0} n^{-\beta }/\log (n) + n^{-\beta }. \end{aligned}$$

We conclude the proof by substituting \(\beta ' = 2 \alpha + \beta \) and pointing out that by Assumption A7 each iteration of the penalty algorithm targeting \(\pi _{n}\) accesses data via a mechanism from \({\mathcal {D}}_{c, \alpha , n, \sigma }\)P-a.s. \(\square \)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yıldırım, S., Ermiş, B. Exact MCMC with differentially private moves. Stat Comput 29, 947–963 (2019). https://doi.org/10.1007/s11222-018-9847-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-018-9847-x

Keywords

Navigation