Multiple Privacy Regimes Mechanism for Local Differential Privacy

Abstract

Local differential privacy (LDP), as a state-of-the-art privacy notion, enables users to share protected data safely while the private real data never leaves user’s device. The privacy regime is one of the critical parameters balancing between the correctness of the statistical result and the level of user’s privacy. In the majority of current work, authors assume that the privacy regime is totally determined by the service provider and dispatched to all users. However, it is inelegant and unpromising for all users to accept the same privacy level in real world. In this paper, we propose a new LDP estimation method MLE which is applicable for the scenario of multiple privacy regimes. MLE uses the idea of parameter estimation to merge the results generated by users of different privacy levels. We also propose an extension of MLE to handle the situation when all users’ regimes are in a continuous distribution. We also provide an Adapt estimator which assigns users to use different LDP schemes based on their regimes, and it performs better than the estimator with only one fixed LDP scheme. Experiments show that our methods provide a higher level of accuracy than previous proposals in this multiple regimes scenario.

9 Appendix

A. Proof of Theorem 1

Proof

$({\hat{f_{(1)}}, \hat{f_{(2)}}, ...,\hat{f_{(M)}}})$ are drawn from different normal distributions, normal distribution has probability density function as follows:

$$\begin{aligned} g(x)=\frac{1}{\sqrt{2\pi } \sigma ^2}exp(-\frac{(x-u)^2}{2\sigma ^2}) \end{aligned}$$

According to probability density function g(x), we know the closer estimation $\hat{f_{(m)}}$ is to the expectation, the greater the $g(\hat{f_{(m)}})$. For ease of calculation, we use Eq. 2 to ignore the effect of $f_i$ on variance. $g(\hat{f_{(m)}})$ actually has only one variable–expectation. Separately bring each $\hat{f_{(m)}}$ into function and multiply these functions according to maximum likelihood, we get the final target function which needs to be maximized.

$$\begin{aligned} F(f) =\prod _{m=1}^{M}{g_m(f)} \end{aligned}$$

We first turn it to logarithmic function $y = ln(F(f))$, and after derivation, the first derivative and the two derivative of F(f) are obtained sequentially.

$$\begin{aligned} y' = \frac{\partial ln(F(f))}{\partial f} = - \sum \nolimits _{m=1}^{M}\frac{\hat{f_{(m)}}-f}{\sigma _m^2} \end{aligned}$$

$$\begin{aligned} y'' = \frac{y'}{\partial f} = \sum \nolimits _{m=1}^{M}\frac{1}{\sigma _m^2} \end{aligned}$$

Through simple analysis, $y''$ is always bigger than 0 and $y'$ is a strictly monotone increasing function. So F(f) is a convex function with a max value. Then set the first derivative function to zero, here when $\hat{f}=(\sum _{m=1}^{M}\frac{\hat{f_{(m)}}}{\sigma _m^2}) / (\sum _{m=1}^{M}\frac{1}{\sigma _m^2})$, we can get the maximum of the F(f).

B. Proof of Theorem 2

Proof

First use $t_m$ to denote $var(\hat{f_{(m)}})$, the final estimation using maximum likelihood is $\hat{f}=(\sum _{m=1}^{M}\frac{\hat{f_{(m)}}}{t_m}) / (\sum _{m=1}^{M}\frac{1}{t_m})$. When we calculate the variance of $\hat{f}$ as follows:

$$\begin{aligned} var(\hat{f})=var(\sum \nolimits _{m=1}^{M}\frac{\hat{f_{(m)}}}{t_m} / \sum \nolimits _{m=1}^{M}\frac{1}{t_m}) \end{aligned}$$

Since the estimations $f_m (m\in [M])$ are independent of each other, and $t_m$ here is actually a constant number.

$$\begin{aligned} var(\hat{f}) = \sum \nolimits _{m=1}^{M}(\frac{var(\hat{f_m})}{t_m^2}) / (\sum \nolimits _{m=1}^{M}\frac{1}{t_m})^2 = 1 /\sum \nolimits _{m=1}^{M}\frac{1}{t_m} \end{aligned}$$

C. Proof of Lemma 3

Proof

We still judge the accuracy of the final estimation from the perspective of variance. The Lemma 1 shows base rappor’s estimation variance is $var(\hat{f_i} )= \frac{e^{\epsilon /2}}{n (e^{\epsilon /2} -1)^2}$, for the sake of simplicity, let’s first assume $e^{\epsilon /2}\gg 1$ and use $t_m$ to denote $var(\hat{f_{(m)}})$. So that $t_m=(1/(n_m e^{\epsilon _m/2}))$.

We are clear that the $\hat{f}$ ’s variance and $\hat{f_{(m)}}$’s variance are the same format, because f is regarded as using Base RAPPOR on the whole population while all users have the same privacy regime $\epsilon '$.

Combining the above equations and Theorem 2 together, we can find $\epsilon '=2*ln \frac{\sum n_m*exp(\epsilon _m)}{\sum n_m}$. If $e^{\epsilon /2}\gg 1$ doesn’t hold in some situation, the calculation can still be based on the above formula and the result will become a little more complicated.