Towards Understanding the fairness of differentially private margin classifiers

Abstract

Margin classifiers, such as Support Vector Machine, are usually critical in the high-stakes decision domains. In recent years, differential privacy has been widely employed in margin classifiers to protect user privacy. However, incorporating differential privacy into margin classifiers might adversely cause the fairness issue in the sense that differentially private margin classifiers have significantly different true positive rates on different groups that are determined by sensitive attributes (e.g. race). In order to address this issue, we are motivated to identify the factor that dominates the fairness of differentially private margin classifiers based on well-designed experiments and further analysis. We first conduct an empirical study on three classical margin classifiers learned via three representative differentially private empirical risk minimization algorithms, respectively. The empirical result shows that the fairness of differentially private margin classifiers strongly depends on the fairness of their non-private versions. We then analyze how differential privacy impacts the fairness of margin classifiers and confirm the empirical study results. In a general sense, our study shows that when non-private margin classifiers are fair, the fairness of their differentially private counterparts can be ensured.

Appendices

A: Deviation properties of AMP, PSGD, DPSGD

In this section, we identify the deviation properties of AMP, PSGD and DPSGD.

The pseudocodes of AMP are shown in Algorithm 1. According to the design of AMP, we identify its deviation property as follows.

Proof of Theorem 1

AMP follows $(\frac{n\gamma }{\Lambda }+(\sqrt{2p\log \frac{2}{\alpha }})( \frac{4L}{\Lambda \epsilon _3}(1+\sqrt{2\log \frac{1}{\delta _1}}) + \frac{n\gamma }{\Lambda \epsilon _2}(1+\sqrt{2\log \frac{1}{\delta _2}}))$,$\alpha$)-deviation.

Proof of Theorem 1

The utility guarantee of AMP contains two parts. First, it bounds the distance between optimal model parameters $\theta _{approx}$ under private loss function and optimal model parameters $\theta ^*$ under non-private loss function. Second, it bounds the distance between private output $\theta _{out}$ and $\theta _{approx}$. The first bound is $\frac{2n\left\| \mathbf {b_1} \right\| _2}{\Lambda }$ (see inequality 10 of [4]). The second bound is $\frac{n\gamma }{\Lambda } + \left\| \mathbf {b_2} \right\| _2$(see inequality 5 of [4]). Therefore, the total bound of the deviation of model parameters is $\frac{n(\gamma +2\left\| \mathbf {b_1} \right\| _2)}{\Lambda } + \left\| \mathbf {b_2} \right\| _2$, where $\mathbf {b_1}$ and $\mathbf {b_2}$ are distributed as $\mathcal N(0,\sigma _1^2I_{p\times p}), \mathcal N(0,\sigma _2^2I_{p\times p})$, here $\sigma _1$ = $\frac{\frac{2L}{n}(1+\sqrt{2\log \frac{1}{\delta _1}})}{\epsilon _3}$, $\sigma _2$=$\frac{\frac{n\gamma }{\Lambda }(1+\sqrt{2\log \frac{1}{\delta _2}})}{\epsilon _2}$.

According to Lemma 2 in [43]: with probability $\ge 1-\frac{\alpha }{2}$,

$$\begin{aligned} \left\| \mathbf {b_s} \right\| _2 \le \sigma _s\sqrt{2p\log \frac{2}{\alpha }} \end{aligned}$$

AMP thus follows $(\frac{n\gamma }{\Lambda }+(\sqrt{2p\log \frac{2}{\alpha }})( \frac{4L}{\Lambda \epsilon _3}(1+\sqrt{2\log \frac{1}{\delta _1}}) + \frac{n\gamma }{\Lambda \epsilon _2}(1+\sqrt{2\log \frac{1}{\delta _2}}))$,$\alpha$)-deviation.

We show the pseudocodes of PSGD in Algorithm 2. We then identify the deviation property of PSGD.

Proof of Lemma 2

PSGD follows ($\frac{2p\ln (p/\alpha )kTL\eta }{n\epsilon }$,$\alpha$)-deviation.

Proof of Lemma 2

The sensitivity of PSGD is $\frac{2kTL\eta }{n}$ (see Corollary 1 in [5]). As the noise is directly added on the final model, the Euclidean distance between private model and non-private model is the $L_2$ norm of the noise, which is distributed as Gamma distribution $\Gamma$(p, $\frac{2kTL\eta }{n\epsilon }$). According to Theorem 2 in [5]: for the noise vector $\kappa$, whose $L_2$ norm is distributed according to the Gamma distribution $\Gamma (p, \Delta )$, we have that with probability at least 1-$\alpha$, $\left\| \kappa \right\| _2 \le p\Delta ln(\frac{p}{\alpha })$. Therefore, PSGD follows ($\frac{2p\ln (p/\alpha )kTL\eta }{n\epsilon }$,$\alpha$)-deviation.

We then identify the deviation property of DPSGD under a strong convexity and continuity assumption on loss functions. The pseudocodes of DPSGD are shown in Algorithm 3.

Proof of Lemma 3

When applying DPSGD to optimize a $\Delta$-strongly convex and $L_2$-Lipchitz continuous loss function, if we set learning rate as $\frac{1}{\Delta t}$, DPSGD follows ($\frac{4(L^2 + p\sigma ^2)}{\Delta ^2T\alpha }$,$\alpha$)-deviation.

Proof of Lemma 3

Let $G_t$ as the gradient at iteration $t$, according to Theorem 2.4 of [22],

$$\begin{aligned} \mathbb {E}[\left\| G_t \right\| _{2}^{2}] \le L^2 + p\sigma ^2 \end{aligned}$$

Then according to Lemma 1 of [44],

$$\begin{aligned} \mathbb {E}[\left\| \theta _{t} - \theta ^* \right\| _2] \le \frac{4(L^2 + p\sigma ^2)}{\Delta ^2t} \end{aligned}$$

Finally, according to Markov inequality,

$$\begin{aligned} Pr(\left\| \theta _{priv} - \theta ^* \right\| _2 \le \frac{4(L^2 + p\sigma ^2)}{\Delta ^2T\alpha }) \ge 1-\alpha \end{aligned}$$

The deviation properties of AMP, PSGD and DPSGD show that $\lambda$ is inversely propertional to $\alpha$. Therefore, they deviate private hyperplane from the original hyperplane little with high probability.

B: Empirical Results of the Rest Three Datasets

We train Linear SVM, Kernal SVM and LR models on German, Student, Arrhythmia datasets under the same setting with that of Section 4. The test results are shown in Figures 10, 11 and 12. Even though with large variances, from the average results, we can find that when a significant TPR gap exists in the non-private model, the private models will have larger TPR gaps. On the other hand, when the TPR gaps of non-private models are negligible, the private models will have similar, even smaller, TPR gaps with the non-private models. We then explain why the TPR gaps of margin classifiers trained on these three datasets have such large variances.

Fig. 10

TPR gaps of non-private and differentially private SVM models trained on German, Student, and Arrhythmia datasets

Fig. 11

TPR gaps of non-private and differentially private Kernel SVM model strained on German_185,0.3, Student_225,0.1, and Arrhythmia_390,0.3 datasets. The subscripts of datasets indicate the dimension of the target feature space and standard variance of kernel function approximation method

Fig. 12

TPR gaps of non-private and differentially private LR models trained on German, Student, and Arrhythmia datasets

The overview of German, Student, Arrhythmia datasets is shown in Table 6. The sizes of these three datasets are all less than or equal to 1,000. Consequently, the sizes of their testing datasets are less than or equal to 200. Even though labels are balanced distributed and different groups have the same number of data samples, the number of ‘positive’ samples of each group in testing datasets is less than or equal to 50. Therefore, the inversion of one data sample’s prediction changes the TPR of the corresponding group by at least 2%. As a result, the test results of these datasets are greatly impacted by the randomness of noise sampling, and all have large variances.

Table 6 Overview of supplementary datasets