A Framework for Evaluating Client Privacy Leakages in Federated Learning

Abstract

Federated learning (FL) is an emerging distributed machine learning framework for collaborative model training with a network of clients (edge devices). FL offers default client privacy by allowing clients to keep their sensitive data on local devices and to only share local training parameter updates with the federated server. However, recent studies have shown that even sharing local parameter updates from a client to the federated server may be susceptible to gradient leakage attacks and intrude the client privacy regarding its training data. In this paper, we present a principled framework for evaluating and comparing different forms of client privacy leakage attacks. We first provide formal and experimental analysis to show how adversaries can reconstruct the private local training data by simply analyzing the shared parameter update from local training (e.g., local gradient or weight update vector). We then analyze how different hyperparameter configurations in federated learning and different settings of the attack algorithm may impact on both attack effectiveness and attack cost. Our framework also measures, evaluates, and analyzes the effectiveness of client privacy leakage attacks under different gradient compression ratios when using communication efficient FL protocols. Our experiments additionally include some preliminary mitigation strategies to highlight the importance of providing a systematic attack evaluation framework towards an in-depth understanding of the various forms of client privacy leakage threats in federated learning and developing theoretical foundations for attack mitigation.

7 Appendices

1.1 7.1 Proof of Theorem 1

Assumption 1

(Convexity). we say f(x) is convex if

$$\begin{aligned} f(\alpha x + (1-\alpha )x') \le \alpha f(x) + (1-\alpha ) f(x'), \end{aligned}$$

(3)

where $x,x'$ are data point in $\mathbb {R}^d$, and $\alpha \in [0,1]$.

Lemma 1

If a convex f(x) is differentiable, we have:

$$\begin{aligned} f(x') -f(x) \ge \langle \nabla f(x), x'-x \rangle . \end{aligned}$$

(4)

Proof

Equation 3 can be rewritten as: $\frac{f(x'+\alpha (x-x'))-f(x')}{\alpha } \le f(x)-f(y).$ When $\alpha \rightarrow 0$, we complete the proof.

Assumption 2

(Lipschitz Smoothness). With Lipschitz continuous on the differentiable function f(x) and Lipschitz constant L, we have:

$$\begin{aligned} || \nabla f(x) - \nabla f(x') \le L||x-x'||, \end{aligned}$$

(5)

Lemma 2

If f(x) is Lipschitz-smooth, we have:

$$\begin{aligned} f(x^{t+1}) -f(x^{t}) \le - \frac{1}{2L}||\nabla f(x^T)||^2_2 \end{aligned}$$

(6)

Proof

Using the Taylor expansion of f(x) and the uniform bound over Hessian matrix, we have

$$\begin{aligned} f(x') \le f(x) + \langle \nabla f(x), x'-x \rangle + \frac{L}{2}||x'-x||^2_2. \end{aligned}$$

(7)

By inserting $x' = x - \frac{1}{L}\nabla f(x)$ into Eq. 5 and Eq. 7, we have:

$$\begin{aligned} f(x - \frac{1}{L}\nabla f(x)) -f(x)&\le - \frac{1}{L} \langle \nabla f(x), \nabla f(x) \rangle + \frac{L}{2}||\frac{1}{L} \nabla f(x)||^2_2 = - \frac{1}{2L}||\nabla f(x)||^2_2 \end{aligned}$$

Lemma 3

(Co-coercivity). A convex and Lipschitz-smooth f(x) satisfies:

$$\begin{aligned} \langle \nabla f(x') - \nabla f(x), x'-x \rangle \ge \frac{1}{L} || \nabla f(x') - \nabla f(x)|| \end{aligned}$$

(8)

Proof

Due to Eq. 5,

$$\begin{aligned} \langle \nabla f(x') - \nabla f(x), x'-x \rangle&\ge \langle \nabla f(x') - \nabla f(x), \frac{1}{L}(\nabla f(x') - \nabla f(x)) \rangle = \frac{1}{L} || \nabla f(x') - \nabla f(x)|| \end{aligned}$$

Then we can proof the attack convergence theorem: $f(x^T)-f(x^*) \le \frac{2L||x^0-x^*||^2}{T}.$

Proof

Let f(x) be convex and Lipschitz-smooth. It follow that

$$\begin{aligned} ||x^{t+1}-x^*||^2_2&= ||x^t-x^*-\frac{1}{L} \nabla f(x^t)||^2_2 \nonumber \\&= ||x^t-x^*||^2_2 - 2\frac{1}{L}\langle x^t-x^*, \nabla f(x^t) \rangle + \frac{1}{L^2}||\nabla f(x^t)||^2_2 \nonumber \\&\le ||x^t-x^*||^2_2 - \frac{1}{L^2}||\nabla f(x^t)||^2_2 \end{aligned}$$

(9)

Equation 9 holds due to Eq. 8 in Lemma 3. Recall Eq. 6 in Lemma 2, we have:

$$\begin{aligned} f(x^{t+1}) -f(x^*) \le f(x^{t}) -f(x^*) - \frac{1}{2L}||\nabla f(x^t)||^2_2. \end{aligned}$$

(10)

By applying convexity,

$$\begin{aligned} f(x^t)-f(x^*)&\le \langle \nabla f(x^t), x^t-x^* \rangle \nonumber \\&\le ||\nabla f(x^t)||_2||x^t-x^*|| \nonumber \\&\le ||\nabla f(x^t)||_2||x^1-x^*||. \end{aligned}$$

(11)

Then we insert Eq. 11 into Eq. 10:

$$\begin{aligned}&f(x^{t+1}) -f(x^*) \le f(x^{t}) -f(x^*) - \frac{1}{2L}\frac{1}{||x^1-x^*||^2}( f(x^t)-f(x^*))^2 \nonumber \\&\Rightarrow \frac{1}{f(x^{t}) -f(x^*)} \le \frac{1}{f(x^{t+1}) -f(x^*)} - \beta \frac{f(x^t)-f(x^*)}{f(x^{t+1})-f(x^*)} \end{aligned}$$

(12)

$$\begin{aligned}&\Rightarrow \frac{1}{f(x^{t}) -f(x^*)} \le \frac{1}{f(x^{t+1}) -f(x^*)} - \beta \end{aligned}$$

(13)

$$\begin{aligned}&\Rightarrow \beta \le \frac{1}{f(x^{t+1}) -f(x^*)} - \frac{1}{f(x^{t}) -f(x^*)}, \end{aligned}$$

(14)

where $\beta = \frac{1}{2L}\frac{1}{||x^1-x^*||^2}$. Equation 12 is done by divide both side with $(f(x^{t+1}) -f(x^*))(f(x^{t}) -f(x^*))$ and Eq. 13 utilizes $f(x^{t+1}) -f(x^*) \le f(x^{t}) -f(x^*)$. Then, following by induction over $t=0,1,2,..T-1$ and telescopic cancellation, we have

$$T\beta \le \frac{1}{f(x^{T}) -f(x^*)} - \frac{1}{f(x^{0}) -f(x^*)} \le \frac{1}{f(x^{T}) -f(x^*)}.$$

$$\begin{aligned}&T\beta \le \frac{1}{f(x^{T}) -f(x^*)} - \frac{1}{f(x^{0}) -f(x^*)} \le \frac{1}{f(x^{T}) -f(x^*)} \end{aligned}$$

(15)

$$\begin{aligned}&\Rightarrow \frac{T}{2L}\frac{1}{||x^1-x^*||^2} \le \frac{1}{f(x^{T}) -f(x^*)} \end{aligned}$$

(16)

$$\begin{aligned}&\Rightarrow f(x^{T}) -f(x^*) \le \frac{2L||x^0-x^*||^2}{T}. \end{aligned}$$

(17)

Thus complete the proof.