Privacy-Preserving Split Learning via Pareto Optimal Search

Abstract

With the rapid development of deep learning, it has become a trend for clients to perform split learning with an untrusted cloud server. The models are split into the client-end and server-end with features transmitted in between. However, features are typically vulnerable to attribute inference attacks to the input data. Most existing schemes target protecting data privacy at the inference, but not at the training stage. It remains a significant challenge to remove private information from the features while accomplishing the learning task with high utility.

We found the fundamental issue is that utility and privacy are mostly conflicting tasks, which are hardly handled by the linear scalarization commonly used in previous works. Thus we resort to the multi-objective optimization (MOO) paradigm, seeking a Pareto optimal solution according to the utility and privacy objectives. The privacy objective is formulated by the mutual information between feature and sensitive attributes and is approximated by Gaussian models. In each training iteration, we select a direction that balances the dual goal of moving toward the Pareto Front and toward the users’ preference while keeping the privacy loss under the preset threshold. With a theoretical guarantee, the privacy of sensitive attributes is well preserved throughout training and at convergence. Experimental results on image and tabular datasets reveal our method is superior to the state-of-the-art in terms of utility and privacy.

Appendices

A Notations

We list all notations in this paper in Table 6 for better understanding.

Table 6. Notations used in this paper

B Proof for Loss Formulation

Proof

$\textbf{Lower} \, \textbf{bound} \, \textbf{of} \, \textbf{I}(\textbf{z};\textbf{y}).$ According to [13], for any conditional distribution q(y|z), it holds that

$$\begin{aligned} I(z;y) \ge H(y)+\mathbb {E}_{z,y}\log q(y|z). \end{aligned}$$

(13)

Hence the lower bound of I(z; y) can be defined as

$$\begin{aligned} \mathcal {L}=H(y)+\max _{\phi } \mathbb {E}_{z,y} \log q_{\phi }(y|z). \end{aligned}$$

(14)

The model with parameter $\phi$ is exactly the classifier performing the original task using uploaded feature z. For fixed z, the better the classifier is trained, the better estimation result it will achieve. Since H(y) is a constant, we only need to optimize the second term of $\mathcal {L}$, which can be translated into minimizing the cross entropy loss. Denoting the cross entropy loss as $CE(\cdot )$, the utility loss can be defined as:

$$ L_u = CE(g(f(x)),y). $$

$\textbf{Upper} \, \textbf{Bound} \, \textbf{of} \, \textbf{I}(\textbf{z};\textbf{s}).$ Assuming s is a discrete variable, the mutual information can be written as

$$\begin{aligned} I(z;s)=\sum _{a}p(s_a) \int p(z|s_a)\log \frac{p(z|s_a)}{\sum _{b}p(s_b)p(z|s_b)} \text {d} z. \end{aligned}$$

(15)

By Jensen’s Inequality, I(z; s) is upper bounded by:

$$\begin{aligned} \mathcal {U} =\sum _{a}\sum _{b \ne a}p(s_a)p(s_b)KL[p(z|s_a)||p(z|s_b)]. \end{aligned}$$

(16)

C Proof of Lemma 2

Proof

We prove by contradiction. The formula in (10) can be written as

$$\begin{aligned} ||C\boldsymbol{\beta } - \boldsymbol{a}||_2^2 = ||G^T \boldsymbol{d}||_2^2 + ||\boldsymbol{a}||_2^2 - 2\boldsymbol{a}^T G^T \boldsymbol{d}. \end{aligned}$$

(17)

If $\boldsymbol{a}^T G^T \boldsymbol{d} < 0$ at convergence, then $||C\boldsymbol{\beta } - \boldsymbol{a}||_2^2 > ||\boldsymbol{a}||_2^2$, which is greater than the result at $\boldsymbol{d} = \boldsymbol{0}$. Hence it contradicts with the fact that the current solution is optimal. Thus $\boldsymbol{a}^T G^T \boldsymbol{d} \ge 0$ holds by solving (10).

D Proof of Lemma 3

Proof

When rank(C) = rank(G) = m, we can always find $\boldsymbol{\beta }^{*}$ such that $C\boldsymbol{\beta }^{*} = \boldsymbol{l}$. Due to the range constraint of $\boldsymbol{\beta }$, we need to scale $\boldsymbol{\beta }^{*}$ to the range of $[-1,1]^m$ without altering the direction of $C\boldsymbol{\beta }^{*}$. Hence we could get $G^T \boldsymbol{d} = C \boldsymbol{\beta } = k \boldsymbol{l} \succ \boldsymbol{0} (k>0)$, and $\boldsymbol{a}^T G^T \boldsymbol{d} = k \boldsymbol{a}^T \boldsymbol{l} = 0$. The last equality holds due to our claim that $\boldsymbol{a}^T \boldsymbol{l} = 0$.

E Proof of Theorem 1

Proof

If $\theta$ is a Pareto optimal solution in optimizing (12), by Pareto Criticality ([5], ch4), G has rank $m-1$. Hence rank(C) = rank(G) = $m-1$. Note that $\boldsymbol{a} \ne \boldsymbol{0}$ at this point, otherwise the optimization terminates for $\boldsymbol{d} = \boldsymbol{0}$.

Let $\mathcal {S} = \{ C \boldsymbol{\beta } | \boldsymbol{\beta } \in [-1,1]^m\}$, Col(C) be the column space of C, and Null(C) be the null space of C. It is clear that $\mathcal {S} \subseteq Col(C)$, and Col(C) and Null(C) are orthogonal complement spaces. By minimizing the objective of (12), $C \boldsymbol{\beta }$ turns out to be the best approximation of $\boldsymbol{a}$ on $\mathcal {S}$. Now we prove by contradiction. Assuming $\boldsymbol{d} = \boldsymbol{0}$, we have $C \boldsymbol{\beta } = G^{T} \boldsymbol{d} = \boldsymbol{0}$. Thus the orthogonal projection of $\boldsymbol{a}$ on $\mathcal {S}$ is $\boldsymbol{0}$. Hence the orthogonal projection of $\boldsymbol{a}$ on Col(C) is $\boldsymbol{0}$, which means $\boldsymbol{a} \in Null(C)$. Due to the orthogonality of $\boldsymbol{a}$ and $\boldsymbol{l}$, $\boldsymbol{l} \in Col(C)$, which means $\exists \boldsymbol{\alpha } \ \mathrm {s.t.} \ C \boldsymbol{\alpha } = G^T G \boldsymbol{\alpha } = \boldsymbol{l} \succeq \boldsymbol{0}$. So we can find a direction $\boldsymbol{d} = G \boldsymbol{\alpha }$ to decrease all losses, which is contradictory to the condition of Pareto optimality. Therefore, $\boldsymbol{d}$ is not $\boldsymbol{0}$.

According to Lemma 1, Lemma 2 and Lemma 3, the direction $\boldsymbol{d}$ found by (12) always satisfies $\boldsymbol{a}^T G^T \boldsymbol{d} \ge 0$, so with proper step size, $\mu$ keeps decreasing.