Privacy Leakage in Privacy-Preserving Neural Network Inference

Abstract

The community has seen many attempts to secure machine learning algorithms from multi-party computation or other cryptographic primitives. An interesting 3-party framework (SCSDF hereafter) for privacy-preserving neural network inference was presented at ESORICS 2020. SCSDF defines several protocols for non-linear activation functions including ReLU, Sigmoid, etc. In particular, these protocols reckon on a protocol DReLU (derivative computation for ReLU function) they proposed as a building block. All protocols are claimed secure (against one single semi-honest corruption and against one malicious corruption). Unfortunately, the paper shows that there exists grievous privacy leakage of private inputs during SCSDF executions. This would completely destroy the framework security. We first give detailed cryptanalysis on SCSDF from the perspective of the real-ideal simulation paradigm and indicate that these claimed-secure protocols do not meet the underlying security model. We then go into particular steps in SCSDF and demonstrate that the signs of input data would be inevitably revealed to the (either semi-honest or malicious) third party responsible for assisting protocol executions. To show such leakage more explicitly, we perform plenteous experiment evaluations on the MNIST dataset, the CIFAR-10 dataset, and CFD (Chicago Face Database) for both ReLU and Sigmoid non-linear activation functions. All experiments succeed in disclosing original private data of the data owner in the inference process. Potential countermeasures are recommended and demonstrated as well.

A A Preliminary

1.1 A.1 A.1 Neural Network

A neural network usually executes in a layer-by-layer fashion. It mainly consists of two different layers: linear layers and non-linear layers. The computation of linear layers (either a fully connected layer or a convolutional layer) generally depends on matrix multiplication. The non-linear layer could be performed by several different activation functions, e.g., ReLU and Sigmoid. ReLU function is formulated as $f(x)=\max (0,x)$ and Sigmoid as $f(x)= \frac{1}{1+e^{-x}}$.

1.2 B.2 A.2 Fixed-Point Number

Neural networks usually operate on floating-point numbers that are not suitable for some cryptographic primitives. Thus MPC frameworks usually use the encoding of fixed-point numbers. A fixed-point value is defined as an l-bit integer by using 2’s complement representation where the bottom d bits denote the decimal ($d<l$) and the MSB denotes the sign of the number, i.e., $\textrm{MSB}(x)=1$ if x is negative, and 0 otherwise.

1.3 C.3 A.3 Addictive Secret Sharing

In the framework SCSDF, all values are 2-out-of-2 secret shared between the server and the client. We say x is 2-out-of-2 secret shared in $\mathbb {Z}_{2^l}$ between $P_0$ and $P_1$ if $[x]_0, [x]_1\in \mathbb {Z}_{2^l}$ such that $x = [x]_0 +[x]_1$ and $[x]_i$ is held by $P_i$, $i \in \{0,1\}$.

Sharing Protocol $\pi _\textrm{Share}$. $P_i$ generates a sharing of its input x by sampling $r \in _R \mathbb {Z}_{2^l}$ and sending $x-r$ to $P_{1-i}$. $P_i$ gets $[x]_i=r$ and $P_{1-i}$ gets $[x]_{1-i}=x-r$.

Reconstruction Protocol $\pi _\textrm{Rec}$. To reconstruct x, the parties mutually exchange their missing share, then each of them can compute $x = [x]_0 +[x]_1$.

Addition Operations. Addictive sharing is linear in the sense that given two shared values [x] and [y], the parties can get $[z] =[x]+[y]$ by local computation.

Multiplication Operations $\pi _\textrm{Multi}$. Given [x], [y], the goal of protocol $\pi _\textrm{Multi}$ is to generate [z] where $z = x \cdot y$. It can be performed based on preassigned Multiplication Triples [7]. A multiplication triple is a triple $(a,b,c) \in Z_{2^l}$ such that $a\cdot b = c$. Two parties hold secret sharing of a triple (a, b, c). $P_i$ computes $[e]_i=[x]_i-[a]_i$ and $[f]_i = [y]_i- [b]_i$, then call $\pi _\textrm{Rec}$ to reconstruct e and f. $P_i$ sets $z_i=-i \times e\times +f \times x_i+e \times y_i+c_i$. The multiplication operations can be easily extended to matrices.

1.4 D.4 A.4 Threat Model

Similar to [23], we consider an adversary who can corrupt only one of the three parties in the semi-honest model or malicious model.

A party corrupted by a semi-honest adversary follows protocol steps but tries to learn additional information from received messages. Our security definition uses the real-ideal paradigm [9] which is a general method to prove protocol security. In the real world, the parties interact with each other according to the specification of a protocol $\pi$. In the ideal world, the parties have access to a trusted third party (TTP) that implements an ideal functionality $\mathcal {F}$. The executions in both worlds are coordinated by an environment $\mathcal {Z}$, who chooses the inputs for the parties and plays the role of a distinguisher between the real and ideal executions. We say that $\pi$ securely realizes the ideal functionality $\mathcal {F}$ if for any adversary $\mathcal {A}$ in the real world, there exists an adversary $\mathrm Sim$ (called a simulator) in the ideal world, such that no $\mathcal {Z}$ can distinguish an execution of the protocol $\pi$ with the parties and $\mathcal {A}$ from an execution of the ideal functionality $\mathcal {F}$ with the parties and $\mathrm Sim$. To be more formal, we give the following definition.

Definition 1

A protocol $\pi$ securely realizes an ideal functionality $\mathcal {F}$ if for any adversary $\mathcal {A}$, there exists an adversary $\mathrm Sim$ (called a simulator) such that, for any environment $\mathcal {Z}$, the following holds:

$$\begin{aligned} \mathrm REAL_{\pi ,\mathcal {A},\mathcal {Z},\lambda } \overset{c}{\approx }\ IDEAL_{\mathcal {F},Sim,\mathcal {Z},\lambda } \end{aligned}$$

where $\overset{c}{\approx }$ denotes computational indistinguishability, $\lambda$ denotes security parameter, $\mathrm REAL_{\pi ,\mathcal {A},\mathcal {Z},\lambda }$ represents the view of $\mathcal {Z}$ in the real protocol execution with $\mathcal {A}$ and the parties, and $\mathrm IDEAL_{\mathcal {F},Sim,\mathcal {Z},\lambda }$ represents the view of $\mathcal {Z}$ in the ideal execution with the functionality $\mathcal {F}$, the simulator $\mathrm Sim$ and the parties.

The environment’s view includes (without loss of generality) all messages that honest parties send to the adversary as well as the outputs of the honest parties.

Malicious Security: In malicious security model, an adversary may arbitrarily deviate from protocol specification. Araki et al. [6] formalize the notion of privacy against malicious adversaries in client-server model using an indistinguishability-based argument: for any two inputs of honest parties, the views of the adversary in protocol executions are indistinguishable. This notion is weaker than full simulation-based malicious security because it does not guarantee protocol correctness in the presence of malicious behavior. However, it does provide a guarantee of the privacy of the protocol. SCSDF [23] and SecureNN consider above malicious security model.