Privacy preserving perceptron learning in malicious model

Abstract

Privacy preserving data mining algorithms are proposed to protect the participating parties’ data privacy in data mining processes. So far, most of these algorithms only work in the semi-honest model that assumes all parties follow the algorithms honestly. In this paper, we propose two privacy preserving perceptron learning algorithms in the malicious model, for horizontally and vertically partitioned data sets, respectively. So far as we know, our algorithms are the first perceptron learning algorithms that can protect data privacy in the malicious model.

Appendix: Security proofs

In this section, we give formal security proofs for our algorithms in the malicious model. In particular, we adopt the security definitions in [19], which adopt the security definitions in [5] and apply them in the two-party case. The security of a protocol is formally defined by comparing the executions of this protocol in the real-life model that an active static adversary is present and in the ideal model that an incorruptible trusted party is available also.

Definition 1

The Real-Life Model (two-party case) [5] Let π be a two-party protocol and ${k\in\mathbb{N}}$ be the security parameter. In the real-life model, each party $P^i\,(i\in\{0,1\})$ has a secret input x ⁱ _s and a public input x ⁱ _p . After executing the protocol, each party gets a private output y ⁱ _j and returns a public output y ⁱ _p . Let A be an adversary that can corrupt one of the two players and $C\in\{0,1\}$ be the index of the corrupted party. A receives the public input and output of all parties.

Let $\overrightarrow{x}=(x^0_s,x^0_p,x^1_s,x^1_p)$ be the two parties’ input, $\overrightarrow{r} = (r^0,r^1,r^A)$ be the random input of P ⁰, P ¹ and A, and $a\in{\{0,1\}}^{\ast}$ be the A’s auxiliary input. After the execution of π in the real-life model with given input $\overrightarrow{x}$ and under the attack of A, denote by ${ADVR}_{\pi,A}(k,\overrightarrow{x},C,a,\overrightarrow{r})$ and ${EXEC}^i_{\pi,A}(k,\overrightarrow{x},C,a,\overrightarrow{r})$ the output of the adversary and the output of party P ⁱ, respectively. Let

$$ \begin{aligned} EXEC_{{\pi ,A}} (k,\vec{x},C,a,\vec{r}) & = (ADVR_{{\pi ,A}} (k,\vec{x},C,a,\vec{r}), \\ & \quad EXEC_{{\pi ,A}}^{0} (k,\vec{x},C,a,\vec{r}), \\ & \quad EXEC_{{\pi ,A}}^{1} (k,\vec{x},C,a,\vec{r})) \\ \end{aligned} $$

and denote by ${EXEC}_{\pi,A}(k,\overrightarrow{x},C,a)$ the random variable ${EXEC}_{\pi,A}(k,\overrightarrow{x},C,a,\overrightarrow{r})$ when $\overrightarrow{r}$ is uniformly chosen.

Finally, we define a distribution ensemble EXEC _π,A with security parameter k and index $(\overrightarrow{x},C,a)$ by

$$ EXEC_{\pi,A} = {\{EXEC_{\pi,A}(k,\overrightarrow{x},C,a)\}}_{k\in{\mathbb{N}},\overrightarrow{x}\in{({\{0,1\}}^\ast)}^4,C\in\{0,1\}, a\in{\{0,1\}}^\ast}. $$

Definition 2

The Ideal Model (two-party case) [5] Let ${f: \mathbb{N}\times{({\{0,1\}}^\ast)}^{4}\times{\{0,1\}}^\ast\to{({\{0,1\}}^\ast)}^{4}}$ be a polynomial-time-bounded probabilistic two-party function. Define the inputs and outputs of f as follows

$$ f(k,x^0_s,x^0_p,x^1_s,x^1_p,r) = (y^0_s,y^0_p,y^1_s,y^1_p) $$

In the ideal model, each party $P^i\,(i\in\{0,1\})$ sends her input (x ⁱ _s , x ⁱ _p ) to an incorruptible trusted party. The party draws r uniformly random and returns (y ⁱ _j , y ⁱ _p ) to party P ⁱ. The entire procedure takes place in the presence of an active static adversary S. At the beginning of the procedure, S sees both parties’ public inputs and also the corrupted party P ^C’s private input, substitutes (x ^C _s , x ⁱ _j ) with $({x^C_s}^\prime,{x^C_p}^\prime)$ of his choice. Therefore, f is evaluated by the trusted party using the modified inputs. Similarly, we define

$$ \begin{aligned} {IDEAL}_{f,S}(k,\overrightarrow{x},C,a,\overrightarrow{r})&=({ADVR}_{f,S}(k,\overrightarrow{x},C,a,\overrightarrow{r}),\\ &{IDEAL}^0_{f,S}(k,\overrightarrow{x},C,a,\overrightarrow{r}), \\ &{IDEAL}^1_{f,S}(k,\overrightarrow{x},C,a,\overrightarrow{r})) \\ \end{aligned} $$

and a distribution ensemble by

$$ IDEAL_{f,S} = {\{IDEAL_{f,S}(k,\overrightarrow{x},C,a)\}}_{k\in{\mathbb{N}},\overrightarrow{x}\in{({\{0,1\}}^\ast)}^4,C\in\{0,1\}, a\in{\{0,1\}}^\ast}. $$

Definition 3

Security in the Real-Life Model (two-party case) [5] Let f be a two-party function and let π be a two-party protocol. π securely evaluates f in the malicious model if for any polynomial-time bounded active static adversary A, there exists an polynomial-time bounded ideal-model adversary S such that

$$ IDEAL_{f,S}\stackrel{c}{\approx}EXEC_{\pi,A} $$

where $\stackrel{c}{\approx}$ denotes the computational indistinguishability between two distribution ensembles.

We need to combine several secure function evaluation protocols to construct our perceptron learning protocol. In order to prove the security of the composed protocol, we first prove the composed protocol is secure when these secure sub-protocols can be called as oracles (the corresponding model is called hybrid model [5]). Then, using the composition theorem in [2], we can conclude the composed protocol that replaces the oracle calls with corresponding secure protocols is still secure.

Definition 4

The Hybrid Model (two-party case) [5] In the $(g_1,\ldots, g_m)$-hybrid model, the execution of a protocol π proceeds as in the real-life model, except that the parties have oracle access to a trusted party for evaluating m two-party functions $(g_1,\ldots, g_m).$ These functions are evaluated as in the ideal model. Similarly we define the protocol’s output with the distribution ensemble:

$$ EXEC^{g_1,\ldots, g_m}_{\pi,A} = {\{EXEC^{g_1,\ldots, g_m}_{\pi,A}(k,\overrightarrow{x},C,a)\}}_{k\in{\mathbb{N}},\overrightarrow{x}\in{({\{0,1\}}^\ast)}^4,C\in\{0,1\}, a\in{\{0,1\}}^\ast}. $$

Definition 5

Security in the Hybrid Model (two-party case) [5] Similar to the security definition in the real-life model, the security in the hybrid model can be defined by requiring for any adversary S in the hybrid model, there exists a polynomial-time bounded adversary S such that

$$ IDEAL_{f,S}\stackrel{c}{\approx}EXEC^{g_1,\ldots, g_m}_{\pi,A}. $$

Now we prove our algorithms’ security in the hybrid model by replacing the secure fundamental algorithms PKP, PMC, TTD, SVE, SVB, SXR, SSB, SVZ, and SCA used in our algorithms with the corresponding oracle calls to the trusted party in the hybrid model.

Theorem 1

Given the fundamental algorithms above are secure in the malicious model, Algorithm 2 is secure in the hybrid model.

Proof

Without loss of generality, we assume P ⁰ is corrupted by the adversary A in the hybrid model. For simplicity of presenting, we will use A as a subroutine, as well as the simulators for the secure algorithms in constructing the corresponding adversary in the ideal model.

For any adversary A operating in the hybrid model, given the description of A, the private inputs ${\bf X}^{\bf \prime}_{\bf 1},\ldots, {\bf X}^{\bf \prime}_{\bf n0}$ and the final output $\overline{{\bf W}},$ we construct an adversary S _A in the ideal model as follows:

In step 0.1, S _A runs A to get ${\bf W}^{\bf 0}, \overline{{\bf W}^{\bf 0}}$ and $PKP(\overline{{\bf W}^{\bf 0}})$. S _A runs the S _PKP , the simulator of POK, by giving the state of A and $\overline{{\bf W}^{\bf 0}}$ as the inputs to S _PKP . If S _PKP outputs that the proof fails, S _A terminates the protocol, otherwise, sets the state of A the state returned by S _PKP . S _A simulates the honest party P ¹ by generating random W ¹ and constructing correct zero-knowledge proof. S _A feeds A with the proof. Then, S _A runs A to compute $\overline{{\bf W}} = \overline{{\bf W}^{\bf 0}}+_h\overline{{\bf W}^{{\bf 1}}}{\bf .}$

In step 0.2, S _A runs A to get $\overline{{\bf X}^{\bf \prime}_{\bf k}}$ and $PKP(\overline{{\bf X}^{\bf \prime}_{\bf k}})$ for every ${\bf X}^{\bf \prime}_{\bf k}$ that P ⁰ owns, and feeds S _PKP with state of A and the proof $PKP(\overline{{\bf X}^{\bf \prime}_{\bf k}}).$ If the simulator outputs that the proof fails, S _A terminates the protocol, otherwise, sets the state of A the state returned by the simulator. S _A simulates the honest party P ¹ by generating random ${\bf Y}^{\bf \prime}_{\bf k}$ for all ${\bf Y}^{\bf \prime}_{\bf k}$ that P ¹ owns and constructing correct zero-knowledge proofs. S _A feeds A with these proofs.

In step 0.3, S _A runs A to get $\overline{{\varvec{\Updelta}} {\bf W}}.$

In step 1.1, S _A runs A to update $\overline{{\bf W}}.$

In step 2.1, for any sample ${\bf X}^{\bf \prime}_{\bf k}$ that belongs to P ⁰. S _A runs A to get $\overline{{\bf W}\cdot{\bf X}^{\bf \prime}_{\bf k}}$ and proofs $PMC(\overline{x_{kj}^{\prime}}, \overline{w_j}, \overline{{x^{\prime}_{kj}} \cdot w_j}$) for every $j\in[0,p].$ Then, S _A feeds A’s state and the proofs to S _PMC , the simulator of PMC, sequentially. If any proof fails according to the outputs of the simulator, S _A terminates the protocol, otherwise, sets the state of A to the state returned by the last run of S _PMC . For any sample ${\bf X}^{\bf \prime}_{\bf k}$ that belongs to P ¹, S _A simulates the honest party P ¹ by computing $\overline{{\bf W}\cdot{\bf X}^{\bf \prime}_k},$ generating $PMC(\overline{x_{kj}^{\prime}}, \overline{w_j}, \overline{{x^{\prime}_{kj}} \cdot w_j})$ for every $j\in[0,p]$ correctly and feeds A with these proofs.

In step 2.2, S _A feeds S _SCA , the simulator of SCA, with state of A and $\overline{{\bf W}\cdot{\bf X}^{\bf \prime}_{\bf k}},\,\overline{0}$ as inputs. If the simulator outputs that the computation is incorrect, S _A terminates the protocol, otherwise, sets the state of A the state returned by the simulator.

In step 2.3, if sample X _k is owned by P ⁰, S _A runs A to get $\overline{{\bf X}^{\bf \prime}_{\bf k}\cdot(1-r_k)},$ and the proofs $PMC(\overline{{x^{\prime}_{kj}}}, \overline{1-r_k}, \overline{{x^{\prime}_{kj}} \cdot (1-r_k)})$ for every $j\in[0,p].$ S _A feeds the state of A and $PMC(\overline{{x^{\prime}_{kj}}}, \overline{1-r_k}, \overline{{x^{\prime}_{kj}} \cdot (1-r_k)})$ as the input of S _P MC sequentially for $j\in[0,p],$ if any proofs fails, S _A terminates the protocol, otherwise, sets the state of A the state returned by the last run of the simulator. Then, S _A runs A to update the $\overline{{\varvec{\Updelta} {\bf W}}}.$ If sample X _k is owned by P ¹, S _A simulates honest party P ¹ by constructing the required zero-knowledge proofs and feeds A with these proofs. Then, S _A runs A to update the $\overline{{\varvec{\Updelta}} {\bf W}}.$

Note that, in each step, S _A either runs A directly or uses simulators to simulate the real world by passing the ciphertexts or zero-knowledge proofs as inputs. It is straightforward to see A’s states in the first case are identical in both two worlds. Also in the second case, the ciphertexts and the zero-knowledge proofs are generated using a semantically secure cipher, and the views of A in both worlds are computationally indistinguishable. Therefore, in both cases, the states of A in both worlds are identical in both worlds for every step, and the outputs in both worlds should be computationally indistinguishable.    □

Theorem 2

Given the fundamental sub-algorithms are secure in the malicious model, Algorithm 3 is secure in the hybrid model.

Proof

Without loss of generality, we assume P ⁰ is corrupted by the adversary A in the hybrid model. For simplicity of presenting, we will use A as a subroutine, as well as the simulators for the secure algorithms in constructing the corresponding adversary in the ideal model.

For any adversary A operating in the hybrid model, given the description of A, the private inputs ${\bf X}^{\bf \prime}_{\bf 1},\ldots, {\bf X}^{\bf \prime}_{\bf n}$ and the final output $\overline{{\bf W}},$ we construct an adversary S _A in the ideal model as follows:

In step 0.1, S _A runs A to get ${\bf W}^{\bf 0}, \overline{{\bf W}^{\bf 0}}$ and $PKP(\overline{{\bf W}^{\bf 0}}).$ S _A runs the S _PKP , the simulator of POK, by giving the state of A and $\overline{{\bf W}^{\bf 0}}$ as the inputs to S _PKP . If S _PKP outputs that the proof fails, S _A terminates the protocol, otherwise, sets the state of A the state returned by S _PKP . S _A simulates the honest party P ¹ by generating random W ¹ and constructing correct zero-knowledge proof. S _A feeds A with the proof. Then, S _A runs A to compute $\overline{{\bf W}} = \overline{{\bf W}^{\bf 0}}+_h\overline{{\bf W}^{{\bf 1}}}$.

In step 0.2, S _A runs A to get $\overline{{\bf X}^{\bf \prime}_{\bf k}}$ and $PKP(\overline{{\bf X}^{\bf \prime}_{\bf k}})$ for every ${\bf X}^{\bf \prime}_{\bf k}$ that P ⁰ owns, and feeds S _PKP with state of A and the proof $PKP(\overline{{\bf X}^{\bf \prime}_{\bf k}}).$ If the simulator outputs that the proof fails, S _A terminates the protocol, otherwise, sets the state of A the state returned by the simulator. S _A simulates the honest party P ¹ by generating random ${\bf Y}^{\bf \prime}_{\bf k}$ for all ${\bf Y}^{\bf \prime}_{\bf k}$ that P ¹ owns and constructing correct zero-knowledge proofs. S _A feeds A with these proofs.

In step 0.3, S _A runs A to get $\overline{{\varvec{\Updelta} {\bf W}}}.$

In step 1.1, S _A runs A to update $\overline{{\bf W}}.$

In step 2.1, S _A runs A to get $\overline{{\bf W}^{\bf x}\cdot{\bf X}^{\bf \prime}_{\bf k}}$ and proofs $PMC(\overline{x_{kj}^{\prime}}, \overline{w^x_j}, \overline{{x^{\prime}_{kj}} \cdot w_j^x})$ for every $j\in[0,p].$ Then S _A feeds A’s state and the proofs to S _PMC , the simulator of PMC, sequentially. If any proof fails according to the outputs of the simulator, S _A terminates the protocol, otherwise, sets the state of A to the state returned by the last run of S _PMC . For any sample ${\bf Y}^{\bf \prime}_{\bf k}$ that belongs to P ¹, S _A simulates the honest party P ¹ by computing $\overline{{\bf W}^{\bf y}\cdot{{\bf Y}^{\bf \prime}_{\bf k}}},$ generating $PMC(\overline{y_{kj}^{\prime}}, \overline{w^y_j}, \overline{{y^{\prime}_{kj}} \cdot w_j^y})$ for every $j\in[0,q]$ correctly and feeds A with these proofs.

In step 2.2, S _A runs A to get $\overline{{\bf Z}_{\bf k}^{\bf \prime}\cdot{{\bf W}}}.$

In step 2.3, given the state of A,  S _A feeds S _SCA , the simulator of SCA, with $\overline{{\bf Z}^{\bf \prime}_{\bf k}\cdot{\bf W}},\,\overline{0}$ as inputs. If the simulator outputs that the computation is incorrect, S _A terminates the protocol, otherwise, sets the state of A the state returned by the simulator.

In step 2.4, S _A runs A to get $\overline{{\bf X}^{\bf \prime}_{\bf k}\cdot(1-r_k)},$ and the proofs $PMC(\overline{{x^{\prime}_{kj}}}, \overline{1-r_k}, \overline{{x^{\prime}_{kj}} \cdot (1-r_k)})$ for every $j\in[0,p].$ S _A feeds $PMC(\overline{{x^{\prime}_{kj}}}, \overline{1-r_k}, \overline{{x^{\prime}_{kj}} \cdot (1-r_k)})$ as the input of S _P MC sequentially for $j\in[0,p],$ if any proofs fails, S _A terminates the protocol, otherwise, sets the state of A the state returned by the last run of the simulator. Then, S _A simulates the honest party P ¹ by computing the $\overline{{\bf Y}^{\bf \prime}_{\bf k}\cdot(1-r_k)}$ and generating correct proofs $PMC(\overline{{y^{\prime}_{kj}}}, \overline{1-r_k}, \overline{{y^{\prime}_{kj}} \cdot (1-r_k)})$ for every $j\in[0,q].$ S _A feeds A with these proofs.

In step 2.5, S _A runs A to update the $\overline{{\varvec{\Updelta}} {\bf W}}.$

Similarly, in each step, S _A either runs A directly or uses simulators to simulate the real world by passing the ciphertexts or zero-knowledge proofs as inputs. It is straightforward to see A’s states in the first case are identical in both two worlds. Also in the second case, the ciphertexts and the zero-knowledge proofs are generated using a semantically secure cipher, the views of A in both worlds are computationally indistinguishable. Therefore, in both cases, the states of A in both worlds are identical in both worlds for every step, and the outputs in both worlds should be computationally indistinguishable.    □